How to use DLT Expectations for uniqueness checks on a dataset?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-15-2024 04:58 PM
I am using dlt through python to build a DLT pipeline. One of things I would like to do is to check that each incoming row does not exist in the target table; i want to be sure that each row is unique.
I am confused because it seems like this is not possible through dlt expect. I am under the impression that each expectation works on row-level, not on a table-level. Morever I am not seeing anything about uniqueness constraints and am reading that PKs are not enforced. So, is this possible?
#dlt #expect
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-19-2024 06:55 AM
I also have the same doubt, about the implementation of the uniqueness rule