- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-10-2022 08:22 AM
Hi,
I'm exploring the DLT with AutoLoader feature and wondering where are the schema and checkpoint hide? I want to wipe these two to reset/reinitialize the flow but unlike the "regular" AutoLoader the checkpoint and schema folder are not there.
Thanks in advance.
- Labels:
-
Autoloader
-
DLT
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-11-2022 12:23 PM
@Alexander Plepler , There is a storage option in pipeline settings - A path to a DBFS directory for storing checkpoints and tables created by the pipeline.
Additionally, delta is registered in metastore, so the table schema is there.
My blog: https://databrickster.medium.com/
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-18-2022 06:37 AM
I will. Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-26-2022 04:16 AM
Thanks Fatma,
I still can't find the place where the generated schema stored, but the initial problem is resolved by "Full Refresh" feature of DLT's job. So, I guess I'm ok.
Thanks.
