Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-15-2025 03:48 AM
Hello @mattstyl-ff
Before doing this: try test by dropping the table, delete pysical files as well also,
Clean Any Custom/Residual Paths
paths are :
abfss://storage-dm-int-container@devdomaindmdbxint01.dfs.core.windows.net/_schema8/
/Volumes/dev-data-domain/bronze/test/_checkpoint5
please also monitor the event logs.
Lets dont need to set schemaLocation or checkpointLocation. As DLT automatically manages both under its _dlt_metadata directory.
df = (
session.readStream
.format("cloudFiles")
.option("cloudFiles.format", "csv")
.option("header", "true")
.option("inferSchema", "true")
.load(csv_file_path)
)
query = (
df.writeStream
.format("delta")
.outputMode("append")
.trigger(once=True)
.toTable("`dev-data-domain`.bronze.delta_table_pipeline3")
)
