- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-16-2026 05:48 PM - edited 03-16-2026 05:52 PM
Hi Anish,
Yes, @ dlt.view (or @ dp.temporary_view in the newer Lakeflow Declarative Pipelines syntax) will work for your use case. The key difference is that a view does not materialize/persist data as a physical Delta table -- it acts as a temporary, virtual definition within the pipeline. So if your goal is to avoid creating a duplicate bronze table, using @ dlt.view instead of @ dlt.table is the right approach. The view defines the transformation logic without writing a separate copy of the data.
One thing to keep in mind: the partition pruning limitation I described earlier applies at the streaming read level, not at the materialization level. Streaming reads track the Delta transaction log incrementally and do not push down partition filters. That behavior is the same regardless of whether the downstream dataset is defined as a @ dlt.view or @ dlt.table. So the view avoids the duplicate table, but the streaming source will still scan all partitions as new files arrive.
If you need partition-level filtering on the source, you would need a batch read (non-streaming) with explicit filter pushdown, but that changes the incremental processing semantics. For most bronze ingestion patterns, the streaming approach with a view is the right tradeoff.
Sources: