- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-22-2026 02:26 PM
Hi @IM_01,
Good question. It's the terminology (temporary) that is probably causing the confusion.
In Declarative Pipelines, @dp.table(temporary=True, ...) creates a real Delta table on disk, just like a normal table. The only difference is visibility. It is not published to Unity Catalog or the Hive Meta Store. It is accessible only inside that pipeline. It also persists across pipeline runs for the lifetime of the pipeline, not just for a single run/session. Because it’s still a proper Delta table under the hood, all the physical table features apply... including partition_cols. The engine creates an internal table in the pipeline’s internal schema with the same partitioning as for a non‑temporary table. This is also explained in a technical blog here.
In your example, trips_data_quarantine is a physically partitioned Delta table on is_quarantined (so reads from it can prune on that column). This is private to the pipeline (not queryable from outside, not visible in UC) and is cleaned up when the pipeline is deleted.
Partitioning is fully honoured. Temporary here means pipeline‑private, not ephemeral in-memory temp view.
Does this answer your question?
If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***