- 822 Views
- 4 replies
- 1 kudos
Resolved! DLT Compute: "Ephemeral" Job Compute vs. All-purpose compute 2.0 ... WHY?
Hi there, this is a follow-up from a discussion I started last monthSolved: Re: DLT Compute: "Ephemeral" Job Compute vs. All-p... - Databricks Community - 71661Based on what was discussed, I understand that it's not possible to use "All Purpose Clust...
- 822 Views
- 4 replies
- 1 kudos
- 1 kudos
@ChristianRRL regarding on why DLT doesn't allow you to use all-purpose clusters: 1. The DLT runtime is derived from the shared compute DBR, it's not the same runtime and has different features than the common all-purpose runtime. A DLT pipeline is n...
- 1 kudos
- 957 Views
- 1 replies
- 0 kudos
Materialized Views Still In Public Preview?
The title is self-explanatory, but I'm curious if anyone has any inclination as to when Materialized Views will enter General Availability? I'm surprised it's not already in GA considering (A) it's been a year or more since Databricks originally anno...
- 957 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, if the documentation states it, it's still in preview.
- 0 kudos
- 4623 Views
- 5 replies
- 3 kudos
Resolved! DLT Job Clusters: Continuous vs Triggered Cluster Start Times
Hi there,I'm curious if anyone is able to definitively help me answer how DLT Job Clusters operate/run.For example, the following is my baseline understanding of DLT Job Clusters. If I run a Triggered DLT Pipeline (e.g. daily) the job cluster takes m...
- 4623 Views
- 5 replies
- 3 kudos
- 3 kudos
Hey there! Thanks a bunch for being part of our awesome community! We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution...
- 3 kudos
- 2247 Views
- 4 replies
- 2 kudos
Purpose of DLT Table table_properties > quality:medallion
Hi there, silly question here but can anyone help me understand what practical purpose does labelling the table_properties with "quality":"<specific_medallion>"? For example: @Dlt.table( comment="Bronze live streaming table for Test data", name="...
- 2247 Views
- 4 replies
- 2 kudos
- 2 kudos
@ChristianRRL, Welcome, and thank you for being a part of our community! We highly value your inquiries, and we applaud your active participation. We kindly invite you to take some time to review the responses provided and select the solution that be...
- 2 kudos
- 3247 Views
- 2 replies
- 1 kudos
DLT Primary Key Deduplication: Expectations vs. Constraints vs. Other?
I'm trying to figure out what's the best way to "de-duplicate" data via DLT. Currently, my only leads are:Manage data quality with Delta Live Tables | Databricks on AWSVia "Drop invalid records"Constraints on Databricks | Databricks on AWSVia "pre-de...
- 3247 Views
- 2 replies
- 1 kudos
- 1 kudos
Hey @ChristianRRL ,Based on my understanding you want to de-duplicate your data during your DLT pipeline processing unfortunately I was not able to find a solution to this when I ran into this problem due to the native feature limitations.Limitations...
- 1 kudos