cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

ChristianRRL
by Contributor III
  • 1005 Views
  • 3 replies
  • 0 kudos

Resolved! DLT Compute: "Ephemeral" Job Compute vs. All-purpose compute 2.0 ... WHY?

Hi there, this is a follow-up from a discussion I started last monthSolved: Re: DLT Compute: "Ephemeral" Job Compute vs. All-p... - Databricks Community - 71661Based on what was discussed, I understand that it's not possible to use "All Purpose Clust...

  • 1005 Views
  • 3 replies
  • 0 kudos
Latest Reply
raphaelblg
Honored Contributor II
  • 0 kudos

@ChristianRRL regarding on why DLT doesn't allow you to use all-purpose clusters: 1. The DLT runtime is derived from the shared compute DBR, it's not the same runtime and has different features than the common all-purpose runtime. A DLT pipeline is n...

  • 0 kudos
2 More Replies
ChristianRRL
by Contributor III
  • 979 Views
  • 1 replies
  • 0 kudos

Materialized Views Still In Public Preview?

The title is self-explanatory, but I'm curious if anyone has any inclination as to when Materialized Views will enter General Availability? I'm surprised it's not already in GA considering (A) it's been a year or more since Databricks originally anno...

ChristianRRL_0-1717507701190.png
  • 979 Views
  • 1 replies
  • 0 kudos
Latest Reply
EugeniaMZ
New Contributor II
  • 0 kudos

Yes, if the documentation states it, it's still in preview.

  • 0 kudos
ChristianRRL
by Contributor III
  • 4901 Views
  • 3 replies
  • 3 kudos

Resolved! DLT Job Clusters: Continuous vs Triggered Cluster Start Times

Hi there,I'm curious if anyone is able to definitively help me answer how DLT Job Clusters operate/run.For example, the following is my baseline understanding of DLT Job Clusters. If I run a Triggered DLT Pipeline (e.g. daily) the job cluster takes m...

  • 4901 Views
  • 3 replies
  • 3 kudos
Latest Reply
melbourne
Contributor
  • 3 kudos

Ideally one would expect clusters used for DLT pipeline to terminate after the pipeline execution has finished. However, while running in `development` environment, you'll notice it doesn't terminate on its own, whereas in `production` it terminates ...

  • 3 kudos
2 More Replies
ChristianRRL
by Contributor III
  • 2320 Views
  • 3 replies
  • 1 kudos

Purpose of DLT Table table_properties > quality:medallion

Hi there, silly question here but can anyone help me understand what practical purpose does labelling the table_properties with "quality":"<specific_medallion>"? For example: @Dlt.table( comment="Bronze live streaming table for Test data", name="...

  • 2320 Views
  • 3 replies
  • 1 kudos
Latest Reply
Priyanka_Biswas
Valued Contributor
  • 1 kudos

Hi @ChristianRRL Thank you for reaching out.  Labeling table properties with "quality": "<specific_medallion>" (like bronze, silver, gold) in DLT serves a practical purpose in a data architecture pattern known as the "Medallion Architecture." This ar...

  • 1 kudos
2 More Replies
ChristianRRL
by Contributor III
  • 3413 Views
  • 2 replies
  • 1 kudos

DLT Primary Key Deduplication: Expectations vs. Constraints vs. Other?

I'm trying to figure out what's the best way to "de-duplicate" data via DLT. Currently, my only leads are:Manage data quality with Delta Live Tables | Databricks on AWSVia "Drop invalid records"Constraints on Databricks | Databricks on AWSVia "pre-de...

Get Started Discussions
Auto Loader
autoloader
Delta Live Table
Delta Live Table Pipeline
dlt
  • 3413 Views
  • 2 replies
  • 1 kudos
Latest Reply
Palash01
Valued Contributor
  • 1 kudos

Hey @ChristianRRL ,Based on my understanding you want to de-duplicate your data during your DLT pipeline processing unfortunately I was not able to find a solution to this when I ran into this problem due to the native feature limitations.Limitations...

  • 1 kudos
1 More Replies
Labels