Databricks Community

PearceR · 06-01-2023

Good Morning,I am having some issues with my DLT pipeline. I have a scenario where I am loading in bronze-silver tables programatically from a SQL database (each row corresponds to a table to create). This leaves me in a situation where sometimes onl...

PearceR · 04-21-2023

Hello community :).I am currently implementing some pipelines using DLT. They are working great for my medalion architecture for landed json in bronze -> silver (using apply_changes) then materialized gold views ontop.However, I am attempting to crea...

PearceR

Good luck!

PearceR

In the end we ditched DLT, gives nice advantages ot a quick setup but it couldn't handle the entity scaling. Can do everything dlt can do in the native setup (and workflows are handy) so I would recommend the switch.

PearceR · 07-01-2024

Me and my team have also had the same issue, twice this month. Any update on the cause? We are reviewing reverting to using %run on notebooks to get around it...

PearceR · 01-09-2024

I had to scrap using DLT in the end for so many entities, the other option is to run in continuously but this is expensive.

PearceR · 07-16-2023

I have been having similar problems, any insight into this would be great.

Databricks Community

User Stats

User Activity

try and except in DLT pipelines

custom upsert for delta live tables apply_changes()

Re: Delta Live Tables: Too much time to do the "setting up"

Re: Delta Live Tables: Too much time to do the "setting up"

Re: [Errno 11] resource temporarily unavailable

Re: Delta Live Tables: Too much time to do the "setting up"

Re: Delta Live Tables: Too much time to do the "setting up"