Hi @Anish_2,
Looking at your pipeline DAG, the issue is that you have two separate APPLY CHANGES INTO flows both targeting the same silver table (ag_vlc_hist), one from ag_swt_vlchistory_historical and one from ag_swt_vlchistory. When you define mult...
Hi @antgei ,
Thanks for sharing your experience and the workaround. You raise a valid point -- the platform should ensure task files are fully synced before attempting execution, regardless of API rate limiting on the backend. When a job has hundreds...
Hi IM_01,
You can set pipelines.reset.allowed as a table property directly in your pipeline definition. The approach depends on whether you are using Python or SQL:
Python:
@dlt.table(
table_properties={"pipelines.reset.allowed": "true"}
)
def my...
Hi Anish,
There are several things that could explain why a pipeline with fewer tables takes longer to initialize and load. The number of tables alone does not determine pipeline performance — it depends on the complexity of each flow, data volumes, ...
Hi zahid6793,
Thanks for engaging with the thread. To address your question about the void datatype persisting in the Unity Catalog UI after a full refresh: this can happen when the target table metadata in Unity Catalog has not been fully rebuilt. A...
