- 4521 Views
- 1 replies
- 3 kudos
Best practice of handling Data Duplication in a Delta Live Table Pipeline with daily batch data source.
I am building a data pipeline using Delta Live table in Azure Databricks to move data from a raw data table to a feature table and model inference results table. However, I am concerned about the potential for duplication issues in future operations,...
- 4521 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi @Chengcheng Guo​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 3 kudos
- 11389 Views
- 10 replies
- 4 kudos
Resolved! Databricks SQL restful API to query delta table
As of today, can we use any Databricks SQL Restful API to query Delta Tables stored in ADLS from any external UI?There is some information mentioned over this link https://docs.databricks.com/sql/api/index.html? but not sure how to use them!!Checked ...
- 11389 Views
- 10 replies
- 4 kudos
- 4 kudos
I just saw this today.. https://www.databricks.com/blog/2023/03/07/databricks-sql-statement-execution-api-announcing-public-preview.html
- 4 kudos
- 991 Views
- 1 replies
- 0 kudos
- 991 Views
- 1 replies
- 0 kudos
- 0 kudos
Currently I don;t think this feature is available , however their can be workaround
- 0 kudos
- 947 Views
- 1 replies
- 0 kudos
- 947 Views
- 1 replies
- 0 kudos
- 0 kudos
Not need, we can query tables that are created on CSV and Parquet as well
- 0 kudos
- 949 Views
- 0 replies
- 0 kudos
Can you do conditional upserts using delta?
For instance, replacing an existing record only if it has a newer processed time field.
- 949 Views
- 0 replies
- 0 kudos