- 5170 Views
- 1 replies
- 3 kudos
Best practice of handling Data Duplication in a Delta Live Table Pipeline with daily batch data source.
I am building a data pipeline using Delta Live table in Azure Databricks to move data from a raw data table to a feature table and model inference results table. However, I am concerned about the potential for duplication issues in future operations,...
- 5170 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi @Chengcheng Guo​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 3 kudos
- 13005 Views
- 10 replies
- 4 kudos
Resolved! Databricks SQL restful API to query delta table
As of today, can we use any Databricks SQL Restful API to query Delta Tables stored in ADLS from any external UI?There is some information mentioned over this link https://docs.databricks.com/sql/api/index.html? but not sure how to use them!!Checked ...
- 13005 Views
- 10 replies
- 4 kudos
- 4 kudos
I just saw this today.. https://www.databricks.com/blog/2023/03/07/databricks-sql-statement-execution-api-announcing-public-preview.html
- 4 kudos
- 1126 Views
- 1 replies
- 0 kudos
- 1126 Views
- 1 replies
- 0 kudos
- 0 kudos
Currently I don;t think this feature is available , however their can be workaround
- 0 kudos
- 1051 Views
- 1 replies
- 0 kudos
- 1051 Views
- 1 replies
- 0 kudos
- 0 kudos
Not need, we can query tables that are created on CSV and Parquet as well
- 0 kudos
- 1073 Views
- 0 replies
- 0 kudos
Can you do conditional upserts using delta?
For instance, replacing an existing record only if it has a newer processed time field.
- 1073 Views
- 0 replies
- 0 kudos