Databricks Community

prem14f · 07-30-2024

Hi @dennyglee, @Retired_mod.If I am writing data into a Delta table using delta-rs and a Databricks job, but I lose some transactions, how can I handle this?Given that Databricks runs a commit service and delta-rs uses DynamoDB for transaction logs, ...

prem14f · 07-30-2024

Hi @dennyglee ,If I am writing data into a Delta table using delta-rs and a Databricks job, but I lose some transactions, how can I handle this?Given that Databricks runs a commit service and delta-rs uses DynamoDB for transaction logs, how can we ha...

prem14f · 07-26-2024

Hi, could you please help me here? How can i use this configuration in DataBricks? So I will maintain my transcription logs there, and with Parallel, I can use the Delta-RS job.spark.conf.set("spark.delta.logStore.s3a.impl", "io.delta.storage.S3Dynam...

Databricks Community

User Stats

User Activity

Handling Concurrent Writes to a Delta Table by delta-rs and Databricks Spark Job

Re: Is Concurrent Writes from multiple databricks clusters to same delta table on S3 Supported?

Re: Delta Lake S3 multi-cluster writes - DynamoDB