Databricks

palzor · 08-14-2022

I am loading avro files into the detla tables. I am doing this for multiple tables and some files are big like (2-3GB) and most of them are small like in few MBs.I am using autoloader to load the data into the delta tables.My question is:What is the ...

palzor · 04-20-2022

Hi,I am trying to use CDC for delta live table, and when when I run the pipeline second time I get an error :org.apache.spark.sql.streaming.StreamingQueryException: Query tbl_cdc [id = ***-xx-xx-bf7e-6cb8b0deb690, runId = ***-xxxx-4031-ba74-b4b22be05...

palzor · 04-25-2022

@Hubert Dudek , thanks for your answer, well we have files that we are loading and when we are running the pipeline for the new file that comes in then we get this error. So I think its an append rather than an update.

Databricks

User Stats

User Activity

What is the best practice while loading delta table , do I infer the schema or provide the schema?

Getting error when using CDC in delta live table

Re: Getting error when using CDC in delta live table