Databricks Community

shan-databricks · 08-31-2025

How to load all the previous day's data only into the newly added column of the existing delta table? Is there any option available to do that without writing any logic?

shan-databricks · 07-22-2025

I have one file that has 100 rows and in which two rows are bad data and the remaining 98 rows is good data, but when I use the bad records' path, it completely moves the file to the bad records' path, which has good data as well, and it should move ...

shan-databricks · 02-18-2025

Need help to resolve the issue Error : com.databricks.sql.transaction.tahoe.DeltaAnalysisException: [_LEGACY_ERROR_TEMP_DELTA_0007] A schema mismatch detected when writing to the Delta table.I am using the below code and my JSON is dynamically changi...

shan-databricks · 02-13-2025

I have 50 tables and will increase gradually, so I want to create a single workflow to orchestrate the job and run it table-wise. Is there an option to do this in Databricks workflow?

shan-databricks · 07-23-2025

I have already analysed the issue and, yes, the schema doesn't match one of the rows, and it moved the complete file into badRecords and I have seen the behavior and that's fine, and thanks for the response.

shan-databricks · 07-23-2025

I am using the same in my code, but instead of moving only bad data to badRecordsPath, it is moving complete file into badRecordsPath, which has good data as well in the same file.

shan-databricks · 07-22-2025

I am using the Autoloader features spark.readStream and writeStream, and have used the option(badRecordsPath), so when I use either Permissive or Drop malformed or FailFast, I get an exception, like if 'badRecordsPath' is specified mode is not allowe...

shan-databricks · 07-22-2025

I am using the Autoloader features spark.readStream and writeStream, and have used the option(badRecordsPath), so when I use either Permissive or Dropmalformed or FailFast, I get an exception, like if 'badRecordsPath' is specified mode is not allowed...

shan-databricks · 02-16-2025

Thank you, I will try this.

Databricks Community

User Stats

User Activity

How to load all the previous day's data only into the newly added column of the existing delta table

Databricks Autoloader BadRecords path Issue

LEGACY_ERROR_TEMP_DELTA_0007 A schema mismatch detected when writing to the Delta table.

Databricks Workflow Orchestration

Re: Databricks Autoloader BadRecords path Issue

Re: Databricks Autoloader BadRecords path Issue

Re: Databricks Autoloader BadRecords path Issue

Re: Databricks Autoloader BadRecords path Issue

Re: Databricks Workflow Orchestration