Databricks

tw1 · 02-15-2023

Current Cluster Config:Standard_DS3_v2 (14GB, 4 Cores) 2-6 workersStandard_DS3_v2 (14GB, 4Cores) for driverRuntime: 10.4x-scala2.12We want to overwrite a temporary delta table with new records. The records will be load by another delta table and tran...

tw1 · 02-26-2023

Hi,thank you for your help!We tested the configuration settings and it runs without any errors.Could you give us some more information, where we can find some documentation about such settings. We searched hours to fix our problem. So we contacted th...

tw1 · 02-21-2023

Hi,1. we changed our large string column to an array column2. we use a bigger instance for the driverSo we can handle 500 files in our notebook transformation.This runs the most time, but sporadically the same error occurs.I could't change our UDF to...

tw1 · 02-17-2023

Thank you for your answer. I will test it and give you a feedback.

tw1 · 02-15-2023

Hi, we use currently 250 files in one notebook, with approximately 8MB per file. (But normally we want to run 500.000 files)

Databricks

User Stats

User Activity

Can't write / overwrite delta table with error: oxxxx.saveAsTable. (Driver Error: OutOfMemory)

Re: Can't write / overwrite delta table with error: oxxxx.saveAsTable. (Driver Error: OutOfMemory)

Re: Can't write / overwrite delta table with error: oxxxx.saveAsTable. (Driver Error: OutOfMemory)

Re: Can't write / overwrite delta table with error: oxxxx.saveAsTable. (Driver Error: OutOfMemory)

Re: Can't write / overwrite delta table with error: oxxxx.saveAsTable. (Driver Error: OutOfMemory)