cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

[TASK_WRITE_FAILED] Task failed while writing rows to abfss

FrancisApel
New Contributor II

I am trying to insert into an already created delta table in Unity Catalog. I am getting the error:

[TASK_WRITE_FAILED] Task failed while writing rows to abfss://xxxx@xxxxxxxxxxxxxxxx.dfs.core.windows.net/__unitystorage/catalogs/xxxxxxxx-c6c8-45d8-ac3f-7ade50a0bbed/tables/76b5d99e-d355-4808-95d6-xxxxxxxxxxx.

The insert involves joining 17 tables (actually temporary view of tables) to get the data needed to insert. The code is created like follows:

 

 
INSERT INTO catalog1.schema1.product
(column1, column2, column3...)
SELECT a.col1, b.col2, c.col1...
FROM table1 a
LEFT JOIN table2 b on a.col1=b.col1
LEFT JOIN table3 C on a.col1=c.col1
...
 
I tried this with a warehouse cluster and a spark cluster and got the same error.
I also created a simple test table in the same UC catalog/schema and was able to insert into it without any issues (so it's not a security related issue).
 
Any help in how to further troubleshoot this issue would be greatly appreciated.
4 REPLIES 4

Atanu
Esteemed Contributor

Can you please share the driver log associated with this error. Would like to see the full error thread.

FrancisApel
New Contributor II

Where would I find the driver log for Serverless SQL Warehouse?

chemajar
New Contributor III

Hi, Francis.

Did you resolve the issue ?

NikunjKakadiya
New Contributor II

Any chance this issue got resolved?
I am also seeing the same error when I am trying to incrementally read the system tables using the read stream method and writing it using the writestream method. This generally comes for the audit table but other tables runs fine without any issues.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group