[TASK_WRITE_FAILED] Task failed while writing rows to abfss

[FrancisApel]

I am trying to insert into an already created delta table in Unity Catalog. I am getting the error:

[TASK_WRITE_FAILED] Task failed while writing rows to abfss://xxxx@xxxxxxxxxxxxxxxx.dfs.core.windows.net/__unitystorage/catalogs/xxxxxxxx-c6c8-45d8-ac3f-7ade50a0bbed/tables/76b5d99e-d355-4808-95d6-xxxxxxxxxxx.

The insert involves joining 17 tables (actually temporary view of tables) to get the data needed to insert. The code is created like follows:

INSERT INTO catalog1.schema1.product

(column1, column2, column3...)

SELECT a.col1, b.col2, c.col1...

FROM table1 a

LEFT JOIN table2 b on a.col1=b.col1

LEFT JOIN table3 C on a.col1=c.col1

...

I tried this with a warehouse cluster and a spark cluster and got the same error.

I also created a simple test table in the same UC catalog/schema and was able to insert into it without any issues (so it's not a security related issue).

Any help in how to further troubleshoot this issue would be greatly appreciated.

[Atanu]

Can you please share the driver log associated with this error. Would like to see the full error thread.

[FrancisApel]

Where would I find the driver log for Serverless SQL Warehouse?

[chemajar]

Hi, Francis.

Did you resolve the issue ?

[NikunjKakadiya]

Any chance this issue got resolved?
I am also seeing the same error when I am trying to incrementally read the system tables using the read stream method and writing it using the writestream method. This generally comes for the audit table but other tables runs fine without any issues.