topic Re: Data bricks issue writing to Azure synapse in Data Engineering

Data bricks issue writing to Azure synapse

DwrCymru — Mon, 28 Nov 2022 11:21:08 GMT

Process is to pickup data from an external table in Azure Synapse and combine with other tables and write to another synapse tables. Data for external table is in Azure storage. It was fine for months ,all of sudden last week it errored out with error

Not able to validate external location because The remote server returned an error: (403) Forbidden. [ErrorCode = 105215] [SQLState = S0001]

Its at the end while writing the new records to synapse. Can someone please hep as all pipelines are falling over while writing to synapse.

Pipelines with dataflows are fine, its just the databricks one while writing to synapse .Its bit urgent. Not really sure what is causing this issue and where would be the fix

Re: Data bricks issue writing to Azure synapse

UmaMahesh1 — Mon, 28 Nov 2022 11:37:06 GMT

Hi @Husnaara Nagri

Looks like an issue to access Azure Storage Account. Are you using Managed Service Identity to access Azure Storage ?

Re: Data bricks issue writing to Azure synapse

UmaMahesh1 — Mon, 28 Nov 2022 11:39:16 GMT

Also do check out this blog from this stack overflow question.

https://stackoverflow.com/questions/66747544/databricks-write-back-to-azure-synapse-error

https://chinnychukwudozie.com/2020/11/13/write-data-from-azure-databricks-to-azure-synapse-analyticsformerly-sql-dw-using-adls-gen-2/

Re: Data bricks issue writing to Azure synapse

DwrCymru — Mon, 28 Nov 2022 11:51:09 GMT

Thanks UmaD,it was working fine for months ,not sure what could trigger a change that would cause failure to write to synapse.Read is fine too.

Re: Data bricks issue writing to Azure synapse

DwrCymru — Mon, 28 Nov 2022 11:51:30 GMT

Have checked this stackoverflow link,seems to be in place