Streamsets to Databricks Integration failure

Subbu_G
Databricks Partner

‎01-30-2025 06:01 PM

Hi Team,

While trying to ingest data from ADLS gen 2 to Databricks through Streamsets. I am getting below error

Configuration fs.azure.account.key.xxx.dfs.core.windows.net is not available.

Able to make connection from Streamsets to databricks using sql warehouse JDBC connection. 

 

0 Kudos

Alberto_Umana
Databricks Employee
Databricks Employee

‎02-01-2025 06:29 PM

Hello @Subbu_G,

Ensure that the Spark configuration for the ADLS Gen 2 account key is correctly set. The configuration should follow the format:

spark.hadoop.fs.azure.account.key.<storage-account-name>.dfs.core.windows.net <your-storage-account-key>

0 Kudos