cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Enable credential passthrough Option is not available in new UI for Job Cluster

Deepak_Kandpal
New Contributor III

Hi All,

I am trying to add new workflow which require to use credential passthrough, but when I am trying to create new Job Cluster from Workflow -> Jobs -> My Job, the option of Enable credential passthrough is not available.

Is there any other way to enable this.

image 

Regards,

Deepak

1 ACCEPTED SOLUTION

Accepted Solutions

Rostislaw
New Contributor III

assuming your Excel file is located on ADLS you can add a service principal to the cluster configuration. see: https://docs.microsoft.com/en-us/azure/databricks/data/data-sources/azure/azure-storage#--access-azu...

With Unity Catalog you could probably also use authorization based on the external path.

View solution in original post

3 REPLIES 3

Debayan
Esteemed Contributor III

Hi Deepak, Thanks for reaching out to community.databricks.com.

Job cluster is not having option to enable credential passthrough.

Deepak_Kandpal
New Contributor III

Hi Debayan,

Thankyou for your response. I am working on a project where it require to read and process Excel file and I am using Maven Spark Excel library to read the excel file, and it is successfully reading on All purpose cluster with credential passthrough option. However now we need to put this notebook into schedule using workflows and due to lack of credential passthrough option for Job Cluster we are getting below error.

Error: Failure to initialize configuration Invalid configuration value detected for fs.azure.account.key

Is there a workaround to run this notebook with Job Cluster? As we want to run all workflows using a service account.

Regards,

Deepak

Rostislaw
New Contributor III

assuming your Excel file is located on ADLS you can add a service principal to the cluster configuration. see: https://docs.microsoft.com/en-us/azure/databricks/data/data-sources/azure/azure-storage#--access-azu...

With Unity Catalog you could probably also use authorization based on the external path.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group