cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Init script failure after workspace upload

Aiden-Z
New Contributor III

We have a pipeline in Azure Devops that deploys init scripts to the workspace folder on an Azure Databricks resource using the workspace API (/api/2.0/workspace/import), we use format "AUTO" and overwrite "true" to achieve this. After being uploaded the init script will fail and the cluster will is unable to be started (reason: init script failure). This behavior will persist until someone opens the init script in the workspace (without making any changes) and then the cluster will be able to run the init script normally. 

1 ACCEPTED SOLUTION

Accepted Solutions

Aiden-Z
New Contributor III

If anyone else comes across this problem, the issue was a deployment powershell script was changing LF to CRLF before upload in the init script. The solution was to upload with LF line endings in the pipeline. 

View solution in original post

4 REPLIES 4

Kaniz_Fatma
Community Manager
Community Manager

Hi @Aiden-Z, Could you please ensure that your Azure DevOps pipeline has the necessary permissions to run the init script on the Azure Databricks cluster?

 If the problem persists, I recommend checking the logs provided by the pipeline run summary page for...

Aiden-Z
New Contributor III

The azure devops pipeline never runs the script, it only uploads it to the workspace. The script should be run by the cluster on initialization, which after opening it on the workspace file system in the browser, it does on subsequent initialization of the cluster. If the file is uploaded and not opened in the browser, the init script will fail. Sorry, didn't mean to reply at the top level before.

Aiden-Z
New Contributor III

The init script only generates logs in the scenario that it runs, not when it fails.

Aiden-Z
New Contributor III

If anyone else comes across this problem, the issue was a deployment powershell script was changing LF to CRLF before upload in the init script. The solution was to upload with LF line endings in the pipeline. 

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!