cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Init script failure after workspace upload

Aiden-Z
New Contributor III

We have a pipeline in Azure Devops that deploys init scripts to the workspace folder on an Azure Databricks resource using the workspace API (/api/2.0/workspace/import), we use format "AUTO" and overwrite "true" to achieve this. After being uploaded the init script will fail and the cluster will is unable to be started (reason: init script failure). This behavior will persist until someone opens the init script in the workspace (without making any changes) and then the cluster will be able to run the init script normally. 

1 ACCEPTED SOLUTION

Accepted Solutions

Aiden-Z
New Contributor III

If anyone else comes across this problem, the issue was a deployment powershell script was changing LF to CRLF before upload in the init script. The solution was to upload with LF line endings in the pipeline. 

View solution in original post

3 REPLIES 3

Aiden-Z
New Contributor III

The azure devops pipeline never runs the script, it only uploads it to the workspace. The script should be run by the cluster on initialization, which after opening it on the workspace file system in the browser, it does on subsequent initialization of the cluster. If the file is uploaded and not opened in the browser, the init script will fail. Sorry, didn't mean to reply at the top level before.

Aiden-Z
New Contributor III

The init script only generates logs in the scenario that it runs, not when it fails.

Aiden-Z
New Contributor III

If anyone else comes across this problem, the issue was a deployment powershell script was changing LF to CRLF before upload in the init script. The solution was to upload with LF line endings in the pipeline. 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group