Init script failure after workspace upload

[Aiden-Z]

We have a pipeline in Azure Devops that deploys init scripts to the workspace folder on an Azure Databricks resource using the workspace API (/api/2.0/workspace/import), we use format "AUTO" and overwrite "true" to achieve this. After being uploaded the init script will fail and the cluster will is unable to be started (reason: init script failure). This behavior will persist until someone opens the init script in the workspace (without making any changes) and then the cluster will be able to run the init script normally. 

[Kaniz_Fatma]

Hi @Aiden-Z, Could you please ensure that your Azure DevOps pipeline has the necessary permissions to run the init script on the Azure Databricks cluster?

 If the problem persists, I recommend checking the logs provided by the pipeline run summary page for...

[Aiden-Z]

The azure devops pipeline never runs the script, it only uploads it to the workspace. The script should be run by the cluster on initialization, which after opening it on the workspace file system in the browser, it does on subsequent initialization of the cluster. If the file is uploaded and not opened in the browser, the init script will fail. Sorry, didn't mean to reply at the top level before.

[Aiden-Z]

The init script only generates logs in the scenario that it runs, not when it fails.

[Aiden-Z · Accepted solution]

If anyone else comes across this problem, the issue was a deployment powershell script was changing LF to CRLF before upload in the init script. The solution was to upload with LF line endings in the pipeline.