Dask-Databricks init script not working

rbauer
New Contributor

Hello everybody ! 

 

I am trying to use the Dask-Databricks distribution (https://github.com/dask-contrib/dask-databricks?tab=readme-ov-file)

i set up the required init-script according to the instructions on the Github page and had no problems there, however the cluster is not able to start always crashing with this error message : failed: Script exit status is non-zero

Unfortunately i don't know the first thing about scripting or init scripts and couldn't find anything to fix this

This is the script i used : 

#!/bin/bash

# Install Dask + Dask Databricks
/databricks/python/bin/pip install --upgrade dask[complete] dask-databricks

# Start Dask cluster components
dask databricks run

 

Thank you for your help !