Dask-Databricks init script not working

rbauer — Wed, 28 Feb 2024 14:32:59 GMT

Hello everybody !

I am trying to use the Dask-Databricks distribution (https://github.com/dask-contrib/dask-databricks?tab=readme-ov-file)

i set up the required init-script according to the instructions on the Github page and had no problems there, however the cluster is not able to start always crashing with this error message : failed: Script exit status is non-zero

Unfortunately i don't know the first thing about scripting or init scripts and couldn't find anything to fix this

This is the script i used :

#!/bin/bash

# Install Dask + Dask Databricks
/databricks/python/bin/pip install --upgrade dask[complete] dask-databricks

# Start Dask cluster components
dask databricks run

Thank you for your help !

topic Dask-Databricks init script not working in Data Engineering

Dask-Databricks init script not working