Runtime 11.3 LTS not working in my production

[biafch]

Hello,

I have a cluster with Runtime 11.3 LTS in my production. Whenever I start this up and try to run my notebooks it's giving me error: Failure starting repl. Try detaching and re-attaching the notebook.

I have a cluster with the same Runtime in my DEV environment and here it's working perfectly fine.

When I downgrade the runtime of my cluster to 10.4 LTS in my production it's working as well.

What can I do to solve this?

[biafch]

Bump

[biafch · Accepted solution]

Just in case anyone needs to know how to solve this in the future.

Apparently one of my clusters was suddenly having library compatibility issues. Mainly between pandas,numpy and pyarrow.

So I fixed this by forcing specific versions in my global init script. I did the following:

/databricks/python/bin/pip install pandas==2.2.2
/databricks/python/bin/pip install numpy==1.26.4
/databricks/python/bin/pip install pyarrow==7.0.0

And this has solved the problem for me.