topic databricks-cli in Machine Learning

databricks-cli

skelchtermans — Tue, 16 Jul 2024 07:19:36 GMT

Hello! I am trying to use the databricks asset bundles through the webui on a databricks compute cluster. However to use this I need the databricks-cli library. I tried to install it on a cluster like described in the documentation using the curl command: Install or update the Databricks CLI | Databricks on AWS

%sh

curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sh

Output: "Installed Databricks CLI v0.223.2 at /root/bin/databricks."

However, when i tried to run databricks -v it says that it doesn't know the command databricks, meaning it didn't install properly. Apparently /root/bin/databricks isn't available in the PATH env variable. To fix this I can just add the location to the PATH variable, but this seems like a convoluted way to get this started.

What I am wondering is, am I not supposed to use the databricks-cli on your own compute clusters? And if it should be compatible with your own platform then is this not a rediculously convoluted way to be able to use your own library on your own platform?

Re: databricks-cli

szymon_dybczak — Tue, 16 Jul 2024 07:53:28 GMT

Hi @skelchtermans ,

There is native support if you use web terminal. But you must first turn it on on workspace. Look at below guide:

https://docs.databricks.com/en/compute/web-terminal.html#cli-workspace

Re: databricks-cli

skelchtermans — Tue, 16 Jul 2024 08:21:10 GMT

I have access to a webterminal on a personal compute with a single user acces mode but when I run "databricks -v" it still states that the command databricks is unknown. Do you mean that integration of databricks-cli needs to be turned on on a workspace level?

Re: databricks-cli

szymon_dybczak — Tue, 16 Jul 2024 09:18:54 GMT

Yeah, to enable it follow below guide:

https://docs.databricks.com/en/admin/clusters/web-terminal.html

One you done it, you should be able to run datbricks command. In below guide the even use assets bundles commands as an example:

https://docs.databricks.com/en/compute/web-terminal.html#cli-workspace

Re: databricks-cli

skelchtermans — Thu, 18 Jul 2024 06:29:17 GMT

Thank you for your help! I read over the part of the runtime of your cluster which has to be 15.0 or more in the documentation you linked. I checked and my compute was still on a LTS 14.3 runtime version, which was the cause.