topic Re: requirements.txt with cluster libraries in Get Started Discussions

requirements.txt with cluster libraries

sujan1 — Thu, 02 May 2024 09:02:00 GMT

Cluster libraries are supported from version 15.0 - Databricks Runtime 15.0 | Databricks on AWS.

How can I specify requirements.txt file path in the libraries in a job cluster in my workflow? Can I use relative path? Is it relative from the root of the repository? If absolute, how to specify it?

Re: requirements.txt with cluster libraries

462098 — Thu, 24 Oct 2024 16:57:25 GMT

To use the new "requirements.txt" feature in your cluster do the following:

Change your cluster's "Databricks Runtime Version" to 15.0 or greater (example: "15.4 LTS ML (includes Apache Spark 3.5.0, Scala 2.12)").
Navigate to the cluster's: "Libraries" tab -> "Install new" button -> "(o) Workspace".
Just below the "Workspace File Path" you should now see the text "requirements.txt" added to this prompt: "Choose from workspace or upload a whl/wheelhouse.zip/jar/zip/tar/requirements.txt file to install as a library"
Navigate to and select the proper Workspace folder or Repo to select the proper "requirements.txt" file to use for your cluster. NOTE: Any "requirements.txt" file within the Workspace folders structure should now show as bold (not grayed out).
Click the "Install" button.

Re: requirements.txt with cluster libraries

mishravk7250 — Wed, 23 Apr 2025 09:47:22 GMT

how to install requirement.txt using github action.

- name: Install workspace requirements.txt on cluster

env:

CLUSTER_ID: ${{ secrets.DATABRICKS_CLUSTER_ID }}

run: |

databricks libraries install \

--cluster-id "$CLUSTER_ID" \

--whl "dbfs:/FileStore/engineering_workflows/requirements.txt"
But this does not work