Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-02-2024 02:02 AM
Cluster libraries are supported from version 15.0 - Databricks Runtime 15.0 | Databricks on AWS.
How can I specify requirements.txt file path in the libraries in a job cluster in my workflow? Can I use relative path? Is it relative from the root of the repository? If absolute, how to specify it?
1 ACCEPTED SOLUTION
Accepted Solutions
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-24-2024 09:57 AM
To use the new "requirements.txt" feature in your cluster do the following:
- Change your cluster's "Databricks Runtime Version" to 15.0 or greater (example: "15.4 LTS ML (includes Apache Spark 3.5.0, Scala 2.12)").
- Navigate to the cluster's: "Libraries" tab -> "Install new" button -> "(o) Workspace".
- Just below the "Workspace File Path" you should now see the text "requirements.txt" added to this prompt: "Choose from workspace or upload a whl/wheelhouse.zip/jar/zip/tar/requirements.txt file to install as a library"
- Navigate to and select the proper Workspace folder or Repo to select the proper "requirements.txt" file to use for your cluster. NOTE: Any "requirements.txt" file within the Workspace folders structure should now show as bold (not grayed out).
- Click the "Install" button.
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-24-2024 09:57 AM
To use the new "requirements.txt" feature in your cluster do the following:
- Change your cluster's "Databricks Runtime Version" to 15.0 or greater (example: "15.4 LTS ML (includes Apache Spark 3.5.0, Scala 2.12)").
- Navigate to the cluster's: "Libraries" tab -> "Install new" button -> "(o) Workspace".
- Just below the "Workspace File Path" you should now see the text "requirements.txt" added to this prompt: "Choose from workspace or upload a whl/wheelhouse.zip/jar/zip/tar/requirements.txt file to install as a library"
- Navigate to and select the proper Workspace folder or Repo to select the proper "requirements.txt" file to use for your cluster. NOTE: Any "requirements.txt" file within the Workspace folders structure should now show as bold (not grayed out).
- Click the "Install" button.

