Newbie DAB question regarding wheels

jimbender
New Contributor II

‎01-20-2025 07:06 AM

I am trying to build a wheel using a DAB.  It errors saying I don't have permissions to install my wheel onto a cluster I am have been given.  Is it possible to just upload the wheel to a subdir the /Shared directory and use it from there instead of installing it to a cluster?  And if so, could I ask for the DAB config to do this?  Thank You!!

0 Kudos

Walter_C
Databricks Employee
Databricks Employee

‎01-20-2025 07:35 AM

May I know the exact error you are getting on the cluster?

You can use the following code to use a wheel in a shared folder:

resources:
  jobs:
    my-job:
      name: my-job
      tasks:
        - task_key: my-task
          new_cluster:
            spark_version: 13.3.x-scala2.12
            node_type_id: i3.xlarge
            data_security_mode: USER_ISOLATION
            num_workers: 1
          python_wheel_task:
            entry_point: main
            package_name: my_package
            libraries:
              - whl: /Shared/path/to/your/wheel-file.whl
0 Kudos