Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-10-2025 01:20 AM
I have added an extra-index-url in the default package repository in databricks which points to a repository in azure artifact. The libraries from it are getting installed on job cluster but is not working on the all purpose cluster. Below is the relevant config for the all purpose cluster:
{
"cluster_id": "some-id",
"data_security_mode": "USER_ISOLATION",
"effective_spark_version": "16.4.x-scala2.12",
"release_version": "16.4.8",
"autoscale": {
"min_workers": 1,
"max_workers": 15,
"target_workers": 1
},
"init_scripts_safe_mode": false,
"spec": {
"cluster_name": "cluster_name",
"spark_version": "16.4.x-scala2.12",
"autotermination_minutes": 15,
"instance_pool_id": "some-pool",
"driver_instance_pool_id": "some-pool",
"data_security_mode": "USER_ISOLATION",
"autoscale": {
"min_workers": 1,
"max_workers": 15
},
"apply_policy_default_values": false
}
}