How to configure a Job-Compute for Unity Catalog Access? (Q/A)

PabloCSD — Fri, 05 Sep 2025 17:46:19 GMT

If you need to access tables that are in a volume of Unity Catalog (UC), with the following configuration will work:

targets: dev: mode: development default: true workspace: host: https://<workspace>.azuredatabricks.net/ run_as: user_name: <user_email> resources: jobs: etl_process: job_clusters: - job_cluster_key: ${bundle.target}-${bundle.name}-job-cluster new_cluster: autoscale: min_workers: 2 max_workers: 8 spark_version: "14.3.x-scala2.12" driver_node_type_id: "Standard_F16s_v2" node_type_id: "Standard_L4s" data_security_mode: "SINGLE_USER" runtime_engine: "STANDARD" azure_attributes: first_on_demand: 1 availability: SPOT_WITH_FALLBACK_AZURE spot_bid_max_price: 100 spark_conf: spark.databricks.unityCatalog.enabled: "true" spark.sql.catalog.unity: "com.databricks.sql.managedcatalog.UnityCatalogProvider" spark.sql.catalog.spark_catalog: "org.apache.spark.sql.delta.catalog.DeltaCatalog"

Re: How to configure a Job-Compute for Unity Catalog Access? (Q/A)

Khaja_Zaffer — Sat, 06 Sep 2025 10:17:23 GMT

Hello @PabloCSD

Good day!

Are you asking or like what are you expectations?

Additions to this:

You cannot create or register tables (managed or external) with locations pointing to volumes, as this is explicitly not supported—tables must use tabular storage locations instead.

To access data within UC volumes, use path-based methods (e.g., spark.read.load('/Volumes/catalog/schema/volume/path/to/file') or dbutils.fs.ls('/Volumes/...')), assuming the user specified in run_as has the required UC privileges like READ VOLUME or WRITE VOLUME on the volume.

topic How to configure a Job-Compute for Unity Catalog Access? (Q/A) in Data Engineering

How to configure a Job-Compute for Unity Catalog Access? (Q/A)

Re: How to configure a Job-Compute for Unity Catalog Access? (Q/A)