Databricks Volumes (especially Unity Catalog (UC) volumes) often have strict execution context requirements and typically expect the workload to run in Databricks-managed clusters or notebooks where the specialized file system and security context are present. Your description suggests a known friction point: container services, which may not fully replicate the Databricks runtime's native context, cause access to Volumes to fail—even if credentialed access to raw S3 works fine.

Are Volumes Intentionally Not Accessible from Container Services?

Yes, this is generally by design as of late 2025. Databricks Volumes and UC volumes are mounted using filesystem drivers and security controls that expect a Databricks-managed execution context. When jobs are run in externalized container services (like Docker/Podman containers spun up outside the Databricks control plane), the Volumes file system layer is often unavailable:

• Native "/Volumes" and Unity Catalog paths depend on Databricks' FUSE/DBFS/VFS overlays.

• These overlays are only present in Databricks-provisioned environments (clusters, serverless compute, or the interactive Databricks notebook context).

• Externalized workloads via Databricks Container Services or custom drivers (like REST API-triggered containers or Kubernetes pods) typically do NOT have direct access to these overlays, leading to permission errors or mount failures.

Official Documentation on Execution Contexts and Volume Access

Databricks documentation does clarify this restriction, but it is often scattered:

• Unity Catalog and Volumes: The official Unity Catalog documentation notes that Volumes are only accessible from Databricks clusters with Unity Catalog enabled, and not from all external interfaces. Only Databricks-interactive or workflow clusters can resolve "/Volumes/" paths.

• DBFS and REST API/Containers: The documentation also notes that paths like "/Volumes", "/mnt" (for legacy DBFS mounts), and related VFS overlays are not available in:

  • Jobs running on external custom Kubernetes clusters using Databricks Container Services.

  • Direct REST API containers, or any compute that runs outside the Databricks cluster control plane.

• Recommended Approach: For workloads that need to access data both inside and outside Databricks, official best practice is to:

  • Use direct access to cloud object storage (e.g., S3 paths) for jobs that may run outside native Databricks compute contexts.

  • Avoid using "/Volumes" or "/mnt" mounts outside of Databricks-managed clusters.

• Permissions: Even with Unity Catalog privileges and correct instance profile configuration, the FUSE driver and security context are missing in container service jobs; hence, access fails.

References

Summary

• Volumes and UC paths are intentionally unavailable in Databricks jobs executed via container services or externalized REST API-launched containers.

• Official documentation for Unity Catalog and data access paths explicitly limits access to Databricks-managed clusters and does not support mounting within containers that lack the Databricks FUSE/VFS environment.

• Direct S3 access remains available everywhere you have credentials, and is the officially recommended approach for hybrid workloads.