Azure Databricks - Exporting data frame to external volume

[lion_king_84]

Hello,

I am reading a Delta table and exporting it to an external volume. The Unity Catalog external volume points to an Azure Data Lake Storage container.

When I run the code below, I encounter the error message shown below. (When I export the data to a managed volume, the operation completes successfully.)

Could you please help?

Converting to Pandas...
Creating Excel in memory...
Writing to: /Volumes/dev_catalog/silver_schema/external_volume1/outputfolder/competitor_data.xlsx
❌ Error writing to volume: An error occurred while calling o499.cp.
: com.databricks.sql.managedcatalog.acl.UnauthorizedAccessException: PERMISSION_DENIED: Request for user delegation key is not authorized. Details: None
	at com.databricks.sql.managedcatalog.client.ErrorDetailsHandlerImpl.wrapServiceException(ErrorDetailsHandler.scala:119)
	at com.databricks.sql.managedcatalog.client.ErrorDetailsHandlerImpl.wrapServiceException$(ErrorDetailsHandler.scala:88)

!pip install openpyxl

%restart_python

df = spark.read.table('dev_catalog.silver_schema.silver_table')

# For Excel files:
def save_as_excel_to_external_volume(df, volume_path, filename="data.xlsx", sheet_name="Sheet1"):
    """Save DataFrame as Excel using dbutils.fs"""
    import pandas as pd
    from io import BytesIO
    import base64
    
    volume_path = volume_path.rstrip('/')
    full_path = f"{volume_path}/{filename}"
    
    print("Converting to Pandas...")
    pandas_df = df.toPandas()
    
    print("Creating Excel in memory...")
    excel_buffer = BytesIO()
    pandas_df.to_excel(excel_buffer, index=False, sheet_name=sheet_name, engine='openpyxl')
    excel_bytes = excel_buffer.getvalue()
    
    print(f"Writing to: {full_path}")
    try:
        # For binary files, write to temp then copy
        temp_path = f"/tmp/{filename}"
        with open(temp_path, 'wb') as f:
            f.write(excel_bytes)
        
        # Copy from temp to volume using dbutils
        dbutils.fs.cp(f"file:{temp_path}", full_path)
        
        # Clean up temp
        dbutils.fs.rm(f"file:{temp_path}")
        
        print(f"✓ Successfully saved to {full_path}")
        return full_path
    except Exception as e:
        print(f"❌ Error writing to volume: {e}")
        raise
        

volume_path = "/Volumes/dev_catalog/silver_schema/external_volume1/outputfolder/"

save_as_excel_to_external_volume(df, volume_path, "competitor_data.xlsx", "CompetitorData")

 Thanks

[nayan_wylde · Accepted solution]

For UC external volumes on ADLS Gen2, Databricks often needs to generate a user delegation SAS to access the storage path from compute securely. Generating that SAS requires first calling Get User Delegation Key.
If the Access Connector / Managed Identity / Service Principal only has container-level “Storage Blob Data Reader/Contributor” permissions, it may still fail because Get User Delegation Key typically requires permissions at the storage account scope (not only container scope).

1) Identify the identity used by the External Location.

In Unity Catalog, your external location points to a storage credential. That storage credential is backed by an Access Connector for Azure Databricks (managed identity) or a service principal.

You need to grant Azure Storage permissions to that identity.

2) Grant both:

Storage Blob Data Contributor (or Owner) — for read/write/list on blobs
Storage Blob Delegator — to allow generating user delegation keys (this is the usual missing one)

Scope matters: assign at the Storage Account level (or higher). Container-level role assignments frequently aren’t enough for delegation keys.
Recommended RBAC:
At Storage Account scope: Storage Blob Delegator
At Container (or filesystem) scope: Storage Blob Data Contributor (or narrower if your org requires)

[saurabh18cs]

Hi @lion_king_84 are we missing write volume permissions on this schema in UC?