Hi Community,

I previously reached out regarding creating shortcuts in Microsoft Fabric for Databricks Delta Live Tables (DLT) managed through Unity Catalog, specifically when the data resides in Azure Data Lake Storage (ADLS) and appears encrypted.

After some experimentation, I’ve found a solution that allows seamless access to these Unity Catalog-managed tables in Microsoft Fabric while ensuring compatibility and security.

Solution for Creating Shortcuts in Microsoft Fabric:

1. Create a Schema in Microsoft Fabric:
   Set up a schema in Fabric that matches the catalog and schema used in Databricks for consistency.

2. Add a New Shortcut in the Schema:

   • Navigate to the newly created schema.
   • Click on "New Shortcut" and select ADLS Gen 2 as the data source.

3. Set Up the Connection:
   Use an existing connection to your ADLS Gen2 storage or create a new one. Ensure you configure the proper credentials and access permissions.

4. Choose Path View:
   While adding the shortcut, select "Path View" instead of "Browse View".

5. Provide Shortcut Details:

   • Specify the shortcut name.
   • Add the ADLS URL and sub-path for the Delta tables.
   • To automate retrieving the sub-paths, use the provided script below

Feel free to reach out if you have further questions or suggestions to enhance this approach!

Code -

def get_delta_table_location(table_catalog: str, table_schema: str, table_name: str) -> str:
    """
    Get the location of a Delta table using its catalog, schema, and table name.

    Parameters:
        table_catalog (str): Catalog name of the table.
        table_schema (str): Schema name of the table.
        table_name (str): Table name.

    Returns:
        str: The location of the Delta table in ADLS.
    """
    full_table_name = f"{table_catalog}.{table_schema}.{table_name}"
    df = spark.sql(f"DESCRIBE TABLE EXTENDED {full_table_name}")
    location_row = df.filter("col_name = 'Location' and data_type LIKE 'abfss://%' ").first()
    
    if not location_row:
        raise Exception(f"Location not found for table: {full_table_name}")
    
    return location_row["data_type"]

# List of tables to process
tables = [
    {"catalog": "your_catalog", "schema": "your_schema", "table": "table_1"},
    {"catalog": "your_catalog", "schema": "your_schema", "table": "table_2"}
]

# Initialize an empty list for storing results
table_paths = []

# Process each table and fetch its location
for table in tables:
    try:
        location = get_delta_table_location(table["catalog"], table["schema"], table["table"])
        # Transform location to a relative path as needed
        relative_path = location.replace("abfss://your_container@your_adls_account.dfs.core.windows.net/", "/your_container/")
        table_paths.append((table["catalog"], table["schema"], table["table"], relative_path))
    except Exception as e:
        table_paths.append((table["catalog"], table["schema"], table["table"], f"Error: {str(e)}"))

# Convert results into a Spark DataFrame for visualization
from pyspark.sql import Row
result_df = spark.createDataFrame(
    [Row(Catalog=row[0], Schema=row[1], Table_Name=row[2], Path=row[3]) for row in table_paths]
)

# Display the results
result_df.display()