How to read Databricks UniForm format tables present in ADLS

Demudu
New Contributor II

‎12-25-2024 05:06 AM

We have Databricks UniForm format (iceberg) tables are present in azure data lake storage (ADLS) which has already integrated with Databricks unity catalog. How to read Uniform format tables using Databricks as a query engine?

fmadeiro
Contributor II

‎12-25-2024 06:45 PM

  1. Query Using Unity Catalog:

    • SQL:
      sql
      Copiar código
      SELECT * FROM catalog_name.schema_name.table_name;
    • PySpark:
      python
      Copiar código
      df = spark.sql("SELECT * FROM catalog_name.schema_name.table_name") df.display()

  2. Direct Access by Path: If not using Unity Catalog:

    python
    Copiar código
    iceberg_table_path = "abfss://<container>@<storage_account>.dfs.core.windows.net/<path_to_table>" df = spark.read.format("iceberg").load(iceberg_table_path) df.display()

  3. Optimize Performance:

    • Query by partitions:
      sql
      Copiar código
      SELECT * FROM catalog_name.schema_name.table_name WHERE partition_column = 'value';
    • Cache data:
      python
      Copiar código
      df.cache()

  4. Optional Features:

    • Time Travel:
      sql
      Copiar código
      SELECT * FROM catalog_name.schema_name.table_name VERSION AS OF 123456;

View solution in original post

Demudu
New Contributor II
In response to fmadeiro

‎12-31-2024 07:35 AM

Thank you very much!

 

0 Kudos