- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Tuesday
@HZ your understanding of Spectrum's Delta reader is correct: it requires manifests. The alternative is Redshift's Iceberg access through Glue catalog federation, which AWS documents for Unity Catalog. These are separate integration paths.
For your Lakeflow tables, the setup to validate is:
1. Enable Compatibility Mode in the table definition.
For streaming tables and materialized views, the Databricks guide specifies these properties at creation time:
TBLPROPERTIES ( 'delta.universalFormat.enabledFormats' = 'compatibility', 'delta.universalFormat.compatibility.location' = 's3://<bucket>/<table-path>' )
The destination must be empty and covered by a registered external location where you have CREATE EXTERNAL TABLE. Compatibility Mode generates a read-only version with Delta and Iceberg metadata; it does not configure Redshift access for you.
2. Federate Unity Catalog into Glue.
Follow the Lake Formation setup. It uses a Databricks service principal for catalog access and an IAM role for access to the underlying S3 data. For this setup, that storage access needs to cover the compatibility data.
3. Connect Redshift to the federated database.
The Redshift guide covers creating a Glue database resource link, granting Lake Formation permissions, and querying through an external schema:
CREATE EXTERNAL SCHEMA uc_lakeflow FROM DATA CATALOG DATABASE 'your_resource_link_database' IAM_ROLE 'arn:aws:iam::<account-id>:role/<redshift-role>';
For this external-schema route, the catalog, Redshift compute and S3 bucket must be in the same Region.