DLT with unity catalog and ML
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-30-2023 06:50 AM
We are currently using DLT with unity catalog. DLT tables are created as materialized views in a schema inside a catalog.
When we try to access these materialized view using a ML runtime (ex. 13.0 ML) cluster, it says, that we must use Single User security mode. However, Single User security mode cannot access materialized views. It throws the error [MATERIALIZED_VIEW_OPRATION_NOT_ALLOWED.REQUIRES_SHARED_COMPUTE].
Is there any way to use DLT with unity catalog and ML all combined? We could create a notebook that copies the DLT materialized views into a Delta table but then there doesn't seem much of a point to using DLT.
Are we using DLT with Unity Catalog incorrectly? Should it only be used for bronze ingest/silver layer transformation and then we use Delta tables for gold layer tables?
- Labels:
-
DLT
-
Unity Catalog