Why Shared Access Mode for Unity Catalog enabled DLT pipeline?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-17-2024 09:34 AM
Hello all,
I am trying to use an RDD API in a Unity Catalog enabled Delta Live Tables pipeline.
I am getting an error because Unity Catalog enabled DLT can only run on "shared access mode" compute, and RDD APIs are not supported on shared access compute for security reasons.
Is there a reason that Unity Catalog enabled DLT is shared access mode only? Can we expect to see Unity Catalog enabled DLT pipelines that can run on single user compute anytime soon?
Unity Catalog with DLT doc (says shared compute only)
Compute access mode restrictions for Unity Catalog (says no RDD API on shared access mode)
Thank you!
Hayden