DLT piplines with UC
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-21-2024 08:47 AM
I try to run a (very simple) DLT pipeline in with a resulting materialized table is published in UC schema with a managed storage location defined (within an existing EXTERNAL LOCATION). Accoding to the documentation:
Publishing to schemas that specify a managed storage location is supported only in the preview channel.
So - with no surprises - running this pipeline in the current channel ends with:
com.databricks.pipelines.common.CustomException: [DLT ERROR CODE: EXECUTION_SERVICE_STARTUP_FAILURE]
Schemas with specified storage locations are not currently supported for UC enabled pipelines.But when I switch to the preview channel, it does not work either, but this time the error is different:
com.databricks.pipelines.common.CustomException: [DLT ERROR CODE: EXECUTION_SERVICE_STARTUP_FAILURE]
Schema tstexternal has a specified storage location that does not match the pipeline's root location: NoneIs there a chance for it to make it working? Can I setup "the pipeline's root location" somehow?
Any help welcome.
Thx
Pawel