Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-01-2024 11:59 PM
Hey @Retired_mod ,
Thanks for the response!
I tried the above with no luck unfortunately:
- I don't have an apply_merge function in my pipeline definition, please find the pipe definition below
- I'm running DBR 14.2
- I don't think databricks connect applies here as this was all set up in the databricks UI
- Thanks for the link to that one, I read it a couple times and I've implemented all the recommendations with no luck.
DLT definition:
CREATE OR REFRESH STREAMING LIVE TABLE raw_testing
AS SELECT *
FROM cloud_files(
"s3://bucket-path",
"csv",
map(
"header", "true",
"sep", "|",
"cloudFiles.useNotifications", "true",
"inferSchema", "true"
)
);
This pipeline works as expected when using HMR as the output location but doesn't work with UC.
Any other thoughts? Is there some way i can escalate this? At this point it feels like a bug.