Databricks Community

Red1 · 12-30-2023

Hey Everyone,I've built a very simple pipeline with a single DLT using auto ingest, and it works, provided I don't specify the output location. When I build the same pipeline but set UC as the output location, it fails when setting up S3 notification...

Red1 · 05-02-2024

Hey @Babu_Krishnan I was! I had to reach out to my Databricks support engineer directly and the resolution was to add "cloudfiles.awsAccessKey" and "cloudfiles.awsSecretKey" to the params as in the screenshot below (apologies, i don't know why the sc...

Red1 · 01-02-2024

Thanks @Retired_mod, UC can connect to the S3 bucket and read the data but it fails when trying to set up the bucket notifications.I'll raise a ticket with support and post back here if I find a resolution.

Red1 · 01-01-2024

Hey @Retired_mod ,Thanks for the response!I tried the above with no luck unfortunately:- I don't have an apply_merge function in my pipeline definition, please find the pipe definition below- I'm running DBR 14.2- I don't think databricks connect app...

Databricks Community

User Stats

User Activity

Autoingest not working with Unity Catalog in DLT pipeline

Re: Autoingest not working with Unity Catalog in DLT pipeline

Re: Autoingest not working with Unity Catalog in DLT pipeline

Re: Autoingest not working with Unity Catalog in DLT pipeline