Hey Everyone,

I've built a very simple pipeline with a single DLT using auto ingest, and it works, provided I don't specify the output location. When I build the same pipeline but set UC as the output location, it fails when setting up S3 notifications, which is entirely bizarre. I've looked at the logs on the DBZ side and request logs in AWS and it looks like DBZ isn't using the instance profile I've set for some reason. Further details below, any help would be greatly appreciated!

Context

• Databricks on AWS
• Deployed 1 week ago so uses all the latest features (unity catalog metastore is the default)

Things I've done

• The instance profile is set in the pipeline settings and it appears in both clusters in the JSON settings
• The same instance profile is used when setting up the pipeline without unity and it correctly creates the SNS/SQS resources without issue, so it's not a permissions thing on the role
• The cluster access settings is set to "Shared" 

Things I've tried

• I set up a security credential in the target unity catalog (by copying the working instance profile) for the bucket but that didn't change anything (and it's my understanding this is only for accessing data, not used for setting up File Notification settings)
• I gave the unity IAM role full access to S3, no difference
• I rebuilt the pipeline, no effect