topic Re: DLT with UC Access Denied sqs in Data Engineering

DLT with UC Access Denied sqs

185369 — Thu, 04 May 2023 08:58:11 GMT

I am going to use the newly released DLT with UC.

But it keeps getting access denied. As I keep tracking the reasons, it seems that an account. ID other than my account ID or Databricks account ID is being requested.

I cannot use '*' in principal attribute aws in sqs access policy for security reasons.

I want to know how to track another account ID.

Re: DLT with UC Access Denied sqs

Priyag1 — Thu, 04 May 2023 09:27:00 GMT

initialize the SDK with an account set up with the required SQS permissions. You can do so in the IAM console

Re: DLT with UC Access Denied sqs

Priyag1 — Thu, 04 May 2023 09:27:58 GMT

Every service on AWS, an SQS queue, and all the other services in your stack using that queue will be configured with minimal permissions, leading to access issues. So, make sure you get your IAM policies set up correctly before deploying to production

Re: DLT with UC Access Denied sqs

185369 — Mon, 08 May 2023 01:25:27 GMT

Dlt sqs streaming works fine if I don't use the Unity catalog. I don't think it's IAM policies problem

Re: DLT with UC Access Denied sqs

Babu_Krishnan — Thu, 02 May 2024 17:47:11 GMT

@185369 Did you solve this issue ? Could you please share some details ?