getting alerts in AWS UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.InsideAWS

anurag_dev — Wed, 24 Jan 2024 06:46:49 GMT

Hi folks,
I have been using IAM instance profile as a credentials provider to Databricks workspace. I am getting this error
"Credentials created exclusively for an EC2 instance using instance role <role-arn> have been used from a remote AWS account 414351767826."

Can someone suggest how to tackle it .
I have tried this link .

Re: getting alerts in AWS UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.InsideAWS

Yeshwanth — Wed, 31 Jan 2024 16:02:30 GMT

Hi @anurag_dev

Databricks runs a commit service that coordinates writes to Amazon S3 from multiple clusters. This service runs in the Databricks control plane. (441371677306)

The S3 commit service helps guarantee consistency of writes across multiple clusters on a single table in specific cases. For example, the commit service helps Delta Lake implement ACID transactions.
In the default configuration, Databricks sends temporary AWS credentials from the data plane to the control plane in the commit service API call. Instance profile credentials are valid for six hours.

If you use AWS GuardDuty and you access data using AWS IAM instance profiles, GuardDuty may create alerts for default Databricks behavior related to Delta Lake, Structured Streaming, Auto Loader, or COPY INTO. These alerts are related to instance credential exfiltration detection, which is enabled by default. These alerts include the title UnauthorizedAccess: IAMUser/InstanceCredentialExfiltration.InsideAWS.

You can configure your Databricks deployment to address GuardDuty alerts related to the S3 commit service by creating an AWS instance profile that assumes the role of your original S3 data access IAM role.

As an alternative to using instance profile credentials, this new instance profile can configure clusters to assume a role with short-duration tokens. This capability already exists in all recent Databricks Runtime versions and can be enforced globally via cluster policies.

Please refer to the following documentation for a detailed explanation:- https://docs.databricks.com/administration-guide/cloud-configurations/aws/s3-commit-service.html#aws-guardduty-alerts-related-to-s3-commit-service

topic getting alerts in AWS UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.InsideAWS in Administration & Architecture

getting alerts in AWS UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.InsideAWS

Re: getting alerts in AWS UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.InsideAWS