topic Re: AWS CLI Commands in Data Engineering

AWS CLI Commands

yatharth — Tue, 13 Feb 2024 12:37:34 GMT

I wish to run aws CLI command in databricks, is there a way i can achieve the same, to be more specific i would like to run:

aws cloudwatch get-metric-statistics --metric-name BucketSizeBytes --namespace AWS/S3 --start-time 2017-03-06T00:00:00Z --end-time 2017-03-07T00:00:00Z --statistics Average --unit Bytes --region us-west-2 --dimensions Name=BucketName,Value=foo-bar Name=StorageType,Value=StandardStorage --period 86400 --output json

Re: AWS CLI Commands

Yeshwanth — Fri, 07 Jun 2024 14:29:38 GMT

@yatharth please check this: https://docs.databricks.com/en/compute/access-mode-limitations.html#network-and-file-system-access-limitations-for-unity-catalog-shared-access-mode:~:text=Cannot%20connect%20to%20the%20instance%20metadata%20service%20(IMDS)%2C%20other%20EC2%20instances%2C%20or%20any%20other%20services%20running%20in%20the%20Databricks%20VPC.%20This%20prevents%20access%20to%20any%20service%20that%20uses%20the%20IMDS%2C%20such%20as%20boto3%20and%20the%20AWS%20CLI.