gchandra
Databricks Employee
Databricks Employee

The workaround will be to Mount the S3 bucket and use that as your source.

 

access_key = ""
secret_key = ""
encoded_secret_key = secret_key.replace("/""%2F")

 

aws_bucket_name = "yourawsbucketname/"
mount_name = "youraliasmountname"

 

# #dbutils.fs.unmountf"/mnt/{mount_name}")
dbutils.fs.mount(f"s3a://{access_key}:{encoded_secret_key}@{aws_bucket_name}"f"/mnt/{mount_name}")

After mounting
goto next cell %fs ls /mnt/youraliasmountname


~