topic Connect databricks community edition to datalake s3/adls2 in Get Started Discussions

Connect databricks community edition to datalake s3/adls2

harsh_Dev — Mon, 24 Feb 2025 07:48:43 GMT

Can anybody know how can i connect with aws s3 object storage with databricks community edition or can i connect with community databricks account or not ?

Re: Connect databricks community edition to datalake s3/adls2

KaranamS — Mon, 24 Feb 2025 18:47:28 GMT

Hi @harsh_Dev ,

You can read from/write to AWS S3 with Databricks Community edition. As you will not be able to use instance profiles, you will need to configure the AWS credentials manually and access S3 using S3 URI. Try below code

spark._jsc.hadoopConfiguration().set("fs.s3a.access.key", "YOUR_ACCESS_KEY") spark._jsc.hadoopConfiguration().set("fs.s3a.secret.key", "YOUR_SECRET_KEY") df = spark.read.csv("s3a://your-bucket-name/path/to/yourfile.csv") df.show()

Re: Connect databricks community edition to datalake s3/adls2

harsh_Dev — Tue, 25 Feb 2025 05:33:09 GMT

thanks for letting me know @KaranamS