Re: How to mount s3 bucket in community edition cl...

findinpath · ‎04-26-2022

I'm using Databricks Community Edition for testing purposes on a OSS project.

I'm spinning up the cluster automatically through Databricks Clusters API.

The automated tests rely on AWS S3 infrastructure, reason why I need to mount the S3 bucket on the cluster.

Is there a way to script this operation?

I have found already several references using `dbutils` for doing the mount of the AWS S3 bucket.

However, I'd like to do the mount from outside of a Databricks notebook.

Anonymous · ‎04-27-2022

There is no way to mount a bucket outside of a notebook. The good thing is you only need to mount a bucket once in a workspace.

Hemant · ‎04-27-2022

You cannot mount s3 in community version of Databricks.

Hemant Soni

findinpath · ‎04-28-2022

@Hemant Kumar I found out that mounting an S3 bucket is possible if using the Web UI within a Notebook.

@Kaniz Fatma yes, I would still need help on this one.

Would it be a viable solution to use a global init script from dbfs for the cluster created via Databricks Clusters API ?

Anonymous · ‎04-28-2022

If you still need the ability to mount from outside a notebook, please contact your CSE to file a feature request.

Prabakar · ‎05-08-2022

@Marius Grama , To mount the S3 bucket please follow the below document. You can mount it only from the notebook and not from the outside.

Please refer to the Databricks official document: mount-an-s3-bucket

to be more clear, in Databricks you can mount S3 using the command "dbutils.fs.mount("s3a://%s" % aws_bucket_name, "/mnt/%s" % mount_name)"

dbutils are not supported outside of notebooks.

findinpath · ‎05-09-2022

I haven't found any solution.

I'm assuming that currently my only option is the usage of Databricks Enterprise to model scenarios involving the mounting of object storage buckets.

How to mount s3 bucket in community edition cluster?