How to mount s3 bucket in community edition cluster?

[findinpath]

I'm using Databricks Community Edition for testing purposes on a OSS project.

I'm spinning up the cluster automatically through Databricks Clusters API.

The automated tests rely on AWS S3 infrastructure, reason why I need to mount the S3 bucket on the cluster.

Is there a way to script this operation?

I have found already several references using `dbutils` for doing the mount of the AWS S3 bucket.

However, I'd like to do the mount from outside of a Databricks notebook.

[Anonymous]

There is no way to mount a bucket outside of a notebook. The good thing is you only need to mount a bucket once in a workspace.

[Hemant]

You cannot mount s3​ in community version of Databricks.

[Kaniz]

Hi @Marius Grama​ , Just a friendly follow-up. Do you still need help or do the above responses help you find the solution? Please let us know.

[findinpath]

@Hemant Kumar​  I found out that mounting an S3 bucket is possible if using the Web UI within a Notebook.

https://towardsdatascience.com/how-i-connect-an-s3-bucket-to-a-databricks-notebook-to-do-analytics-a...

@Kaniz Fatma​  yes, I would still need help on this one.

Would it be a viable solution to use a global init script from dbfs for the cluster created via Databricks Clusters API ?

https://docs.databricks.com/clusters/init-scripts.html#cluster-scoped-init-script-locations

[Anonymous]

If you still need the ability to mount from outside a notebook, please contact your CSE to file a feature request.

[Prabakar]

@Marius Grama​ , To mount the S3 bucket please follow the below document. You can mount it only from the notebook and not from the outside.

Please refer to the Databricks official document: mount-an-s3-bucket

to be more clear, in Databricks you can mount S3 using the command "dbutils.fs.mount("s3a://%s" % aws_bucket_name, "/mnt/%s" % mount_name)"

dbutils are not supported outside of notebooks.

[Kaniz]

Hi @Marius Grama​  , Just a friendly follow-up. Do you still need help, or did you find the solution? Please let us know.

[findinpath]

I haven't found any solution.

I'm assuming that currently my only option is the usage of Databricks Enterprise to model scenarios involving the mounting of object storage buckets.

[Kaniz]

Hi @Marius Grama​ , As @Joseph Kambourakis​  replied above, If you still need the ability to mount from outside a notebook, please get in touch with your CSE to file a feature request.