topic Re: Suddenly can't find the option to uplaod files into Databricks Community Edition in Data Engineering

Suddenly can't find the option to uplaod files into Databricks Community Edition

Younevano — Wed, 16 Oct 2024 20:45:27 GMT

Hi everyone,

I am suddenly unable to find the option to upload my files into Databricks Community Edition today. Please find the same in the screenshot attached. Is anyone else also facing this issue?

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

Vamsi_B — Wed, 16 Oct 2024 21:36:05 GMT

I was about to post the same question, for some reason we are not able to find the dbfs toggle option in settings either!

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

Alexisqc92 — Thu, 17 Oct 2024 01:21:11 GMT

I'm facing the same issue, were you able to find a solution for this? or a work around?

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

gchandra — Thu, 17 Oct 2024 02:15:28 GMT

The workaround will be to Mount the S3 bucket and use that as your source.

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

gchandra — Thu, 17 Oct 2024 02:18:43 GMT

access_key = ""

secret_key = ""

encoded_secret_key = secret_key.replace("/", "%2F")

aws_bucket_name = "yourawsbucketname/"

mount_name = "youraliasmountname"

# #dbutils.fs.unmountf"/mnt/{mount_name}")

dbutils.fs.mount(f"s3a://{access_key}:{encoded_secret_key}@{aws_bucket_name}", f"/mnt/{mount_name}")

After mounting
goto next cell %fs ls /mnt/youraliasmountname

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

thorffin — Thu, 17 Oct 2024 15:01:48 GMT

Anyone found any solution please post here

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

nynaevealmeera — Thu, 17 Oct 2024 23:01:25 GMT

What about the IAM config and all that? Is the dbfs file upload permanently disabled for the community edition?

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

mahdi — Thu, 17 Oct 2024 23:16:37 GMT

Perfect time for this to happen -- right in the middle of a class assignment due in a few days. 🙂
The easiest way to do it is by using dbutils.fs.mv to mv the file from the local system to the dbfs. Here is a complete example:

## create file in the local fs
with open("./test_file.txt", "w") as file:
lines = "\n".join(map(str, range(5)))
file.write(lines)

!cat test_file.txt
>> 0 1 2 3 4

#mv file from local fs to dbf
dbutils.fs.mv("file:/databricks/driver/test_file.txt", "dbfs:/test_file.txt")
>> Out[10]: True

# Read the file as an RDD
file_path = "dbfs:/test_file.txt"
rdd = sc.textFile(file_path)
rdd.collect()
>> Out[11]: ['0', '1', '2', '3', '4']

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

SandraFernando — Fri, 18 Oct 2024 21:06:37 GMT

Many thanks for this gchandra.

Will Databricks ever bring back the upload files option into Databricks Community Edition near future? I never mounted an S3 bucket, so will have to test, teach all my students about S3 buckets, not feasible in the middle of the course. Are you able to provide easy steps to mount S3 buckets without incurring any charges please?
Thank you Sandra

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

gchandra — Fri, 18 Oct 2024 21:13:19 GMT

CE product roadmap is not available, so I cannot comment on that.

I have provided 2 alternate options. AWS offers Free Tier https://aws.amazon.com/free/ up to 1 year and students can benefit from that.

https://community.databricks.com/t5/data-engineering/databricks-community-edition-dbfs-alternative-solutions/td-p/94933

Let me know if you have any further questions.

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

gchandra — Sat, 19 Oct 2024 01:02:36 GMT

It's fixed. You can continue to use Upload.

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

gchandra — Sat, 19 Oct 2024 01:03:10 GMT

It's fixed. You can continue to use Upload.

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

SandraFernando — Sat, 19 Oct 2024 14:49:42 GMT

Thanks gchandra, I hope it is going to stay therefor a long time? thanks Sandra

Re: Suddenly can't find the option to uplaod files into Databricks Community Edition

geraldhopkins — Sat, 28 Dec 2024 22:54:30 GMT

GChandra, did it get removed again? I just signed up for Community free edition, and I don't see an option to Upload files. Please advise.