topic Coursera Apache SQL Course in Get Started Discussions

Coursera Apache SQL Course

nir777 — Tue, 17 Sep 2024 15:33:30 GMT

Hi, I have enrolled into Databricks Apache Spark SQL for Data Analysts course on Coursera.
I am in Module 3, I imported the file through given url in course material. In 3.2Basic Queries I ran the first block which contains following command "%run ../Includes/Classroom-Setup".
Its running well, but when I try to run next cell i.e create table command I am getting error. I am new to all this please guide me to solve the error ASAP. Attaching a screenshot for reference. Thank you!

Re: Coursera Apache SQL Course

filipniziol — Tue, 17 Sep 2024 16:49:27 GMT

Hi @nir777 ,

Run "%fs ls /mnt/training/dataframes/" to see whether you see the file in the folder.
Run "%fs head /mnt/training/dataframes/people-10m.parquet" to see whether you can access the content.

We need to make sure first the file is in the location and also that you have the permissions to access it.

Re: Coursera Apache SQL Course

nir777 — Tue, 17 Sep 2024 17:48:45 GMT

Hi @filipniziol thanks for your answer. I tried the commands but its showing
AccessDeniedException . I don't know how to get permissions.

Re: Coursera Apache SQL Course

filipniziol — Tue, 17 Sep 2024 18:08:28 GMT

Hi @nir777 ,

Have you mounted your s3 bucket correctly?
Could you just run %fs ls /mnt/training and see whether you can see the files or folders?
What is happening:

1. The path is incorrect. So /mnt/training is mounted correctly, but the file is in different folder

2. The file is missing

3. The s3 buckte is mounted incorrectly.

Re: Coursera Apache SQL Course

nir777 — Tue, 17 Sep 2024 18:53:15 GMT

Hi @filipniziol, sharing a screenshot as I am not getting what exactly it means.

Re: Coursera Apache SQL Course

filipniziol — Tue, 17 Sep 2024 19:06:11 GMT

The first line is incorrect, it would need to be %fs ls /mnt/training instead of %python ls /mnt/training.
So you have this mounted:

Are you able to run below code and see the content:

# List the contents of /mnt/training display(dbutils.fs.ls("/mnt/training"))

Re: Coursera Apache SQL Course

filipniziol — Tue, 17 Sep 2024 19:17:52 GMT

If you get AccessDeniedException when trying to list "/mnt/training", then I would:

1. unmount the directory:

dbutils.fs.unmount("/mnt/training")

2. mount the directory once again - check the docs:

access_key = "<aws-access-key>" secret_key = "<aws-secret-key>" encoded_secret_key = secret_key.replace("/", "%2F") aws_bucket_name = "<aws-bucket-name>" mount_name = "<mount-name>" dbutils.fs.mount(f"s3a://{access_key}:{encoded_secret_key}@{aws_bucket_name}", f"/mnt/{mount_name}") display(dbutils.fs.ls(f"/mnt/{mount_name}"))

Re: Coursera Apache SQL Course

OwenChen — Thu, 03 Oct 2024 19:02:16 GMT

Hi, I enrolled in the same coursera "databricks" course, and met the same issue.

I unmounted and re-run the classroom set up as following

However, when I move forward to list the mounted dirctory, the same issue happened.