Databricks Academy Course Issue | Apache Spark Programming with Databricks

[ackerman_chris]

I recently have been studying for the Databricks Spark Developer Associate exam through the Databricks Academy and have come across an issue that needs addressed.

The course that needs addressed is the Apache Spark Programming with Databricks 

While walking through the material in my Databricks Workspace the provided Notebook material that Databricks gives you. I attempt to execute the first cell (%run ./Includes/Classroom-Setup-SQL). While digging into where the cell is referencing, it executes the following.

spark.sql("""CREATE TABLE IF NOT EXISTS events USING parquet OPTIONS (path "{}")""".format(eventsPath))

spark.sql("""CREATE TABLE IF NOT EXISTS sales USING parquet OPTIONS (path "{}")""".format(salesPath))

spark.sql("""CREATE TABLE IF NOT EXISTS users USING parquet OPTIONS (path "{}")""".format(usersPath))

spark.sql("""CREATE TABLE IF NOT EXISTS products USING parquet OPTIONS (path "{}")""".format(productsPath))

displayHTML("")

the namePath variables are represented as follows

module_name = "spark-programming"

spark.conf.set("com.databricks.training.module-name", module_name)

salesPath = "/mnt/training/ecommerce/sales/sales.parquet"

usersPath = "/mnt/training/ecommerce/users/users.parquet"

eventsPath = "/mnt/training/ecommerce/events/events.parquet"

productsPath = "/mnt/training/ecommerce/products/products.parquet"

There is a cell that also executes that references a long script in Student-Environment – which I’ve attached to this posting below.

While doing some research on this issue, I found this other posting on the Community here that points this problem out back in April and the solution was to download the same .dbc file only updated in a Github repo.

Well Databricks has now removed that repo stating the following:

The courseware materials for this course are no longer available through GitHub.

To access the current body of courseware, please sign in to Databricks Academy using one of the following three options:

• Customer Academy
• Partner Academy
• Employee Academy

I have submitted a ticket to training-support@databricks.com and have yet to receive a response back from them. The ticket number for anyone’s reference is #00343266. I do not know what the urgency of this is on their end, I just want to make this post on the community in hopes that it can get some traction. I’m fairly new to Databricks and want to continue my studies for the Spark Developer exam, and there is little to no third-party material out there that I can find that preps you for this exam other then the Databricks Academy’s.

I hope that Databricks can remediate this issue quickly as it is the main course for this exam.

[DatBoi]

Did you find a solution to this? I am running into the same problem.

[ackerman_chris]

Hi @DatBoi,

Unfortunately, I have not. I still have yet to hear back from the Training Support on any update from the ticket I submitted. I have spammed that team countless times just to even see if I can get a "We're aware of this and working on it" response with no avail. I have kind of given up at this point. You could probably gain a little more traction on this issue by submitting a ticket yourself and referencing my ticket number maybe. The Link to submit a ticket is mentioned above.

My two working theories are they're either too lazy to respond to such a ticket and they are indeed aware about the problem and working on it (slowly). Or they maybe working on an entirely new course that they're keeping hush about.

I have gone through other resources to try to get a response. My company has a partnership with databricks and we've nudged them about this problem but still no update.

At this point, I've moved on and attempted to study for the exam other ways.

They have a PDF practice test out there you can find that I've found very useful and others I've read said it's extremely similar to the exam. I've used that as a template and also used a few Udemy courses that have some practice tests. I've taken them and really just reviewed the spark documentation to learn more about it.

If you want to be more hands on, I'd recommend experimenting with databricks by importing your own dataset (which as you probably know, is what is failing when running their notebook) and just running PySpark functions against it to learn it better. -- hope some of these suggestions help 🙂

Last note, it really kind of shocks me that a big company like databricks has left this course in the dark. It's the defacto course that they promote for this exam and it's been broken for almost two months now and thte fact they took down the repo that hosted the working-ish .dbc file and just said "Look at the academy" shows horrible communication internally from them. I would've imagined they'd have fixed this by now.

Anyway, hope all of this helps! 😁

[DatBoi]

Thanks for the detailed response. I created a post here which brought me to your post. I will also submit a ticket (doubt it will really do anything lol). I will also take a look at some of the resources you linked. Hopefully this issue gets fixed soon.