topic Re: Issue with .dbc in the Advanced Data Engineering course in Databricks Academy in Data Engineering

Issue with .dbc in the Advanced Data Engineering course in Databricks Academy

RiyazAliM — Mon, 04 Jul 2022 05:43:36 GMT

The very first notebook of the dbc notebook which is a setup cell fails.

Re: Issue with .dbc in the Advanced Data Engineering course in Databricks Academy

Prabakar — Thu, 07 Jul 2022 10:40:30 GMT

@Riyaz Ali I checked this and it seems the data source is missing in the storage bucket.

Re: Issue with .dbc in the Advanced Data Engineering course in Databricks Academy

RiyazAliM — Sat, 09 Jul 2022 13:38:16 GMT

Hi @Prabakar Ammeappin !

Thanks for your response. Yes, I have checked the corresponding setup files and determined the same.

The question here is, how can the path to the dataset is unavailable and databricks expects us to practice through these dbc notebooks. They won't work with out the source data, no?

Re: Issue with .dbc in the Advanced Data Engineering course in Databricks Academy

RiyazAliM — Sat, 09 Jul 2022 13:41:31 GMT

Hi @Kaniz Fatma !

Thanks for your response. The response from Prabakar isn't the solution but an affirmative that the DBC notebooks mentioned in the Databricks academy for Advanced Data Engineering aren't working because the blob storage setup in the course is completely missing.

If you could get the blob storage re-instated and get the DBC notebooks working that'll be great. I have DE professional coming up in few days.

Thank you 🙂

Re: Issue with .dbc in the Advanced Data Engineering course in Databricks Academy

Niha1 — Fri, 26 Aug 2022 22:38:51 GMT

Hi Kaniz,

I am also facing the same problem.

Kindly let us know how can we practise from the dbc files which we have.

Regards,

Niharika

Re: Issue with .dbc in the Advanced Data Engineering course in Databricks Academy

RiyazAliM — Sat, 27 Aug 2022 03:14:30 GMT

Hey @Niharika Modi

This issue has been resolved for me.

If the issue persists, kindly add snips of the error

Re: Issue with .dbc in the Advanced Data Engineering course in Databricks Academy

Niha1 — Sat, 27 Aug 2022 08:16:59 GMT

Hi Riyaz,

Please find the snippet of the error below--:

AnalysisException: Path does not exist: dbfs:/user/nniha9188@gmail.com/dbacademy/machine_learning/datasets/airbnb/sf-listings/sf-listings-2019-03-06-clean.parquet"

Source-

The source for this dataset is

wasbs://courseware@dbacademy.blob.core.windows.net/scalable-machine-learning-with-apache-spark/v01/

Your dataset directory is

dbfs:/user/nniha9188@gmail.com/dbacademy/machine_learning/datasets

Skipping install of existing dataset.

I have attached the screenshot also.

Kindy help me.

Regards,

Niharika Modi