06-19-2023 01:41 AM
Hi All,
I am unable to execute "Classroom-Setup-02.2" setup in Data Engineering Course. There is the following error: FileNotFoundError: [errno 2] no such file or directory: '/dbfs/mnt/dbacademy-datasets/data-engineer-learning-path/v01/ecommerce/raw/users-historical'
I am using the community edition of Databricks, with ressource's Runtime of 11.3, scala 2.12.
%run ../Includes/Classroom-Setup-02.2
May I know if someone has a solution for it.
Thanks
06-19-2023 09:35 PM
Hi @Mari B
Great to meet you, and thanks for your question!
Let's see if your peers in the community have an answer to your question. Thanks.
06-20-2023 01:24 AM
Hi Vidula,
I'm not sure they can help since it's a Classroom setup problem (Databricks config)... But thanks btw.
06-28-2023 02:39 PM
I had the same problem. There is another post with the same issue. And I got answer there. See below.
Problems with pandas.read_parquet() and path - Databricks - 19655
Wednesday
Inspired by
https://stackoverflow.com/questions/58984925/pandas-missing-read-parquet-function-in-azure-databrick...
I changed
df = pd.read_parquet(path = datasource_path.replace("dbfs:/", '/dbfs/')) # original, error!
df = spark.read.format("parquet").load(datasource_path)
df.to_sql('users', conn, if_exists='replace', index = False)
df.toPandas().to_sql('users', conn, if_exists='replace', index = False)
Excited to expand your horizons with us? Click here to Register and begin your journey to success!
Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!