topic "delta-lake" demo fails with"config" notebook not found when running "00-setup" in Data Engineering

"delta-lake" demo fails with"config" notebook not found when running "00-setup"

RoelofvS — Thu, 09 Jan 2025 10:26:05 GMT

Hi all,

The delta-lake demo used to run fine for us around October 2024. Reinstalling it now, it fails on initialisation.

Using runtime version 15.4 on a trial Databricks installation, and executing

dbdemos.install('delta-lake', overwrite=True, use_current_cluster=True)

Frame 2 in 01-Getting-Started-With-Delta-Lake has the line and error:

%run ./_resources/00-setup $reset_all_data=false Notebook not found: Users/[redacted]/Demos/delta-lake/config

and indeed it does not exist. Tracing into the 00-setup notebook, the call is definitely there.

Looking at the subsequent frames in notebook 01-Getting-Started-With-Delta-Lake, their outputs are all listed as "Command skipped". This is different to what is usually seen in the demos. Also, none of the other DE demos that I installed give problems - they install and run fine, and show valid outputs for each frame prior to running them myself.

Easiest way to test/reproduce, is to run the install command listed at the top, and to check whether 01-Getting-Started-With-Delta-Lake frame 2 runs or not.

Re: "delta-lake" demo fails with"config" notebook not found when running "0

brockb — Thu, 09 Jan 2025 18:59:36 GMT

Hi @RoelofvS , I was able to replicate the issue and reached out to the team that maintains dbdemos, they will get this addressed.

Until that is addressed, you can try manually creating that `config` notebook as follows:

#Note: we do not recommend to change the catalog here as it won't impact all the demo resources such as DLT pipeline and Dashboards. #Instead, please re-install the demo with a specific catalog and schema using dbdemos.install("lakehouse-retail-c360", catalog="..", schema="...") catalog = "main__build" schema = dbName = db = "dbdemos_delta_lake" volume_name = "delta_lake_raw_data"

This resolved the issue for me.

Re: "delta-lake" demo fails with"config" notebook not found when running "0

RoelofvS — Sun, 12 Jan 2025 11:36:55 GMT

Thank you @brockb, your workaround also works for me.

I found that notebook 03-Delta-Lake-Uniform frame 4 also needed a touch:

CREATE OR REPLACE TABLE main__build.dbdemos_delta_lake.repro1 ( -- CREATE OR REPLACE TABLE workspace.default.repro1

Thank you again.