Databricks Community

mjar · 05-24-2024

Recently we have run into an issue using foreachBatch after upgrading our Databricks cluster on Azure to a runtime version 14 with Spark 3.5 with Shared access mode and Unity catalogue.The issue was manifested by ModuleNotFoundError error being throw...

mjar

Hi, @Nastia unfortunately I don't have any answers yet. I have one channel opened with Databricks though, but no news yet.On plus side (well for us) the workflows still work as expected since the magic fix occurred in our environments.

mjar · 05-27-2024

Below you can find the minimal code to reproduce the scenario. which used to cause the error.Do remember that this suddenly started to work as expected, while it used to fail prior to me posting this topic.In any case, a few words on what we are doi...

mjar · 05-27-2024

Hi @daniel_sahal, thanks for getting back.We are using 14.3, Spark 3.5.0, Scala 2.12

Databricks Community

User Stats

User Activity

ModuleNotFoundError when using foreachBatch on runtime 14 with Unity

Re: ModuleNotFoundError when using foreachBatch on runtime 14 with Unity

Re: ModuleNotFoundError when using foreachBatch on runtime 14 with Unity

Re: ModuleNotFoundError when using foreachBatch on runtime 14 with Unity