- 2129 Views
- 1 replies
- 0 kudos
Resolved! Scala app getting NullPointerException while migrating from DBR 7.3 to 9.1 (and above)
We are migrating our Scala jobs from AWS EMR (6.2.1 and Spark version - 3.0.1) to Lakehouse and few of our jobs are failing due to NullPointerException. We tried in Databricks Runtime 7.3 LTS, it is working fine. Because it had same spark version 3.0...
- 2129 Views
- 1 replies
- 0 kudos
- 0 kudos
In one of my code statements, I updated scala Boolean to java.lang.Boolean and this is working fine now. May be in new newer Spark versions, null in scala Boolean isn't supported.
- 0 kudos
- 2513 Views
- 3 replies
- 14 kudos
Notebook fails in job but not in interactive mode
I have this notebook which is scheduled by Data Factory on a daily basis.It works fine, up to today. All of a sudden I keep on getting NullpointerException when writing the data.After some searching online, I disabled AQE. But this does not help.Th...
- 2513 Views
- 3 replies
- 14 kudos
- 14 kudos
After some tests it seems that if I run the notebook on an interactive cluster, I only get 80% of load (Ganglia metrics).If I run the same notebook on a job cluster with the same VM types etc (so the only difference is interactive vs job), I get over...
- 14 kudos