cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

gud4eve
by New Contributor III
  • 1449 Views
  • 1 replies
  • 0 kudos

Resolved! Scala app getting NullPointerException while migrating from DBR 7.3 to 9.1 (and above)

We are migrating our Scala jobs from AWS EMR (6.2.1 and Spark version - 3.0.1) to Lakehouse and few of our jobs are failing due to NullPointerException. We tried in Databricks Runtime 7.3 LTS, it is working fine. Because it had same spark version 3.0...

  • 1449 Views
  • 1 replies
  • 0 kudos
Latest Reply
gud4eve
New Contributor III
  • 0 kudos

In one of my code statements, I updated scala Boolean to java.lang.Boolean and this is working fine now. May be in new newer Spark versions, null in scala Boolean isn't supported.

  • 0 kudos
-werners-
by Esteemed Contributor III
  • 1835 Views
  • 3 replies
  • 14 kudos

Notebook fails in job but not in interactive mode

I have this notebook which is scheduled by Data Factory on a daily basis.It works fine, up to today. All of a sudden I keep on getting NullpointerException when writing the data.After some searching online, I disabled AQE. But this does not help.Th...

  • 1835 Views
  • 3 replies
  • 14 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 14 kudos

After some tests it seems that if I run the notebook on an interactive cluster, I only get 80% of load (Ganglia metrics).If I run the same notebook on a job cluster with the same VM types etc (so the only difference is interactive vs job), I get over...

  • 14 kudos
2 More Replies
Labels