NPE on CreateJacksonParser and Databricks 14.3LTS with Spark StructuredStreaming

[joss]

hello,

I have a spark StructuredStreaming job : the source is a kafka topic in json.

it work find with databricks 14.2, but when a change to 14.3LTS, I have a NPE in CreateJacksonParser:

Caused by: NullPointerException: 
	at org.apache.spark.sql.catalyst.json.CreateJacksonParser$.internalRow(CreateJacksonParser.scala:93)

Do you have any idea what this error is?

Regards

[joss]

Hi ,

thank you for your quick reply,

i found the problem : 

 val newSchema = spark.read.json(df.select("data").as[String]).schema

if "data" have 1 value to null, in 14.2  it work, but with 14.3LTS this function return a NPE

I don't know if it is a bug