The error is as below. The error is intermittent. eg. - The same code throws the below issue for run 3 but doesn't throws issue for run 4. Then again throws issue for run 5.

An error occurred while calling o1509.getCause. Trace:

py4j.security.Py4JSecurityException: Method public synchronized java.lang.Throwable org.datanucleus.exceptions.NucleusException.getCause() is not whitelisted on class class org.datanucleus.exceptions.NucleusDataStoreException

at py4j.security.WhitelistingPy4JSecurityManager.checkCall(WhitelistingPy4JSecurityManager.java:473)

at py4j.Gateway.invoke(Gateway.java:294)

at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)

at py4j.commands.CallCommand.execute(CallCommand.java:79)

at py4j.GatewayConnection.run(GatewayConnection.java:251)

at java.lang.Thread.run(Thread.java:748)

ErrorMessage written to ADLS Successfully..

org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to alter table.