Hi @Dp15 

The error stack trace doesn't point to an error with running Spark SQL inside the SQL UDF but instead with respect to the call to the SparkSession created from inside the UDF; this is not permitted. Furthermore, the UDF runs in a separate execution context that does not have direct access to the main Spark context or session, which is why you see the SystemExit error.

The Spark session should be created outside the UDF, and the queries executed before passing the results to the UDF for further processing. 

Is there a way to run spark sql inside the SQL UDF?

So, you can run sql inside your udf, but you can't initialize the sparksession there.