UDF is more expensive in Spark

That could be the reason for this

Ajay Kumar Pandey