10-20-2021 02:53 PM
I would definitely go with a Python option here, that way you can use PandasUDFs which are faster than scala UDFs. Happy coding!
