Is there any solution to this? Pandera, Evidently and Ydata Profiling break because they don't speak a sql.connect session object. They expect a spark.sql.session.SparkSession it's very frustrating not being to use any of these libraries with the newest DBR :(.  

This post becomes misleading: https://www.databricks.com/blog/2023/04/03/pandas-profiling-now-supports-apache-spark.html