Convert SQL Query to Dataframe

[143260]

Hello,

Being relatively new to the Databricks world, I'm hoping someone can show me how to take a SQL query and put the results into a dataframe. As part of data validation project, I'd like to cross join two dataframes.

[Antoine_B]

From a pyspark notebook, you could do:

df = spark.sql("SELECT * FROM my_table WHERE ...")

Then you can use this df and crossjoin it to another DataFrame

If you are new to databricks, I suggest you should follow some of the self paced lessons in databricks-academy, it helped me a lot when I started. For example: https://www.databricks.com/training/catalog/apache-spark-programming-with-databricks-1875