Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-31-2024 07:31 AM - edited 07-31-2024 07:40 AM
From a pyspark notebook, you could do:
df = spark.sql("SELECT * FROM my_table WHERE ...")
Then you can use this df and crossjoin it to another DataFrame
If you are new to databricks, I suggest you should follow some of the self paced lessons in databricks-academy, it helped me a lot when I started. For example: https://www.databricks.com/training/catalog/apache-spark-programming-with-databricks-1875