How to run a saved query from a Notebook (PySpark)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-29-2023 10:12 AM - edited 12-29-2023 10:17 AM
Hi Team! Noob to Databricks, so apologies if I ask a dumb question.
I have created a relatively large series of queries that fetch and organize the data I want. I'm ready to drive all of these from a Notebook (likely PySpark).
An example query is saved as test1 under my user account (it shows up under the Queries heading in the left navbar). I've also created a Notebook under that same account called Investigations (and both the notebook and query show up under Workspace).
My sample code is simple:
query_name = "test1"
results = spark.sql(f"SELECT * FROM {query_name}")
# Show the results
display(results)
But it produces the TABLE_OR_VIEW_NOT_FOUND error:
AnalysisException: [TABLE_OR_VIEW_NOT_FOUND] The table or view `test1` cannot be found. Verify the spelling and correctness of the schema and catalog. If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and catalog.
I'm sure the issue is just that I'm not properly referencing the query. Help!!