Unable to query using multi-node clusters but works with serverless warehouse & single-node clusters
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
4 weeks ago - last edited 4 weeks ago
We have a schema with 10tables and currently all 4 users have ALL access. When I (or any other user) spin up a serverless SQL warehouse, I am able to query one of the tables (million rows) in SQL Editor and get a response within seconds. `select count(*) from schema.tbl_name`. When I create a compute cluster with 2nodes, and run the same query in a notebook with the attached compute custer using spark.sql("..."), it just spins and spins and spins and never get a result. Creating a similar cluster with just the driver and not the workers (Single node cluster) works!
Serverless SQL warehouse & Single-node cluster - works in seconds
Multi-node cluster - spins with no result in 60mins

