Thank you for your reply; To summarize, I can either (1) for spark cluster create multiple clusters and/or set spark.databricks.maxConcurrentRuns to increase concurrency per cluster, or (2) for SQL Warehouse set multiple clusters in one SQL Warehouse and/or create more SQL Warehouses.

Maybe I have one more additional question. Is hive_metastore somehow bottleneck during concurrent sql select? When I run some sql query, is there a request to hive_metastore on the 'read path', what could cause the bottleneck? And if yes, when I create more spark clusters do these clusters share one hive_metastore? Or hive_metastore is cached per cluster or even per worker?

And in case of SQL Warehouse, how concurrent sql queries are affected by hive_metastore?

Or can you please direct me to the documentation where these details are given? Thank you very much

M.