topic Enabling serverless type for SQL warehouse running on Google Cloud Platform in Warehousing & Analytics

Enabling serverless type for SQL warehouse running on Google Cloud Platform

96286 — Wed, 25 Oct 2023 14:39:15 GMT

I am in the process of connecting Looker to one of my Databricks databases. To reduce startup time on my SQL warehouse cluster I would like to change the type from "Pro" to "Serverless". I cannot find a way to do that and "Serverless" is not an option for me.

Is Serverless not available when running databricks on GCP? I can't find much information in the docs... Thanks.

Re: Enabling serverless type for SQL warehouse running on Google Cloud Platform

glawry — Mon, 04 Mar 2024 15:54:12 GMT

Are Ephemeral Clusters still a feature that is offered? I do not see any docs on them, and I don't see them as an option in the SQL Warehouse Create UI.

Re: Enabling serverless type for SQL warehouse running on Google Cloud Platform

Kayla — Fri, 15 Mar 2024 18:10:48 GMT

Echoing glawry - I'd be fascinated to know if this "Ephemeral clusters" are a thing.

Re: Enabling serverless type for SQL warehouse running on Google Cloud Platform

MichTalebzadeh — Fri, 15 Mar 2024 22:31:49 GMT

Well currently I believe ephemeral clusters are offered on Google Cloud (GCP) Dataproc (tin boxes). Dataproc is mainly for running Spark and Hadoop jobs on GCP (you can add other artefacts to Dataproc as well like Flink etc) . Dataproc is optimized for launching ephemeral clusters quickly for short-lived jobs. So serverless is not on offer on GCP. However, you can utilize autoscaling clusters to adjust size based on workload. On GCP, BigQuery is serverless.