Starting Serverless sql cluster on GCP

[ashraf1395]

Hello there,
I am trying to start a serverless databricks SQL cluster in GCP. I am following this databricks doc: https://docs.gcp.databricks.com/en/admin/sql/serverless.html
I have checked that all my requirements are fulfilled for activating the cluster

I am following the steps as mentioned

1. Go to the feature enablement tab of the account console settings page.

2. A banner at the top of the page prompts you to accept the additional terms. Once you read the terms, click Accept. If you do not see the banner asking you to accept the terms, this step has been completed already.

3. Go to the settings page.

4. Select Compute.

5. Click Manage next to SQL warehouses.

6. Turn on Serverless SQL warehouses (On).

I am not able to find any button or thing to turn on the serverless SQL warehouse, what am I doing wrong?

 

 

[Kaniz · Accepted solution]

Hi @ashraf1395, It seems you’re encountering some confusion while trying to enable the serverless SQL cluster in Databricks on Google Cloud Platform (GCP).

Let’s troubleshoot this together!

First, I appreciate that you’ve followed the steps outlined in the documentation. However, there might be a couple of reasons why you’re not seeing the option to turn on the SQL Serverless warehouse:

1. Compatibility with GCP:

   • Unfortunately, as of now, serverless clusters are not directly available on GCP. Databricks primarily offers ephemeral clusters on GCP Dataproc, which are mainly optimized for runni...¹.
   • If you’re specifically looking for serverless compute, you might need to explore other options or consider using a different cloud provider.

2. Documentation Mismatch:

   • The documentation you’re following might not be up-to-date or might not apply to GCP.
   • I recommend checking the official Databricks documentation specifically for GCP to ensure you’re fol...².

3. Compute Configuration Best Practices:

   • Databricks recommends using both serverless compute and compute policies to simplify configuration decisions.
   • Serverless SQL warehouses require minimal configuration and provide always-available and scalable co...³.

In summary, if you’re specifically looking for serverless compute, it might not be directly available on GCP. However, you can explore other options or consider using a different cloud provider. 

Remember that technology evolves, and sometimes the information we find might not be the most current. Keep exploring and adapting, and you’ll find the right solution! 😊⁴²¹³

[ashraf1395 · Accepted solution]

Thanks @Kaniz, I will look into ephemeral clusters.
Can you share the link for the official databricks on gcp documentation? I didn't find it in the post link you shared.

[Kaniz · Accepted solution]

Hi @ashraf1395, Here are the official Databricks documentation links specifically for the Google Cloud Platform (GCP):

1. Databricks on Google Cloud Documentation:

   • This documentation provides guidance, how-to information, and reference details for using Databricks on GCP. It covers topics related to data science, engineering, machine learning, and SQL environments.
   • You can find it here: Databricks on Google Cloud Documentation¹.

2. Developer Tools and Guidance:

   • If you’re interested in Databricks developer tools and need guidance, this section is helpful. It covers various tools and best practices.
   • Access it here: Developer tools and guidance².

3. Databricks Reference Documentation:

   • For detailed reference information on Databricks APIs, SQL language, command-line interfaces, and more, explore this section.
   • Dive into the reference docs here: Databricks reference documentation³.

Feel free to explore these resources to get a deeper understanding of Databricks on GCP. If you have any more questions or need further assistance, don’t hesitate to ask! 😊⁴

[ashraf1395 · Accepted solution]

I had another question. Though not related to this thread.

Do databricks has any plan for startups, like they have normal free trial

[Kaniz · Accepted solution]

Hi @ashraf1395, Databricks offers a program specifically designed for startups called “Databricks for Startups.” If you’re building data-driven applications and have raised venture capital (VC) funding, this program can be beneficial for your startup.

Here are the key features of Databricks for Startups:

1. Free Credits:

   • Startups accepted into the program receive up to $21,000 in free credits to use on the Databricks Lakehouse Platform.
   • These credits allow you to explore and build data-driven solutions without worrying about infrastructure costs.

2. Expert Advice:

   • As part of the program, you gain access to advice from Databricks experts and the community.
   • Whether you need guidance on architecture, best practices, or troubleshooting, the Databricks community can provide valuable insights.

3. Go-to-Market Support:

   • Databricks helps startups reach more customers by providing access to marketing resources, events, and other promotional opportunities.
   • Leveraging Databricks’ marketing channels can help increase your startup’s visibility.

4. Built on the Databricks Lakehouse:

   • The Lakehouse architecture combines the best of data lakes and data warehouses, allowing you to address all your data, analytics, and AI needs on a single platform.
   • From early-stage development to scaling for growth (even beyond an initial public offering), the Lak...¹.

If you’re a startup founder working on data-driven applications, I encourage you to apply for Databricks for Startups. It’s a great opportunity to leverage Databricks’ powerful platform and accelerate your product development. Best of luck with your startup journey! 😊🚀¹