Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-04-2025 10:02 AM
Ah, got it @pabloratache , I did some digging and here is what I found (learned a few things myself).
Thanks for the detailed context — this behavior is expected for the current Databricks 14‑day Free Trial (“For Work” plan).
What’s happening with your Free Trial
- The 14‑day For Work trial provisions a serverless workspace by design, which exposes SQL Warehouses and serverless compute, not classic All‑Purpose clusters.
-
During the trial, serverless compute is available for notebooks, jobs, and Lakeflow Declarative Pipelines, with scaling capped to 50 DBUs/hr. One SQL warehouse per workspace is allowed (also capped), and GPUs are not available.
-
External network access is limited in trial workspaces, which can affect streaming sources/sinks; the recommended workaround is to upload data into the workspace for use in pipelines.
Answers to your questions
- 1) Is this a provisioning delay?
No — this is by design for the Free Trial serverless workspace, not a delay. -
2) Do I need manual enablement for All‑Purpose Clusters?
Manual enablement isn’t available on Free Trials. To use All‑Purpose clusters, you need to upgrade to a paid workspace (Standard/Premium); then you can create clusters (subject to your cloud account quotas). -
3) Is there a different Free Trial tier that includes PySpark?
The trial does include PySpark, but via serverless compute in notebooks (not classic clusters). In a notebook, click Connect (top right) and choose Serverless to run Python/PySpark code.
How to run PySpark now (without clusters)
- Create or open a notebook, click Connect (top‑right), choose Serverless, set the language to Python, and run your PySpark code there.
# Quick smoke test in a serverless notebook
from pyspark.sql import functions as F
df = spark.range(0, 10).withColumn("ts", F.current_timestamp())
df.write.format("delta").mode("overwrite").saveAsTable("demo.bronze_range")
display(spark.table("demo.bronze_range"))
- You can orchestrate pipelines with Lakeflow Jobs or Jobs on serverless compute during the trial; note the 50 DBUs/hr cap and external network limits.
Why your UI looks SQL‑only under Compute
- In serverless trial workspaces, you won’t see the classic “Clusters” page; compute for notebooks is requested inline via the Connect → Serverless flow rather than through the Clusters UI.
If you need classic All‑Purpose clusters now
- Upgrade to a paid plan and use a traditional workspace deployed in your AWS account; then create All‑Purpose clusters and run PySpark/Streaming as needed (you’ll pay cloud infra costs in addition to DBUs after trial).
FYI: Trial page contact
- For onboarding questions, you can email onboarding‑help@databricks.com from the trial page.
Hope this makes things clear.
Cheers, Louis.