CREATE is not allowed: Cannot CREATE the Streaming Table Serverless Generic Compute

[adriyak]

Hello everyone, 

I am new in Databricks, and currently I am in the 8th lecture of the "Get started with data engineering" module. I came across this problem: [STREAMING_TABLE_OPERATION_NOT_ALLOWED.ST_NOT_ENABLED_ON_SERVERLESS_GENERIC_COMPUTE] The operation CREATE is not allowed: Cannot CREATE the Streaming Table `current_employees_bronze_sdp` in Serverless Generic Compute for your workspace. Enable it by enrolling in the Serverless Generic Compute Materialized View/Streaming Table workspace feature preview. If you do not see the beta feature preview available in workspace, please contact your Databricks representative. SQLSTATE: 42601, and I made sure that I entered serverless compute.

I already have made my Spark declarative pipeline, and the first error was that I couldn't run the SQL queries in the notebook while it was the source file of the pipeline. I moved it outside, and then I came across the overarching problem when I finally ran the queries. In every community note/helpdesk it was said that you must toggle the ST/MV feature preview which I don't have in my settings. 

Is it related to my region, permissions, or is it to do with the Unity catalog? It's a real catch-22 situation for me.

Any help would be appreciated! 

[adnan_alvee]

The error is not primarily a Unity Catalog or permissions issue. It means the workspace does not have the Serverless Generic Compute Materialized View/Streaming Table preview enabled.

There are two important distinctions here:

If you are building a Lakeflow/Spark Declarative Pipeline,  The CREATE OR REFRESH STREAMING TABLE statement belongs in a pipeline source file and should be executed with Run file or Run pipeline in the Lakeflow Pipelines Editor and not run as a normal notebook query. Pipeline source files are evaluated by the pipeline engine.

Now if you are are creating a standalone streaming table from a notebook, Serverless Generic Compute support is currently beta, region-limited, and requires the workspace preview. A SQL warehouse is the simpler workaround; standalone tables can be created or refreshed from a Unity Catalog-enabled Pro or Serverless SQL warehouse.