Parity between Spark.sql and SQL Warehouse for Metric Views & Model Visualization

smpa011
New Contributor II

Hi everyone,

I’m exploring the new Databricks Metric Views (Semantic Layer) and have two questions regarding programmatic management and UI visualization.

1. Parser Disparity: spark.sql vs. SQL Warehouse

I'm noticing that CREATE OR REPLACE VIEW ... WITH METRICS fails with a PARSE_SYNTAX_ERROR when executed via spark.sql() in a notebook, but works perfectly when run in a SQL Warehouse.

Is this architectural limitation by design? Are there plans to incorporate the Metric View DDL into the standard Spark parser so we can manage these programmatically via PySpark?

# This fails on standard clusters but is what I'd like to achieve:
spark.sql("""
CREATE OR REPLACE VIEW sales_metrics
WITH METRICS
LANGUAGE YAML
AS $$
version: 1.1
source: catalog.schema.fact_sales
joins:
  - name: dim_customer
    source: catalog.schema.fact_sales.dim_customer
    on: source.customer_id = dim_customer.customer_id
measures:
  - name: total_amount
    expr: sum(amount)
$$
""")

2. Graphical Data Model Visualization

Coming from a Power BI/SSAS background, I am looking for a way to visualize the relationships defined in the Metric View's YAML (the Star Schema) graphically.

  • Is there a way to view an Entity Relationship Diagram (ERD) for Metric Views within Catalog Explorer today?

  • If not, is a graphical "Model View" on the roadmap to help verify complex relationships and join logic visually?

Thanks in advance for the help!