Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Showing results for 
Search instead for 
Did you mean: 

dlt pipeline will not create live tables

New Contributor

I have created a dlt pipeline based four sql notebooks, each containing between 1 and 3 queries.  Each query begins with "create or refresh live table ..." yet each one outputs a materialized view.  I have tried deleting the materialized views and running the pipeline from scratch but it creates views again.

I can't find any examples of other people having this issue. What am I doing wrong? Below is one of the queries.


create or refresh live table table_name
comment "union of table1 and table2 data"
  select * from live.table1 union select * from live.table2;

Community Manager
Community Manager

Hi @mysteryuser000

  1. In Databricks SQL, materialized views are Unity Catalog managed tables that allow you to precompute results based on the latest data in source tables. Unlike other implementations, the results returned reflect the state of data when the materialized v...
  2. When you create a materialized view in a Databricks SQL warehouse, a Delta Live Tables pipeline is created to process refreshes to the materialized view. You can monitor the status of refresh operations in the Delta Live Tables UI.
  3. Now, let’s address your specific query:
COMMENT "union of table1 and table2 data"
SELECT * FROM live.table1 UNION SELECT * FROM live.table2;
  • Ensure that live.table1 and live.table2 exist and contain the expected data.
  • Confirm that there are no other dependencies (e.g., triggers, views) causing the materialized view to be recreated.
  • Remember that materialized views in Databricks SQL reflect the state of data when last refreshed. If the underlying data changes, the materialized view won’t automatically update until explicitly refreshed.
  • To manually refresh the materialized view, use the following command:
REFRESH MATERIALIZED VIEW catalog.schema.view_name;
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!