cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Views in DLT with Private Preview feature Direct Publish

susanne
New Contributor III

Hi everyone,

I am building a dlt Pipeline and there I am using the Direct Publish feature which is as of now still under Private Preview.
While it works well to create streaming tables and write them to another schema than the dlt  default schema, I get the following error message, when I try to create a View and write it to a different schema than the default schema:
AnalysisException: View with multipart name 'gold.fact_customer' is not supported.
I used the following code to create the View:

@dlt.table(
name = target_table,
)
def get_gold_table():
df = spark.sql(f"""{gold_selection}""")
return df
Does Direct Publish in general not support publishing of views to another schema?
Any help would be much appreciated.
Thank you 🙂
Susanne
1 ACCEPTED SOLUTION

Accepted Solutions

Sidhant07
Databricks Employee
Databricks Employee

Hi @susanne ,

Use Tables Instead of Views: If possible, convert your views to tables.

@dlt.table(
    name=target_table,
)
def get_gold_table():
    df = spark.sql(f"""{gold_selection}""")
    return df

Direct Publish Feature: The Direct Publish feature does not support publishing views to another schema. It is intended for tables and materialized views, which can be published to the catalog.

Solution: To publish data to a different schema, consider using a table or materialized view instead of a view, as these can be written to the catalog.

View solution in original post

2 REPLIES 2

Sidhant07
Databricks Employee
Databricks Employee

Hi @susanne ,

Use Tables Instead of Views: If possible, convert your views to tables.

@dlt.table(
    name=target_table,
)
def get_gold_table():
    df = spark.sql(f"""{gold_selection}""")
    return df

Direct Publish Feature: The Direct Publish feature does not support publishing views to another schema. It is intended for tables and materialized views, which can be published to the catalog.

Solution: To publish data to a different schema, consider using a table or materialized view instead of a view, as these can be written to the catalog.

susanne
New Contributor III

Hi Sidhan,
thanks a lot for your reply, it works very well to write materialized views to a different schema than the default schema.
Thanks for your guidance!

Best regards
Susanne

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group