cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Jfoxyyc
by Valued Contributor
  • 1692 Views
  • 2 replies
  • 0 kudos

How to manage dev/prod environments with new unity catalog delta live table integration

Heres my current setup, dev workspace connected to dev keyvault and a prod workspace connected to a prod keyvault. There's a github repo and action syncing the two environments on pull request and all resources created through terraform. This is my n...

  • 1692 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Jordan Fox​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...

  • 0 kudos
1 More Replies
Alex006
by Contributor
  • 5925 Views
  • 8 replies
  • 13 kudos

Publish data to Azure ADLS Gen2 from Delta Live Tables pipelines?

I would like to publish data from from delta live table (DLT) to an Azure ADLS Gen2 storage. Checked the documentation but cannot get it to work. Any insights on how to do so?I've tried to specify the "Storage location" with many combinations of abfs...

Invalid configuration value detected for fs.azure.account.key Storage location
  • 5925 Views
  • 8 replies
  • 13 kudos
Latest Reply
guostong
New Contributor III
  • 13 kudos

I am trying to load from ADLS with DLT, after add the spark configurations I got below errors:org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = 818323fc-80d5-4833-9f46-7d1afc9c5bf7, runId = 722e9aac-0fdd-4206-9d49-68...

  • 13 kudos
7 More Replies
Phani1
by Valued Contributor
  • 977 Views
  • 2 replies
  • 2 kudos

Delta Live tables publish to Unity Catalog

How to enable DLT pipelines for the tables which are in UC. Could you please suggest us the approach or if there is any workaround?

  • 977 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Janga Reddy​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 2 kudos
1 More Replies
RicksDB
by Contributor II
  • 2390 Views
  • 5 replies
  • 5 kudos

DLT and Unity catalog best practices

What are the best practices when using DLT with a unity catalog aware workspace since it is not supported at the moment?Do you create a copy of every table you need in the hive metastore in order to reference them? Or is it recommended to wait until ...

  • 2390 Views
  • 5 replies
  • 5 kudos
Latest Reply
Andrij_D
New Contributor II
  • 5 kudos

Hi @E H​ , the best way to manage this for the time being is to limit DLT to logic that is considered 'internal' i.e., leveraging the default hive metastore. For now at least, this should be treated as a transient landing zone. From here you can refi...

  • 5 kudos
4 More Replies
Aviral-Bhardwaj
by Esteemed Contributor III
  • 3212 Views
  • 2 replies
  • 37 kudos

Understanding DLT pipe lines in Databricks In Databricks, a DLT (Data Live Table) pipeline is a set of data transformations that are applied to data a...

Understanding DLT pipe lines in DatabricksIn Databricks, a DLT (Data Live Table) pipeline is a set of data transformations that are applied to data assets in a defined sequence, in order to clean, enrich, and prepare data for analysis or other purpos...

  • 3212 Views
  • 2 replies
  • 37 kudos
Latest Reply
Meghala
Valued Contributor II
  • 37 kudos

Im thanking @Aviral Bhardwaj​  for this kind of information

  • 37 kudos
1 More Replies
apingle
by Contributor
  • 3541 Views
  • 3 replies
  • 4 kudos

Resolved! DLT not working with Unity Catalog

Hey guys,I am trying to create a delta live table in Unity Catalog as follows:CREATE OR REFRESH STREAMING LIVE TABLE <catalog>.<db>.<table_name> AS SELECT ... However, I get the error: org.apache.spark.sql.AnalysisException: Unsupported SQL statemen...

  • 3541 Views
  • 3 replies
  • 4 kudos
Latest Reply
Harun
Honored Contributor
  • 4 kudos

@Anant Pingle​  As of right now, DLT doesn't support Unity Catalog yet. We can expect this feature in upcomig days.

  • 4 kudos
2 More Replies
venkad
by Contributor
  • 1277 Views
  • 2 replies
  • 7 kudos

Resolved! Unity Catalog support for DLT

Hi,Does the Unity catalog support the Delta Live Tables? If yes, please help me with the documentation link. If not, any update on the feature readiness timeline would help. Thank you!

  • 1277 Views
  • 2 replies
  • 7 kudos
Latest Reply
Kaniz
Community Manager
  • 7 kudos

Hi @Venkadeshwaran K​, We haven’t heard from you since the last response from @Vivian Wilfred​​, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please share it with the community, as it can be helpful...

  • 7 kudos
1 More Replies
Labels