- 1802 Views
- 3 replies
- 0 kudos
Delta Sharing within the same region
Hi Team,I have setup UC and have a Metastore for AUE. Assigned 2 workspaces to the AUE Metastore and both with different set of users and group . 1 workspace is the data provider and the other is a data consumerNow since 2 workspaces are in one Meta...
- 1802 Views
- 3 replies
- 0 kudos
- 0 kudos
Do we have any updates on this in terms of how we can share the data between different workspaces part of the same metastore?
- 0 kudos
- 5672 Views
- 7 replies
- 9 kudos
Resolved! Delta sharing without unity catalog
Hi all,Is there any way to implement delta sharing in databricks without enabling the unity catalog?
- 5672 Views
- 7 replies
- 9 kudos
- 9 kudos
Is it possible to receive data in databricks notebook without enabling the unity catalog?
- 9 kudos
- 6419 Views
- 3 replies
- 0 kudos
SQL Warehouse Cluster - How to configure External Hive Meta Store
In my Databricks regular Clusters, Our clusters are configured to an External Hive Meta Store (Azure SQL Server). I am able to set the External Hive configuration in the Advanced options Spark config setting within the cluster setting. In our setting...
- 6419 Views
- 3 replies
- 0 kudos
- 0 kudos
@RafaelGomez61 I have the some problem and after some exploring, I've configured it and also wrote a blog for the whole steps. You can have a look here: https://medium.com/@judy3.yang/how-to-configure-external-hive-meta-store-for-databricks-sql-wareh...
- 0 kudos
- 5793 Views
- 5 replies
- 4 kudos
Resolved! Access table outside Databricks (hive metastore)
Hi guys, how are you ?How can I access tables outside the Databricks ? (hive metastore)I have a python script in my local structure but I need to access are tables stored in Databricks (hive metastore) , how ???? Have any idea ?Thank you guys
- 5793 Views
- 5 replies
- 4 kudos
- 10330 Views
- 8 replies
- 13 kudos
Publish data to Azure ADLS Gen2 from Delta Live Tables pipelines?
I would like to publish data from from delta live table (DLT) to an Azure ADLS Gen2 storage. Checked the documentation but cannot get it to work. Any insights on how to do so?I've tried to specify the "Storage location" with many combinations of abfs...
- 10330 Views
- 8 replies
- 13 kudos
- 13 kudos
I am trying to load from ADLS with DLT, after add the spark configurations I got below errors:org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = 818323fc-80d5-4833-9f46-7d1afc9c5bf7, runId = 722e9aac-0fdd-4206-9d49-68...
- 13 kudos
- 3323 Views
- 2 replies
- 2 kudos
Delta Live tables publish to Unity Catalog
How to enable DLT pipelines for the tables which are in UC. Could you please suggest us the approach or if there is any workaround?
- 3323 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Janga Reddy​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
- 2 kudos
- 8065 Views
- 2 replies
- 0 kudos
- 8065 Views
- 2 replies
- 0 kudos
- 5639 Views
- 6 replies
- 10 kudos
Issue with Delta Live tables access from workflow ( dlt pipelines )
Hi all,I am trying to write data from external s3 bucket to delta live tables at unity catalog( not dbfs ) from a workflow. I am seeing the following error ( saying catalog namespace is not supported. Please check full error below ) Error Detailspy4...
- 5639 Views
- 6 replies
- 10 kudos
- 1607 Views
- 1 replies
- 2 kudos
- 1607 Views
- 1 replies
- 2 kudos
- 2 kudos
I dont know what is TAClbut in the unity catalog you can have governance like catalog.database.table featurewhere you have to explicitly give access to the databricks data user so you have to give him access separately for catalog and database and ta...
- 2 kudos
- 2617 Views
- 1 replies
- 7 kudos
Resolved! Unity Catalog support for DLT
Hi,Does the Unity catalog support the Delta Live Tables? If yes, please help me with the documentation link. If not, any update on the feature readiness timeline would help. Thank you!
- 2617 Views
- 1 replies
- 7 kudos
- 7 kudos
Hi @Venkadeshwaran K​ ,No, UC does not support DLT as of today. We have this feature on the timeline. We will be having a private preview by end of this year for DLT and UC integration for SQL. There is no ETA yet for GA but you can definitely expect...
- 7 kudos
- 26867 Views
- 4 replies
- 2 kudos
DeltaTable.forPath(spark, path) doesnt recognize table
Hi,I'm working with unity catalog for the last week. I'm refering to delta table by path, as follwing: path='s3://<my_bucket_name>/silver/data/<table_name>DeltaTable.forPath(spark, path)I get an exception that "is not a Delta table"using the table na...
- 26867 Views
- 4 replies
- 2 kudos
- 2 kudos
its even more weired. on one next cells it doesnt... see below older version even by name doesnt work
- 2 kudos
- 1409 Views
- 0 replies
- 6 kudos
Your guide to data and AI governance sessions at Data + AI Summit 2022 We are so excited to welcome you to Data + AI Summit this year. This year’s hyb...
Your guide to data and AI governance sessions at Data + AI Summit 2022We are so excited to welcome you to Data + AI Summit this year. This year’s hybrid conference is jam packed with exciting product announcements, live keynotes, breakout sessions wi...
- 1409 Views
- 0 replies
- 6 kudos
- 3929 Views
- 1 replies
- 24 kudos
Databricks Runtime 10.5 (Beta) �� Auto Loader: new SQL function CLOUD_FILES_STATE You can use the new CLOUD_FILES_STATE function to query the ...
Databricks Runtime 10.5 (Beta) Auto Loader: new SQL function CLOUD_FILES_STATEYou can use the new CLOUD_FILES_STATE function to query the internal state of an Auto Loader stream. Delta Lake: new maxRecordsPerFile option for maximum records written t...
- 3929 Views
- 1 replies
- 24 kudos
- 24 kudos
anyone that got CLOUD_FILES_STATE to work on 10.5 Beta/Azure. Syntax:%sqlSELECT * FROM cloud_files_state('abfss://test@test.dfs.core.windows.net/test/checkpoint/'); i get this error: com.databricks.backend.common.rpc.DatabricksExceptions$SQLExecu...
- 24 kudos
- 14079 Views
- 8 replies
- 6 kudos
Resolved! Enable Unity Catalog and pre-requisite for Delta Sharing
Hi, I would like to use Delta Sharing but I have not understood how to configure the Delta Sharing Server. Is that fully managed by Databricks ? If so I tried to share a table :%sql create share first_shareand I got the following error message with t...
- 14079 Views
- 8 replies
- 6 kudos
- 6 kudos
Hi @Bertrand BURCKER​ ,This feature is not available yet. If you would like to test it or explore it, then you will need to sign up to get more details once the feature is available. In addition, you could reach out to your account team (in case you...
- 6 kudos
- 5874 Views
- 2 replies
- 0 kudos
- 5874 Views
- 2 replies
- 0 kudos
- 0 kudos
You could use delta sharing without unity catalog. It is an open protocol for secure real-time exchange of large datasets. There repo at https://github.com/delta-io/delta-sharing also has a reference implementation of the delta sharing server which c...
- 0 kudos