- 1669 Views
- 3 replies
- 0 kudos
Delta Sharing within the same region
Hi Team,I have setup UC and have a Metastore for AUE. Assigned 2 workspaces to the AUE Metastore and both with different set of users and group . 1 workspace is the data provider and the other is a data consumerNow since 2 workspaces are in one Meta...
- 1669 Views
- 3 replies
- 0 kudos
- 0 kudos
Do we have any updates on this in terms of how we can share the data between different workspaces part of the same metastore?
- 0 kudos
- 5486 Views
- 7 replies
- 9 kudos
Resolved! Delta sharing without unity catalog
Hi all,Is there any way to implement delta sharing in databricks without enabling the unity catalog?
- 5486 Views
- 7 replies
- 9 kudos
- 9 kudos
Is it possible to receive data in databricks notebook without enabling the unity catalog?
- 9 kudos
- 6192 Views
- 3 replies
- 0 kudos
SQL Warehouse Cluster - How to configure External Hive Meta Store
In my Databricks regular Clusters, Our clusters are configured to an External Hive Meta Store (Azure SQL Server). I am able to set the External Hive configuration in the Advanced options Spark config setting within the cluster setting. In our setting...
- 6192 Views
- 3 replies
- 0 kudos
- 0 kudos
@RafaelGomez61 I have the some problem and after some exploring, I've configured it and also wrote a blog for the whole steps. You can have a look here: https://medium.com/@judy3.yang/how-to-configure-external-hive-meta-store-for-databricks-sql-wareh...
- 0 kudos
- 5562 Views
- 5 replies
- 4 kudos
Resolved! Access table outside Databricks (hive metastore)
Hi guys, how are you ?How can I access tables outside the Databricks ? (hive metastore)I have a python script in my local structure but I need to access are tables stored in Databricks (hive metastore) , how ???? Have any idea ?Thank you guys
- 5562 Views
- 5 replies
- 4 kudos
- 9900 Views
- 8 replies
- 13 kudos
Publish data to Azure ADLS Gen2 from Delta Live Tables pipelines?
I would like to publish data from from delta live table (DLT) to an Azure ADLS Gen2 storage. Checked the documentation but cannot get it to work. Any insights on how to do so?I've tried to specify the "Storage location" with many combinations of abfs...
- 9900 Views
- 8 replies
- 13 kudos
- 13 kudos
I am trying to load from ADLS with DLT, after add the spark configurations I got below errors:org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = 818323fc-80d5-4833-9f46-7d1afc9c5bf7, runId = 722e9aac-0fdd-4206-9d49-68...
- 13 kudos
- 3169 Views
- 2 replies
- 2 kudos
Delta Live tables publish to Unity Catalog
How to enable DLT pipelines for the tables which are in UC. Could you please suggest us the approach or if there is any workaround?
- 3169 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Janga Reddy​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
- 2 kudos
- 8014 Views
- 2 replies
- 0 kudos
- 8014 Views
- 2 replies
- 0 kudos
- 5392 Views
- 6 replies
- 10 kudos
Issue with Delta Live tables access from workflow ( dlt pipelines )
Hi all,I am trying to write data from external s3 bucket to delta live tables at unity catalog( not dbfs ) from a workflow. I am seeing the following error ( saying catalog namespace is not supported. Please check full error below ) Error Detailspy4...
- 5392 Views
- 6 replies
- 10 kudos
- 1542 Views
- 1 replies
- 2 kudos
- 1542 Views
- 1 replies
- 2 kudos
- 2 kudos
I dont know what is TAClbut in the unity catalog you can have governance like catalog.database.table featurewhere you have to explicitly give access to the databricks data user so you have to give him access separately for catalog and database and ta...
- 2 kudos
- 2532 Views
- 1 replies
- 7 kudos
Resolved! Unity Catalog support for DLT
Hi,Does the Unity catalog support the Delta Live Tables? If yes, please help me with the documentation link. If not, any update on the feature readiness timeline would help. Thank you!
- 2532 Views
- 1 replies
- 7 kudos
- 7 kudos
Hi @Venkadeshwaran K​ ,No, UC does not support DLT as of today. We have this feature on the timeline. We will be having a private preview by end of this year for DLT and UC integration for SQL. There is no ETA yet for GA but you can definitely expect...
- 7 kudos
- 26015 Views
- 4 replies
- 2 kudos
DeltaTable.forPath(spark, path) doesnt recognize table
Hi,I'm working with unity catalog for the last week. I'm refering to delta table by path, as follwing: path='s3://<my_bucket_name>/silver/data/<table_name>DeltaTable.forPath(spark, path)I get an exception that "is not a Delta table"using the table na...
- 26015 Views
- 4 replies
- 2 kudos
- 2 kudos
its even more weired. on one next cells it doesnt... see below older version even by name doesnt work
- 2 kudos
- 1367 Views
- 0 replies
- 6 kudos
Your guide to data and AI governance sessions at Data + AI Summit 2022 We are so excited to welcome you to Data + AI Summit this year. This year’s hyb...
Your guide to data and AI governance sessions at Data + AI Summit 2022We are so excited to welcome you to Data + AI Summit this year. This year’s hybrid conference is jam packed with exciting product announcements, live keynotes, breakout sessions wi...
- 1367 Views
- 0 replies
- 6 kudos
- 3860 Views
- 1 replies
- 24 kudos
Databricks Runtime 10.5 (Beta) �� Auto Loader: new SQL function CLOUD_FILES_STATE You can use the new CLOUD_FILES_STATE function to query the ...
Databricks Runtime 10.5 (Beta) Auto Loader: new SQL function CLOUD_FILES_STATEYou can use the new CLOUD_FILES_STATE function to query the internal state of an Auto Loader stream. Delta Lake: new maxRecordsPerFile option for maximum records written t...
- 3860 Views
- 1 replies
- 24 kudos
- 24 kudos
anyone that got CLOUD_FILES_STATE to work on 10.5 Beta/Azure. Syntax:%sqlSELECT * FROM cloud_files_state('abfss://test@test.dfs.core.windows.net/test/checkpoint/'); i get this error: com.databricks.backend.common.rpc.DatabricksExceptions$SQLExecu...
- 24 kudos
- 13876 Views
- 8 replies
- 6 kudos
Resolved! Enable Unity Catalog and pre-requisite for Delta Sharing
Hi, I would like to use Delta Sharing but I have not understood how to configure the Delta Sharing Server. Is that fully managed by Databricks ? If so I tried to share a table :%sql create share first_shareand I got the following error message with t...
- 13876 Views
- 8 replies
- 6 kudos
- 6 kudos
Hi @Bertrand BURCKER​ ,This feature is not available yet. If you would like to test it or explore it, then you will need to sign up to get more details once the feature is available. In addition, you could reach out to your account team (in case you...
- 6 kudos
- 5627 Views
- 2 replies
- 0 kudos
- 5627 Views
- 2 replies
- 0 kudos
- 0 kudos
You could use delta sharing without unity catalog. It is an open protocol for secure real-time exchange of large datasets. There repo at https://github.com/delta-io/delta-sharing also has a reference implementation of the delta sharing server which c...
- 0 kudos