cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

RafaelGomez61
by New Contributor
  • 2872 Views
  • 3 replies
  • 0 kudos

SQL Warehouse Cluster - How to configure External Hive Meta Store

In my Databricks regular Clusters, Our clusters are configured to an External Hive Meta Store (Azure SQL Server). I am able to set the External Hive configuration in the Advanced options Spark config setting within the cluster setting. In our setting...

  • 2872 Views
  • 3 replies
  • 0 kudos
Latest Reply
judyy
New Contributor II
  • 0 kudos

@RafaelGomez61 I have the some problem and after some exploring, I've configured it and also wrote a blog for the whole steps. You can have a look here: https://medium.com/@judy3.yang/how-to-configure-external-hive-meta-store-for-databricks-sql-wareh...

  • 0 kudos
2 More Replies
William_Scardua
by Valued Contributor
  • 3866 Views
  • 5 replies
  • 4 kudos

Resolved! Access table outside Databricks (hive metastore)

Hi guys, how are you ?How can I access tables outside the Databricks ? (hive metastore)I have a python script in my local structure but I need to access are tables stored in Databricks (hive metastore) , how ???? Have any idea ?Thank you guys

  • 3866 Views
  • 5 replies
  • 4 kudos
Latest Reply
pokharatours
New Contributor II
  • 4 kudos

Thanks, Pokhara Tour

  • 4 kudos
4 More Replies
Alex006
by Contributor
  • 5859 Views
  • 8 replies
  • 13 kudos

Publish data to Azure ADLS Gen2 from Delta Live Tables pipelines?

I would like to publish data from from delta live table (DLT) to an Azure ADLS Gen2 storage. Checked the documentation but cannot get it to work. Any insights on how to do so?I've tried to specify the "Storage location" with many combinations of abfs...

Invalid configuration value detected for fs.azure.account.key Storage location
  • 5859 Views
  • 8 replies
  • 13 kudos
Latest Reply
guostong
New Contributor III
  • 13 kudos

I am trying to load from ADLS with DLT, after add the spark configurations I got below errors:org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = 818323fc-80d5-4833-9f46-7d1afc9c5bf7, runId = 722e9aac-0fdd-4206-9d49-68...

  • 13 kudos
7 More Replies
Phani1
by Valued Contributor
  • 860 Views
  • 2 replies
  • 2 kudos

Delta Live tables publish to Unity Catalog

How to enable DLT pipelines for the tables which are in UC. Could you please suggest us the approach or if there is any workaround?

  • 860 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Janga Reddy​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 2 kudos
1 More Replies
Garik
by New Contributor III
  • 3165 Views
  • 7 replies
  • 10 kudos

Issue with Delta Live tables access from workflow ( dlt pipelines )

Hi all,I am trying to write data from external s3 bucket to delta live tables at unity catalog( not dbfs ) from a workflow. I am seeing the following error ( saying catalog namespace is not supported. Please check full error below ) Error Detailspy4...

  • 3165 Views
  • 7 replies
  • 10 kudos
Latest Reply
Sulfikkar
Contributor
  • 10 kudos

How long should we wait to use DLT on top of the Unity catalog?

  • 10 kudos
6 More Replies
Gilg
by Contributor II
  • 752 Views
  • 2 replies
  • 0 kudos

Delta Sharing within the same region

Hi Team,I have setup UC and have a Metastore for AUE. Assigned 2 workspaces to the AUE Metastore and both with different set of users and group . 1 workspace is the data provider and the other is a data consumerNow since 2 workspaces are in one Meta...

  • 752 Views
  • 2 replies
  • 0 kudos
Latest Reply
youssefmrini
Honored Contributor III
  • 0 kudos

If the two Workspaces are under the same metastore you don't need Delta Sharing because they already have access to the same data. Unless you want to do data segregation per workspace. if it's the case it's coming soon at the end of february

  • 0 kudos
1 More Replies
Ajay-Pandey
by Esteemed Contributor III
  • 2389 Views
  • 6 replies
  • 9 kudos

Resolved! Delta sharing without unity catalog

Hi all,Is there any way to implement delta sharing in databricks without enabling the unity catalog?

  • 2389 Views
  • 6 replies
  • 9 kudos
Latest Reply
Rishabh264
Honored Contributor II
  • 9 kudos

hey @Ajay Pandey​ refer this link for better understanding https://docs.databricks.com/data-sharing/index.html

  • 9 kudos
5 More Replies
User16868770416
by Contributor
  • 818 Views
  • 1 replies
  • 2 kudos
  • 818 Views
  • 1 replies
  • 2 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 2 kudos

I dont know what is TAClbut in the unity catalog you can have governance like catalog.database.table featurewhere you have to explicitly give access to the databricks data user so you have to give him access separately for catalog and database and ta...

  • 2 kudos
venkad
by Contributor
  • 1267 Views
  • 2 replies
  • 7 kudos

Resolved! Unity Catalog support for DLT

Hi,Does the Unity catalog support the Delta Live Tables? If yes, please help me with the documentation link. If not, any update on the feature readiness timeline would help. Thank you!

  • 1267 Views
  • 2 replies
  • 7 kudos
Latest Reply
Kaniz
Community Manager
  • 7 kudos

Hi @Venkadeshwaran K​, We haven’t heard from you since the last response from @Vivian Wilfred​​, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please share it with the community, as it can be helpful...

  • 7 kudos
1 More Replies
amitca71
by Contributor II
  • 12918 Views
  • 4 replies
  • 2 kudos

DeltaTable.forPath(spark, path) doesnt recognize table

Hi,I'm working with unity catalog for the last week. I'm refering to delta table by path, as follwing: path='s3://<my_bucket_name>/silver/data/<table_name>DeltaTable.forPath(spark, path)I get an exception that "is not a Delta table"using the table na...

image
  • 12918 Views
  • 4 replies
  • 2 kudos
Latest Reply
amitca71
Contributor II
  • 2 kudos

its even more weired. on one next cells it doesnt... see below older version even by name doesnt work 

  • 2 kudos
3 More Replies
Anonymous
by Not applicable
  • 680 Views
  • 0 replies
  • 6 kudos

Your guide to data and AI governance sessions at Data + AI Summit 2022 We are so excited to welcome you to Data + AI Summit this year. This year’s hyb...

Your guide to data and AI governance sessions at Data + AI Summit 2022We are so excited to welcome you to Data + AI Summit this year. This year’s hybrid conference is jam packed with exciting product announcements, live keynotes, breakout sessions wi...

  • 680 Views
  • 0 replies
  • 6 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 3105 Views
  • 2 replies
  • 24 kudos

Databricks Runtime 10.5 (Beta)  &#xd83d;&#xdc49; Auto Loader: new SQL function CLOUD_FILES_STATE You can use the new CLOUD_FILES_STATE function to query the ...

Databricks Runtime 10.5 (Beta) Auto Loader: new SQL function CLOUD_FILES_STATEYou can use the new CLOUD_FILES_STATE function to query the internal state of an Auto Loader stream. Delta Lake: new maxRecordsPerFile option for maximum records written t...

  • 3105 Views
  • 2 replies
  • 24 kudos
Latest Reply
SørenRavn
New Contributor II
  • 24 kudos

anyone that got CLOUD_FILES_STATE to work on 10.5 Beta/Azure. Syntax:%sqlSELECT * FROM cloud_files_state('abfss://test@test.dfs.core.windows.net/test/checkpoint/'); i get this error: com.databricks.backend.common.rpc.DatabricksExceptions$SQLExecu...

  • 24 kudos
1 More Replies
RantoB
by Valued Contributor
  • 11759 Views
  • 8 replies
  • 6 kudos

Resolved! Enable Unity Catalog and pre-requisite for Delta Sharing

Hi, I would like to use Delta Sharing but I have not understood how to configure the Delta Sharing Server. Is that fully managed by Databricks ? If so I tried to share a table :%sql create share first_shareand I got the following error message with t...

  • 11759 Views
  • 8 replies
  • 6 kudos
Latest Reply
jose_gonzalez
Moderator
  • 6 kudos

Hi @Bertrand BURCKER​ ,This feature is not available yet. If you would like to test it or explore it, then you will need to sign up to get more details once the feature is available. In addition, you could reach out to your account team (in case you...

  • 6 kudos
7 More Replies
Anonymous
by Not applicable
  • 4509 Views
  • 2 replies
  • 0 kudos
  • 4509 Views
  • 2 replies
  • 0 kudos
Latest Reply
sajith_appukutt
Honored Contributor II
  • 0 kudos

You could use delta sharing without unity catalog. It is an open protocol for secure real-time exchange of large datasets. There repo at https://github.com/delta-io/delta-sharing also has a reference implementation of the delta sharing server which c...

  • 0 kudos
1 More Replies
Labels