cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Governance

Forum Posts

DrK
by New Contributor III
  • 9985 Views
  • 2 replies
  • 1 kudos

Databricks Account in Azure

Hi,We have been piloting databricks using unity catalog in development in our azure development subscription. We are looking at plans for moving to production\live which is a completely different azure subscription. I created a workspace in product...

  • 9985 Views
  • 2 replies
  • 1 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 1 kudos

@Andy Skinner​ When you create your workspace that will be tied to your databricks account.your account will be tied to subscription and resource group .from E2 architecture you can create multiple workspaces (in different regions) in one databricks ...

  • 1 kudos
1 More Replies
Kayl669
by New Contributor III
  • 2096 Views
  • 5 replies
  • 3 kudos

Advice regarding retaining environment-specific access when applying Unity Catalog to workspaces split by environment

My org is considering a transition from hive metastore to unity catalog. We currently have a workspace for each of dev/uat/production and each of those provide access to their respective blob storage account data. Unity Catalog sits at the account-le...

  • 2096 Views
  • 5 replies
  • 3 kudos
Latest Reply
Kayl669
New Contributor III
  • 3 kudos

I did find this document which indicates that you can set the initial catalog on cluster start:: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/hive-metastore#differences-from-table-access-control

  • 3 kudos
4 More Replies
Alex006
by Contributor
  • 7327 Views
  • 8 replies
  • 13 kudos

Publish data to Azure ADLS Gen2 from Delta Live Tables pipelines?

I would like to publish data from from delta live table (DLT) to an Azure ADLS Gen2 storage. Checked the documentation but cannot get it to work. Any insights on how to do so?I've tried to specify the "Storage location" with many combinations of abfs...

Invalid configuration value detected for fs.azure.account.key Storage location
  • 7327 Views
  • 8 replies
  • 13 kudos
Latest Reply
guostong
New Contributor III
  • 13 kudos

I am trying to load from ADLS with DLT, after add the spark configurations I got below errors:org.apache.spark.sql.streaming.StreamingQueryException: [STREAM_FAILED] Query [id = 818323fc-80d5-4833-9f46-7d1afc9c5bf7, runId = 722e9aac-0fdd-4206-9d49-68...

  • 13 kudos
7 More Replies
Rishabh-Pandey
by Honored Contributor II
  • 4740 Views
  • 6 replies
  • 15 kudos

Resolved! Unity catalog with delta live tables

hey i want to know how we can create my delta tables in unity catalog by the use of delta live tables pipelines or any other method by which we can create delta live tables to the unity catalog???

  • 4740 Views
  • 6 replies
  • 15 kudos
Latest Reply
Sulfikkar
Contributor
  • 15 kudos

How long should we wait to use DLT on top of the unity catalog?

  • 15 kudos
5 More Replies
AlberTad
by New Contributor
  • 1435 Views
  • 3 replies
  • 0 kudos
  • 1435 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @AlberTad​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we c...

  • 0 kudos
2 More Replies
Phani1
by Valued Contributor
  • 2382 Views
  • 2 replies
  • 2 kudos

Delta Live tables publish to Unity Catalog

How to enable DLT pipelines for the tables which are in UC. Could you please suggest us the approach or if there is any workaround?

  • 2382 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Janga Reddy​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 2 kudos
1 More Replies
Mado
by Valued Contributor II
  • 2622 Views
  • 2 replies
  • 2 kudos

Resolved! Can I get cluster logs in an external storage account?

Hi,I have two questions about cluster logs.Q1) It seems that I can only store logs on DBFS. But, is there any way I can send the cluster logs to an Azure storage account?Note that workspace is not enabled for the Unity Catalog (and is not aimed to be...

image
  • 2622 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Mohammad Saber​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 2 kudos
1 More Replies
knawara
by Contributor
  • 3060 Views
  • 4 replies
  • 1 kudos

Mutliple Hive catalogs

Hello,Is it possible to use multiple Hive catalogs with Databricks? I know it can be done with Unity Catalog, but we're the only workspace admins in our organization. As far as I understand, we'd need account-level access to set up Unity Catalog.Best...

  • 3060 Views
  • 4 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Chris Nawara​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so ...

  • 1 kudos
3 More Replies
mimoliho
by New Contributor II
  • 3971 Views
  • 3 replies
  • 2 kudos

Query information schema under Catalogs from Tableau Desktop - what are the permissions?

Hi there,We have Unity Catalogs enabled in Azure Databricks.I would like to connect to system.information_schema.* or any other *.information_schema.*. Querying in UI or other IDEs works as expected - select * from system.information_schema.tables.No...

no_system_catalog system_catalog syste_catalog_inf system_catalog_table
  • 3971 Views
  • 3 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Egor Larin​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 2 kudos
2 More Replies
190809
by Contributor
  • 1320 Views
  • 2 replies
  • 0 kudos

Understanding the effects of upgrading to Unity Catalog.

Hello, I have been trying to solidify my understanding of what changes occur after I upgrade my data to Unity Catalog. Our current ingestion process incrementally loads new data to tables in the hive metastore which are mounted to an s3 bucket. If I ...

  • 1320 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Rachel Cunningham​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....

  • 0 kudos
1 More Replies
dvmentalmadess
by Valued Contributor
  • 4094 Views
  • 1 replies
  • 1 kudos

Resolved! Do external tables automatically receive external updates?

Based on the instructions for creating an external table (see: https://docs.databricks.com/data-governance/unity-catalog/create-tables.html#create-a-table) I had assumed that external tables were a way to add an existing object store to Unity Catalog...

  • 4094 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

@Mark Miller​ :External tables in Databricks do not automatically receive external updates. When you create an external table in Databricks, you are essentially registering the metadata for an existing object store in Unity Catalog, which allows you ...

  • 1 kudos
489702
by New Contributor II
  • 1603 Views
  • 3 replies
  • 4 kudos

Hide hive_metastore

Does anyone know if its possible to hide the hive_metastore catalog from users or groups.

  • 1603 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Jim Burris​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 4 kudos
2 More Replies
MetaRossiVinli
by Contributor
  • 3130 Views
  • 3 replies
  • 3 kudos

Resolved! In a new workspace without any data using the Unity Catalog, can I hide/delete the hive_metastore, main, samples, and system catalogs?

I am setting up a new workspace that will use the Unity Catalog. I want all data stored in the Unity Catalog in the following catalogs: dev, staging, prod. I want to prevent users from accidentally reading and writing data elsewhere.For the above sit...

  • 3130 Views
  • 3 replies
  • 3 kudos
Latest Reply
Avvar2022
Contributor
  • 3 kudos

@Kevin Rossi​ @John Lourdu​  - I am also new to databricks setting up environment. Bu default "all users" have read access to below mentioned catalogs,my question is - i see an option to revoke read access, is it must have read access to all these ca...

  • 3 kudos
2 More Replies
shiva12494
by New Contributor II
  • 8112 Views
  • 5 replies
  • 1 kudos

Unable to Access external location on GCP from databricks on GCP.

I am trying to add a GCP storage as an external location and read and write Unity catalog enabled delta tables to that external location in GCP databricks. I keep getting the error that the databricks instance service principal doesn't have access to...

Screen Shot 2023-04-14 at 3.28.04 PM
  • 8112 Views
  • 5 replies
  • 1 kudos
Latest Reply
Linda
New Contributor II
  • 1 kudos

Have you found out the solution?

  • 1 kudos
4 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels