cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MadelynM
by Databricks Employee
  • 2085 Views
  • 0 replies
  • 0 kudos

[Recap] Data + AI Summit 2024 - Data Governance | Navigate the explosion of AI, data and tools

Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...

Screenshot 2024-07-03 at 9.27.29 AM.png
  • 2085 Views
  • 0 replies
  • 0 kudos
DouglasMoore
by Databricks Employee
  • 77 Views
  • 0 replies
  • 3 kudos

Backing up your unity catalog metadata

Unity Catalog (UC) tracks the metadata and your cloud storage accounts store the your data. This python script will extract the metadata from {catalog}.information_schema into folders in a storage location. Take this and put into a notebook.  Make da...

  • 77 Views
  • 0 replies
  • 3 kudos
Ashley1
by Contributor
  • 17679 Views
  • 9 replies
  • 10 kudos

Backup Unity Catalog and managed tables

Hi All, Can anyone point me to either documentation or personally tried and tested method of backing up (and restoring) Unity Catalog and its associated managed tables? We're running on Azure and using ADLS Gen2.Regards,Ashley

  • 17679 Views
  • 9 replies
  • 10 kudos
Latest Reply
DouglasMoore
Databricks Employee
  • 10 kudos

Unity Catalog (UC) tracks the metadata and your cloud storage accounts store the your data. This python script will extract the metadata from {catalog}.information_schema into folders in a storage location. Take this and put into a notebook.Make data...

  • 10 kudos
8 More Replies
Tuno986
by New Contributor
  • 1496 Views
  • 3 replies
  • 0 kudos

Implementing Federated Governance in Databricks Unity Catalog

Hi,I am working for a large company that is implementing a Databricks solution. We have multiple domains, each responsible for its own data products, following a data mesh approach.As part of a federated governance model, we need a way to communicate...

  • 1496 Views
  • 3 replies
  • 0 kudos
Latest Reply
KrithiKalai
Databricks Employee
  • 0 kudos

Please checkout this blog, cover the topic in detail. 

  • 0 kudos
2 More Replies
rashij
by New Contributor II
  • 300 Views
  • 4 replies
  • 0 kudos

How to build model lineage programmatically in a cohesive manner?

If a model is registered in Unity Catalog, then what all APIs and SDKs could be used to build the entire lineage for a model? I'm trying to figure out what all do I query to ensure I don't miss any element of the model lineage.Now a model can have be...

  • 300 Views
  • 4 replies
  • 0 kudos
Latest Reply
LRALVA
Valued Contributor II
  • 0 kudos

@rashij You're tackling an important challenge in ML governance. Building a comprehensive lineage for models registered in Unity Catalog requires piecing together information from multiple APIs. Let me outline a more complete approach based on your i...

  • 0 kudos
3 More Replies
parulpaul
by New Contributor III
  • 6253 Views
  • 7 replies
  • 9 kudos

I do not find "Create Catalog "and "Create Metastore" on Azure Databricks.

I am trying to set up Unity Catalog in Azure Databricks following this documentation Unity Catalog, but I do not find "Create Metastore" and "Create Metastore" on Databricks.Please help with the same, how can I enable Unity Catalog.

  • 6253 Views
  • 7 replies
  • 9 kudos
Latest Reply
parulpaul
New Contributor III
  • 9 kudos

This issue is resolved. Thank you @Camilla Lok​ and @Harun for the suggestions. It works fine after enabling Global administrators in Azure Active Directory .

  • 9 kudos
6 More Replies
AnaMocanu
by Contributor
  • 1207 Views
  • 5 replies
  • 2 kudos

How do you add comment metadata to delta live tables?

Hey all,We've got a bunch of business objects we created as delta live tables. these are important as that's what business users will use in dashboards, genie rooms etc. We're trying to enrich the metadata for those but the option is greyed out and I...

AnaMocanu_0-1740084969911.png
  • 1207 Views
  • 5 replies
  • 2 kudos
Latest Reply
joshuat
New Contributor III
  • 2 kudos

Thanks, but that doesn't answer the question ashraf1395. OP's question was how to create/update the field comments in the Catalog GUI. Also note that I've populated the  `comment=` syntax in my Python DLT definition, but the table description or crea...

  • 2 kudos
4 More Replies
evega
by New Contributor III
  • 17737 Views
  • 12 replies
  • 1 kudos

Azure Create Metastore button not available

Dear community, I want to configure the unity catalog with databricks, but the Create MetaStore button does not appear in my Azure Databricks workspace.In the community, I have seen that several users present the same situation ... What is the reason...

  • 17737 Views
  • 12 replies
  • 1 kudos
Latest Reply
kbaig109
New Contributor II
  • 1 kudos

If I am the owner of the Azure Subcription and databricks premium is in my resource group that makes me owner on that RG as well as Databricks i am still not able to see Metastore setting for unity catalog .. have someone found a resolution to this. ...

  • 1 kudos
11 More Replies
AdamMcGuinness
by New Contributor III
  • 35882 Views
  • 5 replies
  • 1 kudos

Metastore - One per Account/Region Limitation

Looking at Databricks’ suggested use of catalogs. My instincts are now leading me to the conclusion having separate metastore for each SDLC environment (dev, test, prod) is preferable. I think if this pattern were followed, this means due to current ...

  • 35882 Views
  • 5 replies
  • 1 kudos
Latest Reply
Andrius
New Contributor II
  • 1 kudos

Glad to see its not just me thinking about multiple metastores. Separate metastores by environment makes total sense. This would have complete isolation between environments, also in your dev, stg, prod you can reuse catalog names without having to u...

  • 1 kudos
4 More Replies
Tommabip
by New Contributor III
  • 3826 Views
  • 1 replies
  • 0 kudos

Resolved! Privileges

I'm working out a terraform script that  creates a job that needs to write files in the volume.When the job runs as me (I have full privileges), it encounters the following error if running with job compute:  Permission error: operation not permitted...

  • 3826 Views
  • 1 replies
  • 0 kudos
Latest Reply
Tommabip
New Contributor III
  • 0 kudos

I think I found the solution, you need to specify the data_security_mode parameter as SINGLE_USER to grant access to the Unity Catalog

  • 0 kudos
SHG97
by New Contributor II
  • 4625 Views
  • 6 replies
  • 10 kudos

Databricks master data management capabilities

Hi there,Please, I am trying to understand if Databricks is able to support master data management capabilities. Particularly, focusing on the following ones:- Integrate and link different data systems: Connect various systems and make sure the data ...

  • 4625 Views
  • 6 replies
  • 10 kudos
Latest Reply
Haritha_Sama
Contributor II
  • 10 kudos

We at Frisco Analytics are a built on partner for Databricks and built and MDM application that is native to Databricks. https://www.lakefusion.ai/ Please contact us at contact@friscoanalytics.com .Thank you!Haritha

  • 10 kudos
5 More Replies
ajbush
by New Contributor III
  • 12988 Views
  • 2 replies
  • 1 kudos

Unity Catalog permissions approach for strict controls when creating tables

Hi all,I'm deploying Unity Catalog into a large enterprise with quite strict controls. I want to give my users some autonomy over creating tables from upstream datasets they have select access on, but also restrict these controls. I've been through a...

  • 12988 Views
  • 2 replies
  • 1 kudos
Latest Reply
noorbasha534
Contributor III
  • 1 kudos

@ajbush did you workout your requirements in a way you wanted? appreciate if you can share insights here.

  • 1 kudos
1 More Replies
Akshith_Rajesh
by New Contributor III
  • 3849 Views
  • 5 replies
  • 4 kudos

Get the thrift hive.metastore.uri for Databricks unity catalog

I am trying to connect to Unity catalog meta store tables using Presto Based on the presto documentation I need to use the below configuration to connect to delta tables in the unity catalog https://prestodb.io/docs/current/connector/hive.htmlSo from...

  • 3849 Views
  • 5 replies
  • 4 kudos
Latest Reply
MathieuDB
Databricks Employee
  • 4 kudos

Hello @December , You should contact your account team if you need more info on this feature which is not GA for the time being. Another option would be to use Iceberg REST interface: https://medium.com/@flyws1993/oss-trino-to-read-tables-from-databr...

  • 4 kudos
4 More Replies
Subhasis
by New Contributor III
  • 4584 Views
  • 1 replies
  • 0 kudos

Unity Catalog

How can I get the unity catalog code base?

  • 4584 Views
  • 1 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

https://github.com/unitycatalog/unitycatalogThis is the OSS version of course.  The databricks version is not available afaik.

  • 0 kudos
Daithi
by New Contributor II
  • 29619 Views
  • 17 replies
  • 3 kudos

Unity Catalog - Error getting sample data in data explorer

I get an error message saying I Error getting sample data, when I try to view sample data from a table in a schema I created in a Unity Catalog. I dropped the schema and table and got a collague to recreate and still the same message. We are both Uni...

image.png
  • 29619 Views
  • 17 replies
  • 3 kudos
Latest Reply
bentennyysonn
New Contributor II
  • 3 kudos

Check your table level ACLs, refresh your session, and verify role inheritance. Even if permissions look the same, there might be a small difference causing the issue. Try comparing your access settings with your colleague’s to spot any discrepancies...

  • 3 kudos
16 More Replies
RamaTeja
by New Contributor II
  • 21498 Views
  • 15 replies
  • 4 kudos

In Azure Databricks Delta Lake not able to see unity catalog databases or tables in the drop down.

I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not...

  • 21498 Views
  • 15 replies
  • 4 kudos
Latest Reply
sijopchalil2023
New Contributor II
  • 4 kudos

Any further updates on this? I also have the same problem. I tried above solutions, but no luck. By default, it shows only default and it's tables and I can preview those tables. If I enter the <catalog>.<schema> manually, it populates all the tables...

  • 4 kudos
14 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels