cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MadelynM
by Databricks Employee
  • 1077 Views
  • 0 replies
  • 0 kudos

[Recap] Data + AI Summit 2024 - Data Governance | Navigate the explosion of AI, data and tools

Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...

Screenshot 2024-07-03 at 9.27.29 AM.png
  • 1077 Views
  • 0 replies
  • 0 kudos
eduardo_marin_n
by New Contributor
  • 730 Views
  • 1 replies
  • 0 kudos

Cannot find SOC compliance report

Hi,I am trying to obtain SOC compliance documents for my company. Despite following the instructions of the AI assistant, I can't find anywhere to submit a support ticket to get these documents. Has anyone had experience with this issue in the past w...

  • 730 Views
  • 1 replies
  • 0 kudos
Latest Reply
jw-dbx
Databricks Employee
  • 0 kudos

You can contact your Databricks account executive (sales representative), they should be able to get you the copy of SOC compliance reports. 

  • 0 kudos
Mamzy10
by New Contributor II
  • 1348 Views
  • 1 replies
  • 0 kudos

Unity catalog tagging through Code

Hi I was wondering how you would apply tags to tables, columns etc in unity catalog through code?One use case i want to have a look at is by automatically applying PI classification against certain tables and columns that contain certain words.Is thi...

  • 1348 Views
  • 1 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

Using sql:https://docs.databricks.com/en/data-governance/unity-catalog/tags.html#manage-tags-using-sql-commandsI am not aware of a python/scala counterpart (but using spark.sql() that is no issue).

  • 0 kudos
Carol2
by New Contributor II
  • 4858 Views
  • 2 replies
  • 2 kudos

unity catalog vs purview

What is the vision of uc and purview , on government 

  • 4858 Views
  • 2 replies
  • 2 kudos
Latest Reply
Idera
New Contributor II
  • 2 kudos

There's some changes to Purview coming out that enable users to create Data Products in Purview as the start of a Data Fabric which is realized in Databricks. ER/Studio allows you to design structures with well-formed data models of your Databricks L...

  • 2 kudos
1 More Replies
vishal-kilaskar
by New Contributor II
  • 1418 Views
  • 2 replies
  • 0 kudos

Column Data type miss-match while databricks catalog federation

We are facing an issue related to the Data type mismatch when performing the federation queries (link).Basic need is to perform union on these both tables from federated(system tables) and system tables (target/current workspace).Problem:  Struct and...

image (8).png image (7).png image (6).png
Data Governance
Data Governance
Foreign Catalog
Lakehouse Federation
  • 1418 Views
  • 2 replies
  • 0 kudos
Latest Reply
dkushari
Databricks Employee
  • 0 kudos

System table data is available at the account level. If you are trying LH Federation for querying system tables isn't that already available in the source workspace?  https://docs.databricks.com/en/admin/system-tables/index.html#what-are-system-table...

  • 0 kudos
1 More Replies
JonLaRose
by New Contributor III
  • 1285 Views
  • 3 replies
  • 0 kudos

Catalog created using a service principal isn't visible

When creating a Unity catalog using a service principal token, the created catalog isn't visible or accessible to other users (even admins) via GUI nor CLI.It seems that only when I change the catalog's ownership to some other user it is visible.Why ...

  • 1285 Views
  • 3 replies
  • 0 kudos
Latest Reply
dkushari
Databricks Employee
  • 0 kudos

Right after the creation of the catalog, add a new grant statement. Check here please.

  • 0 kudos
2 More Replies
manjeettanwar10
by New Contributor II
  • 2346 Views
  • 1 replies
  • 1 kudos

Can we get the snapshot of the change data feed tables at a given time?

We are trying to build the history snapshot for data that we are ingested in the delta tables. We are exploring different option like delta history, change data feed and dbt snapshot.This Question is particularly about the change data feed. So I want...

  • 2346 Views
  • 1 replies
  • 1 kudos
Latest Reply
raphaelblg
Databricks Employee
  • 1 kudos

Hello @manjeettanwar10 , The Delta History API allows you to view the logs of all SQL statements executed on your table, including UPDATE, DELETE, and INSERT commands. When used in conjunction with Delta's Time Travel features, it's possible to retri...

  • 1 kudos
NadithK
by Contributor
  • 3865 Views
  • 3 replies
  • 0 kudos

Removing ADLS Gen 2 path (metastore root) in Unity Catalog Metastore

In my organization, we are using Databricks unity catalog and we have a metastore created for our region which holds all of our workspaces. When we created the metastore last year, we set a metastore root location for it (If I remember correctly, met...

  • 3865 Views
  • 3 replies
  • 0 kudos
Latest Reply
dkushari
Databricks Employee
  • 0 kudos

If you have a way to reproduce metastore object creation then better to delete this metastore a create a new one with no root metastore bucket. That will also ensure that catalog cannot be created without specifying a location. If you have locked dow...

  • 0 kudos
2 More Replies
designstallion
by New Contributor
  • 3078 Views
  • 1 replies
  • 0 kudos

Digital Marketing Agency NJ - Design StallionDesign Stallion is a web design and digital marketing agency in NJ delivering bespoke custom websites and...

Digital Marketing Agency NJ - Design StallionDesign Stallion is a web design and digital marketing agency in NJ delivering bespoke custom websites and online marketing solutions including SEO, PPC, and more.

solution-img
  • 3078 Views
  • 1 replies
  • 0 kudos
Latest Reply
FeliciaWilliam
Contributor
  • 0 kudos

In my experience with digital marketing, leveraging Empowering Growth through Tech Talent Sloboda has been incredibly effective for both product growth and team scaling. Their expertise in tech talent helped us expand our team efficiently and improve...

  • 0 kudos
ChechuIGZ
by New Contributor II
  • 2228 Views
  • 2 replies
  • 2 kudos

Total isolation of credentials

Hi!Recently we encountered a problem with how Databricks handles secrets that does not meet our compliance.We need total isolation of users credentials but seems like the admin role in databricks totally breaks that since a person with that role can ...

  • 2228 Views
  • 2 replies
  • 2 kudos
Latest Reply
dkushari
Databricks Employee
  • 2 kudos

Hi @ChechuIGZ - The value of the secret is redacted. https://docs.databricks.com/en/security/secrets/redaction.html#secret-redaction.  Also remember the following -  

  • 2 kudos
1 More Replies
Snoonan
by Contributor
  • 15570 Views
  • 4 replies
  • 0 kudos

Masking data

Hi all,I need to mask columns in a table for certain users.What is the best way to go about this in Databricks? My workspace is unity catalog enabled.Thanks,Sean

  • 15570 Views
  • 4 replies
  • 0 kudos
Latest Reply
dkushari
Databricks Employee
  • 0 kudos

In UC you can use column masking. See here - https://docs.databricks.com/en/data-governance/unity-catalog/row-and-column-filters.html#what-are-column-masks

  • 0 kudos
3 More Replies
marvin1
by New Contributor III
  • 2573 Views
  • 2 replies
  • 0 kudos

Hostname redaction in delta table

I am ingesting job-cluster failure notifications that we send to OpsGenie into a delta table to automate the creation and tracking of Jira tickets.  The alert notification includes the job run url, which we use to quickly respond to job failures.  Ho...

  • 2573 Views
  • 2 replies
  • 0 kudos
Latest Reply
marvin1
New Contributor III
  • 0 kudos

I tracked this issue down to running the notebook on a shared Unity-enabled cluster.  When scheduling the notebook to run on a job cluster with single user mode, this issue does not occur.

  • 0 kudos
1 More Replies
jv_v
by Contributor
  • 1784 Views
  • 1 replies
  • 0 kudos

Issue with "databricks metastores list" Command - Only One Metastore Listed

I have encountered with the Databricks CLI command databricks metastores list. As per our account setup, we have three metastores configured. However, when I run the command, it only returns information for one metastore instead of listing all three....

  • 1784 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 0 kudos

Hi @jv_v , Good Day!  The command you are running gives you Metastore details attached to your workspace. If you are looking to list out all the metastore present at the Account console, you can try running this below CLI command:  ------------------...

  • 0 kudos
Nivethan
by New Contributor
  • 1804 Views
  • 2 replies
  • 0 kudos

UC Enablement in Databricks workspace for metastores in different region - Azure Cloud

SpoilerWe have a UC enabled on Databricks where we have two Metastores,Metastore in West Europe regionMetastore in North America regionAnd Databricks workspace.Workspace in West Europe regionAnd we want to have a single Databricks workspace for both ...

  • 1804 Views
  • 2 replies
  • 0 kudos
Latest Reply
dkushari
Databricks Employee
  • 0 kudos

Hi, Cross region metastore and workspace (WS) is not possible. It is important to understand the use case and why you have certain data in a region where you do not have a compute (WS) or do not need a compute. Ideally have each region has their own ...

  • 0 kudos
1 More Replies
erigaud
by Honored Contributor
  • 4002 Views
  • 3 replies
  • 1 kudos

Create a catalog with external location from different Metastore

Hello community !Here's my problem : I had a workspace with Unity Catalog enabled, and a catalog stored on an external location (Azure storage account). All my schemas and tables were storage on this storage. However we recently deleted the metastore...

erigaud_0-1713797409085.png
  • 4002 Views
  • 3 replies
  • 1 kudos
Latest Reply
dkushari
Databricks Employee
  • 1 kudos

Hi, Looks like you are creating the catalog at the container level. Are there any other UC objects created pointing to that? Is the external location created at the container level? Can you please have a look at this and see if there is overlapping? ...

  • 1 kudos
2 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels