cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MadelynM
by Databricks Employee
  • 1081 Views
  • 0 replies
  • 0 kudos

[Recap] Data + AI Summit 2024 - Data Governance | Navigate the explosion of AI, data and tools

Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...

Screenshot 2024-07-03 at 9.27.29 AM.png
  • 1081 Views
  • 0 replies
  • 0 kudos
pernilak
by New Contributor III
  • 418 Views
  • 1 replies
  • 0 kudos

Permissions needed for creating a new (delta) share

What permissions are actually needed in order to create a delta share?I have read that you either need to be metastore admin, or aUser with delegated permissions or ownership on both the share and the recipient objects ((USE SHARE + SET SHARE PERMISS...

  • 418 Views
  • 1 replies
  • 0 kudos
Latest Reply
MoJaMa
Databricks Employee
  • 0 kudos

Hi,  Hope the doc link helps: https://docs.databricks.com/en/delta-sharing/create-share.html#requirements Specifically this statement should alleviate your immediate point. Have the USE CATALOG and USE SCHEMA privilege on the catalog and schema that...

  • 0 kudos
nccodemonkey
by New Contributor III
  • 50413 Views
  • 19 replies
  • 10 kudos

Unity Catalog Shared Access Mode - dbutils.notebook.entry_point...getContext() not whitelisted

we are switching over to Unity Catalog and attempting to confirm the ability to run our existing notebooks. I have created a new Shared Unity Catalog Cluster and ran the notebook using the new cluster. Ran into an error attempting to execute a prin...

  • 50413 Views
  • 19 replies
  • 10 kudos
Latest Reply
DouglasMoore
Databricks Employee
  • 10 kudos

To replace dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson() and related APIs try:from dbruntime.databricks_repl_context import get_context token = get_context().apiToken host = get_context().browserHostName get_context().__d...

  • 10 kudos
18 More Replies
andalo
by New Contributor II
  • 804 Views
  • 2 replies
  • 1 kudos

Unity Catalog lineage is lost

Currently, we have implemented Unity Catalog in Databricks and we are facing an issue with lineage. We execute a notebook that fills a table, and upon completion, the lineage is created correctly; all tables are mapped properly, and we can visualize ...

  • 804 Views
  • 2 replies
  • 1 kudos
Latest Reply
gchandra
Databricks Employee
  • 1 kudos

What does your notebook do? Does it drop the existing tables and create a new one? Give more details about your process preferably the source code.

  • 1 kudos
1 More Replies
owlleg6
by New Contributor III
  • 15913 Views
  • 7 replies
  • 3 kudos

Models signature inaccessible in Unity Catalog

I am using Unity Catalog to store my models. My Unity Catalog meta-store is hosted in an Azure Storage Account, which has public network access disabled. Access is restricted to certain IP ranges and private endpoint connections only.Recently, my dev...

Data Governance
azure
models
networking
Unity Catalog
  • 15913 Views
  • 7 replies
  • 3 kudos
Latest Reply
myroslava
New Contributor II
  • 3 kudos

Got issue fixed:The issue was with Private Endpoint: we had PE with dfs sub-resource only, but mlflow uses blob, so when I created PE with blob sub-resource - the issue was fixed. (also blob DNS zone and network link is needed)

  • 3 kudos
6 More Replies
rkalluri-apex
by New Contributor III
  • 2759 Views
  • 1 replies
  • 0 kudos

Cannot create External Location due to HNS on Azure Datarbricks

I have run into an issue with creating an External Location on an ASLS storage container with the following error  :"Hierarchical Namespace (HNS) not enabledAzure storage account does not have hierarchical namespace (HNS) enabled." The Storage accoun...

  • 2759 Views
  • 1 replies
  • 0 kudos
Latest Reply
mt0924
New Contributor II
  • 0 kudos

I ran into the same issue, after enabling the HNS, still getting this error.

  • 0 kudos
TSK
by New Contributor
  • 532 Views
  • 1 replies
  • 0 kudos

Set up GitLab server on Databricks Container Services (DCS)

I would like to set up GitLab and Grafana servers using Databricks Container Services (DCS). The reason is that our development team is small, and the management costs of using EKS are not justifiable. We want to make GitLab and Grafana accessible in...

  • 532 Views
  • 1 replies
  • 0 kudos
Latest Reply
ameame
New Contributor II
  • 0 kudos

 Yes, it’s possible to set up GitLab and Grafana servers using Databricks Container Services (DCS). Here’s a high-level overview of how you can achieve this:Prepare Your Docker Images:Create Docker images for GitLab and Grafana. You can use official ...

  • 0 kudos
jerickson
by New Contributor II
  • 1256 Views
  • 3 replies
  • 0 kudos

CREATE EXTERNAL LOCATION on a publicly available S3 bucket

I would like to create an external location on a publicly available S3 bucket, for which I don't have credentials. I get a syntax error unless I include credentials. Is there a way to do this? CREATE EXTERNAL LOCATION public_bucket URL 's3://public_b...

  • 1256 Views
  • 3 replies
  • 0 kudos
Latest Reply
filipniziol
Contributor III
  • 0 kudos

Hi @jerickson ,I have tested this on Databricks Runtime 14.3 LTS:Install on the cluster the below Maven packages:com.amazonaws:aws-java-sdk-bundle:1.12.262org.apache.hadoop:hadoop-aws:3.3.4Run the below code to read your csv file into dataframe: spar...

  • 0 kudos
2 More Replies
DG1
by New Contributor II
  • 892 Views
  • 0 replies
  • 1 kudos

Data dictionary and glossar capabilities

Hi everyone!I would love to know whether Databricks has the following capabilities and what are the core functionalities of the tool:Data glossary: provides clear definitions of business terms, ensuring that everyone in the organization uses the same...

  • 892 Views
  • 0 replies
  • 1 kudos
jv_v
by Contributor
  • 1874 Views
  • 5 replies
  • 6 kudos

How to Transfer Hive Metastore from One Workspace to Another

Hi everyone,I need to transfer tables present in the Hive metastore from one workspace to another. My ultimate goal is to migrate these tables to Unity Catalog. I already have a code that will handle the migration to Unity Catalog, but I need help wi...

  • 1874 Views
  • 5 replies
  • 6 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 6 kudos

Are these tables managed tables or external tables? This is very important step for your migration. 

  • 6 kudos
4 More Replies
Gilg
by Contributor II
  • 1978 Views
  • 3 replies
  • 0 kudos

Delta Sharing within the same region

Hi Team,I have setup UC and have a Metastore for AUE. Assigned 2 workspaces to the AUE Metastore and both with different set of users and group . 1 workspace is the data provider and the other is a data consumerNow since 2 workspaces are in one Meta...

  • 1978 Views
  • 3 replies
  • 0 kudos
Latest Reply
NikunjKakadiya
New Contributor II
  • 0 kudos

Do we have any updates on this in terms of how we can share the data between different workspaces part of the same metastore?

  • 0 kudos
2 More Replies
Praveen
by New Contributor II
  • 1006 Views
  • 1 replies
  • 0 kudos

Enable external data access on the metastore

Hello Everyone,I have an AWS Databricks premium trial account and I’m trying to enable the configuration "Enable external data access on the metastore" for a metastore. I followed the steps in the official documentation (https://docs.databricks.com/e...

Praveen_0-1727338697909.png
  • 1006 Views
  • 1 replies
  • 0 kudos
Latest Reply
saikumar246
Databricks Employee
  • 0 kudos

Hi @Praveen Thank you for sharing your concern here. You are looking into the Catalog on the account console, but according to the document, you have to do this in the Workspace Catalog UI. Please follow the below steps, Log into any of the workspace...

  • 0 kudos
robbinbajaj
by New Contributor
  • 453 Views
  • 1 replies
  • 0 kudos

Restrict User to see schema using "all purpose compute"

Scenario Under Hive Metastore four diff schema exist S1,S2,S3 and S4 User1 want access to schema S1 but not allowed to see other 3 Schema which can be done using grant permission simple and easy case Same User1 has access to workspace. Now same user ...

  • 453 Views
  • 1 replies
  • 0 kudos
Latest Reply
gchandra
Databricks Employee
  • 0 kudos

The best option for Governance is to upgrade to Unity Catalog.  This is alternate, not the best. The cluster must be enabled for table access control. https://docs.databricks.com/en/data-governance/table-acls/object-privileges.html#requirements

  • 0 kudos
jv_v
by Contributor
  • 2694 Views
  • 1 replies
  • 0 kudos

Issues Creating External Tables in Hive Metastore vs. Unity Catalog

Hi Databricks Community,I’m encountering some challenges when working with external tables in Databricks. I’m using both Unity Catalog and Hive Meta store and have noticed differences in how external tables are handled.Unity Catalog: I can successful...

  • 2694 Views
  • 1 replies
  • 0 kudos
Latest Reply
Imalkov
New Contributor III
  • 0 kudos

hi @jv_v , that's possible to put a path to external storage i.e.:documentationCREATE OR REPLACE TABLE hive_metastore.schema.table LOCATION 's3://depts/finance/sec_filings'Can you share which issue do you have cause it's not clear from your questio...

  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels