cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Governance

Forum Posts

MadelynM
by Databricks Employee
  • 691 Views
  • 0 replies
  • 0 kudos

[Recap] Data + AI Summit 2024 - Data Governance | Navigate the explosion of AI, data and tools

Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...

Screenshot 2024-07-03 at 9.27.29 AM.png
  • 691 Views
  • 0 replies
  • 0 kudos
Matt101122
by Contributor
  • 2646 Views
  • 2 replies
  • 9 kudos

Resolved! Unity Catalog Setup: Why must the first Azure Databricks account admin must be an Azure Active Directory Global Administrator at the time that they first log in to the Azure Databricks account console?

We are attempting to setup Unity Catalog and our security team is requesting justification on why this level of access is required. Why must the first Azure Databricks account admin must be an Azure Active Directory Global Administrator at the time t...

  • 2646 Views
  • 2 replies
  • 9 kudos
Latest Reply
LandanG
Databricks Employee
  • 9 kudos

Hi @Matthew Dalesio​ From our eng. team:"The high privileged is only used to make sure only highly privileged users get access to Databricks account admin role as this is a highly-privileged role and they can make anyone else an account admin. This i...

  • 9 kudos
1 More Replies
-werners-
by Esteemed Contributor III
  • 4540 Views
  • 6 replies
  • 20 kudos

Resolved! Unity catalog: separate workspace or not?

Now that Unity Catalog is GA, I was wondering how y'all implemented it (or planning to do so) on a workspace level.Do you create a separate 'one workspace to rule them all' with the Unity Catalog, or do you create it on an already existing workspace ...

  • 4540 Views
  • 6 replies
  • 20 kudos
Latest Reply
Pat
Honored Contributor III
  • 20 kudos

Are you thinking about creating totally separate workspace just to manage Unity Catalog?I've been playing a bit with the Unity Catalog and I am currently thinking about managing it from one of the existing workspaces. I don't think there is need for ...

  • 20 kudos
5 More Replies
karthik_p
by Esteemed Contributor
  • 1116 Views
  • 1 replies
  • 5 kudos

upgrading external table to unity catalog failing with below error

Is any one came across below issue, based on azure external table upgrade article. what ever tables are being supported in unity catalog they can get upgraded. but other than below "UNSUPPORTED_FILE_SCHEME" Indication, we are not seeing much informat...

  • 1116 Views
  • 1 replies
  • 5 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 5 kudos

Hi @karthik p​ you can raise a support request to databricks they will let you know the limitation of this.You can use the below link for general QnA-Link

  • 5 kudos
AnthonyDang
by New Contributor II
  • 7115 Views
  • 2 replies
  • 2 kudos

Resolved! Unity catalog set up - unable to create table

Hi all, I've gone through the deployment guide to set up Unity catalog as outlined in https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started.I tried creating a table once I had it set up with the below command.CR...

  • 7115 Views
  • 2 replies
  • 2 kudos
Latest Reply
AnthonyDang
New Contributor II
  • 2 kudos

Legend! This solved it for me. I kept checking permissioning issues but I missed a key step in setting up the storage account in the following guide: https://learn.microsoft.com/en-us/azure/storage/blobs/create-data-lake-storage-account.

  • 2 kudos
1 More Replies
YSF
by New Contributor III
  • 3345 Views
  • 4 replies
  • 1 kudos

Unity Catalog with unstructured "lake" data? (.png, .ggzip, .zarr)

I have images, zipped files, and arbitrary binary files in ADLS gen 2. In Unity Catalog I created a storage credential and an external location to the container that contains these files in directories.Is there any way I can view them within Unity Ca...

  • 3345 Views
  • 4 replies
  • 1 kudos
Latest Reply
Chia-Yui_Lee
Databricks Employee
  • 1 kudos

Hi, it is a known bug that External Locations do not work for Azure Storage Account root locations. Please create a folder in the root location and use that as the external location.With the External Location and its permissions properly set up, you ...

  • 1 kudos
3 More Replies
karthik_p
by Esteemed Contributor
  • 1423 Views
  • 3 replies
  • 5 kudos

unity catalog external migration and downtime

HI Team,once after implementing unity catalog and start migrating external tables from legacy hive to unity catalog, i am seeing in articles that we need to make changes to our workloads to be in sync with 3 level name spaceeg: if i have 50 notebooks...

  • 1423 Views
  • 3 replies
  • 5 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 5 kudos

@Aviral Bhardwaj​ this will be normal priority and it takes lot of time to get response.team, we are looking for downtime possibilities and how to handle or change workflows or notebooks to use unity catalog 3 level namespace. do we have any utility ...

  • 5 kudos
2 More Replies
vpismarev
by New Contributor
  • 1552 Views
  • 1 replies
  • 0 kudos

The issue with SQL-endpoint + external hive metastore

HiWe have faced an issue recently (everything worked well 1 week before)We are using exnernal hive metastore:"spark.hadoop.datanucleus.connectionPool.maxPoolSize" : "10"   "spark.sql.hive.metastore.jars" : "/dbfs/ivyjars/*"   "spark.sql.hive.metastor...

  • 1552 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 0 kudos

Generally, this should not be a case. Please raise a support request to databricks

  • 0 kudos
User16868770416
by Contributor
  • 1535 Views
  • 1 replies
  • 2 kudos
  • 1535 Views
  • 1 replies
  • 2 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 2 kudos

I dont know what is TAClbut in the unity catalog you can have governance like catalog.database.table featurewhere you have to explicitly give access to the databricks data user so you have to give him access separately for catalog and database and ta...

  • 2 kudos
parthsalvi
by Contributor
  • 1826 Views
  • 1 replies
  • 2 kudos

getArgument() is not working in DBR 10.4 LTS (Shared Mode) with Unity Catalog

 We're trying to use Notebook widgets in DBR LTS 10.4 (Shared Mode) with Unity Catalogbut we're running into following error.Because of this our notebook dashboard are not working after migrating our data to Unity catalog.Following is the queryCREATE...

image image
  • 1826 Views
  • 1 replies
  • 2 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 2 kudos

unity catalog is different and Data science panel is different , in the above notebook you have to use formatted or this widget will work there and in the unity catalog you have to use get argument formatted in the query

  • 2 kudos
Jitu
by New Contributor II
  • 34209 Views
  • 6 replies
  • 3 kudos
  • 34209 Views
  • 6 replies
  • 3 kudos
Latest Reply
Chaitanya_Raju
Honored Contributor
  • 3 kudos

@Jog Giri​  I also recently encountered a similar scenario, the below code solved my purpose without any issues.import zipfile for i in dbutils.fs.ls('/mnt/zipfilespath/'): with zipfile.ZipFile(i.path.replace('dbfs:','/dbfs'), mode="r") as zip_ref:...

  • 3 kudos
5 More Replies
SanjayJanardhan
by New Contributor III
  • 2861 Views
  • 4 replies
  • 9 kudos

Unity Catalog - Metastore is it at account or workspace

I was trying to understand Unity Catalog, if metastore is at the account level or the each workspace in the account will have there own metastore? My understanding is the metastore cant be shared between two workspace.

  • 2861 Views
  • 4 replies
  • 9 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 9 kudos

@Sanjay Janardhan​ , no i have done this, you can share it in multiple workspaces

  • 9 kudos
3 More Replies
seberino
by New Contributor III
  • 1846 Views
  • 3 replies
  • 4 kudos

How prevent users from being able to create NEW databases? Possible without Unity Catalog?

How prevent users from being able to create NEW databases? Possible without Unity Catalog?I'm working on access control and I know how to remove select and modify privileges on schemas and tables but that's it.

  • 1846 Views
  • 3 replies
  • 4 kudos
Latest Reply
Vartika
Databricks Employee
  • 4 kudos

Hi @Christian Seberino​,Hope everything is going great.Thank you for reaching out to the community with your question. Would you be happy to mark an answer as best so that other members can find the solution more quickly?Cheers!

  • 4 kudos
2 More Replies
apingle
by Contributor
  • 5431 Views
  • 3 replies
  • 4 kudos

Resolved! DLT not working with Unity Catalog

Hey guys,I am trying to create a delta live table in Unity Catalog as follows:CREATE OR REFRESH STREAMING LIVE TABLE <catalog>.<db>.<table_name> AS SELECT ... However, I get the error: org.apache.spark.sql.AnalysisException: Unsupported SQL statemen...

  • 5431 Views
  • 3 replies
  • 4 kudos
Latest Reply
Harun
Honored Contributor
  • 4 kudos

@Anant Pingle​  As of right now, DLT doesn't support Unity Catalog yet. We can expect this feature in upcomig days.

  • 4 kudos
2 More Replies
190809
by Contributor
  • 2356 Views
  • 2 replies
  • 2 kudos

Getting Access forbidden error to AWS when trying to follow the unity catalog quickstart notebook.

HiI have just created a metastore to allow unity catalog to be enabled for my workspace. However I am getting the following error when trying to create a table as per the unity catalog demo notbook:Error in SQL statement: UnauthorizedAccessException:...

  • 2356 Views
  • 2 replies
  • 2 kudos
Latest Reply
190809
Contributor
  • 2 kudos

Thanks for your answer to the second part of my question, super helpful! Can you confirm the first part of my question as well please?

  • 2 kudos
1 More Replies
bob1
by New Contributor III
  • 2259 Views
  • 4 replies
  • 5 kudos

Error with federation for Snowflake in Databricks SQL

Hello, I am trying to test the new federation for Snowflake feature in Databricks SQL, following the docs (https://docs.databricks.com/query-federation/snowflake.html).Environment:Subscription: PremiumCompute: DB SQL (serverless)Metastore: Unity Cata...

  • 2259 Views
  • 4 replies
  • 5 kudos
Latest Reply
bob1
New Contributor III
  • 5 kudos

Hi @Landan George​, thanks for the quick response. I get the following error if I try to run the SQL code from a Notebook:SQLException: No suitable driver found for jdbc:snowflake://app.snowflake.com/region/foo/Any suggestions?

  • 5 kudos
3 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels