cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Governance

Forum Posts

-werners-
by Esteemed Contributor III
  • 3548 Views
  • 6 replies
  • 20 kudos

Resolved! Unity catalog: separate workspace or not?

Now that Unity Catalog is GA, I was wondering how y'all implemented it (or planning to do so) on a workspace level.Do you create a separate 'one workspace to rule them all' with the Unity Catalog, or do you create it on an already existing workspace ...

  • 3548 Views
  • 6 replies
  • 20 kudos
Latest Reply
Pat
Honored Contributor III
  • 20 kudos

Are you thinking about creating totally separate workspace just to manage Unity Catalog?I've been playing a bit with the Unity Catalog and I am currently thinking about managing it from one of the existing workspaces. I don't think there is need for ...

  • 20 kudos
5 More Replies
karthik_p
by Esteemed Contributor
  • 914 Views
  • 1 replies
  • 5 kudos

upgrading external table to unity catalog failing with below error

Is any one came across below issue, based on azure external table upgrade article. what ever tables are being supported in unity catalog they can get upgraded. but other than below "UNSUPPORTED_FILE_SCHEME" Indication, we are not seeing much informat...

  • 914 Views
  • 1 replies
  • 5 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 5 kudos

Hi @karthik p​ you can raise a support request to databricks they will let you know the limitation of this.You can use the below link for general QnA-Link

  • 5 kudos
Gilg
by Contributor II
  • 1208 Views
  • 2 replies
  • 0 kudos

Delta Sharing within the same region

Hi Team,I have setup UC and have a Metastore for AUE. Assigned 2 workspaces to the AUE Metastore and both with different set of users and group . 1 workspace is the data provider and the other is a data consumerNow since 2 workspaces are in one Meta...

  • 1208 Views
  • 2 replies
  • 0 kudos
Latest Reply
youssefmrini
Honored Contributor III
  • 0 kudos

If the two Workspaces are under the same metastore you don't need Delta Sharing because they already have access to the same data. Unless you want to do data segregation per workspace. if it's the case it's coming soon at the end of february

  • 0 kudos
1 More Replies
Ajay-Pandey
by Esteemed Contributor III
  • 3853 Views
  • 6 replies
  • 9 kudos

Resolved! Delta sharing without unity catalog

Hi all,Is there any way to implement delta sharing in databricks without enabling the unity catalog?

  • 3853 Views
  • 6 replies
  • 9 kudos
Latest Reply
Rishabh-Pandey
Honored Contributor II
  • 9 kudos

hey @Ajay Pandey​ refer this link for better understanding https://docs.databricks.com/data-sharing/index.html

  • 9 kudos
5 More Replies
AnthonyDang
by New Contributor II
  • 6187 Views
  • 2 replies
  • 2 kudos

Resolved! Unity catalog set up - unable to create table

Hi all, I've gone through the deployment guide to set up Unity catalog as outlined in https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started.I tried creating a table once I had it set up with the below command.CR...

  • 6187 Views
  • 2 replies
  • 2 kudos
Latest Reply
AnthonyDang
New Contributor II
  • 2 kudos

Legend! This solved it for me. I kept checking permissioning issues but I missed a key step in setting up the storage account in the following guide: https://learn.microsoft.com/en-us/azure/storage/blobs/create-data-lake-storage-account.

  • 2 kudos
1 More Replies
YSF
by New Contributor III
  • 2684 Views
  • 4 replies
  • 1 kudos

Unity Catalog with unstructured "lake" data? (.png, .ggzip, .zarr)

I have images, zipped files, and arbitrary binary files in ADLS gen 2. In Unity Catalog I created a storage credential and an external location to the container that contains these files in directories.Is there any way I can view them within Unity Ca...

  • 2684 Views
  • 4 replies
  • 1 kudos
Latest Reply
Chia-Yui_Lee
New Contributor II
  • 1 kudos

Hi, it is a known bug that External Locations do not work for Azure Storage Account root locations. Please create a folder in the root location and use that as the external location.With the External Location and its permissions properly set up, you ...

  • 1 kudos
3 More Replies
karthik_p
by Esteemed Contributor
  • 1036 Views
  • 3 replies
  • 5 kudos

unity catalog external migration and downtime

HI Team,once after implementing unity catalog and start migrating external tables from legacy hive to unity catalog, i am seeing in articles that we need to make changes to our workloads to be in sync with 3 level name spaceeg: if i have 50 notebooks...

  • 1036 Views
  • 3 replies
  • 5 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 5 kudos

@Aviral Bhardwaj​ this will be normal priority and it takes lot of time to get response.team, we are looking for downtime possibilities and how to handle or change workflows or notebooks to use unity catalog 3 level namespace. do we have any utility ...

  • 5 kudos
2 More Replies
vpismarev
by New Contributor
  • 1044 Views
  • 1 replies
  • 0 kudos

The issue with SQL-endpoint + external hive metastore

HiWe have faced an issue recently (everything worked well 1 week before)We are using exnernal hive metastore:"spark.hadoop.datanucleus.connectionPool.maxPoolSize" : "10"   "spark.sql.hive.metastore.jars" : "/dbfs/ivyjars/*"   "spark.sql.hive.metastor...

  • 1044 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 0 kudos

Generally, this should not be a case. Please raise a support request to databricks

  • 0 kudos
User16868770416
by Contributor
  • 1165 Views
  • 1 replies
  • 2 kudos
  • 1165 Views
  • 1 replies
  • 2 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 2 kudos

I dont know what is TAClbut in the unity catalog you can have governance like catalog.database.table featurewhere you have to explicitly give access to the databricks data user so you have to give him access separately for catalog and database and ta...

  • 2 kudos
parthsalvi
by Contributor
  • 1501 Views
  • 1 replies
  • 2 kudos

getArgument() is not working in DBR 10.4 LTS (Shared Mode) with Unity Catalog

 We're trying to use Notebook widgets in DBR LTS 10.4 (Shared Mode) with Unity Catalogbut we're running into following error.Because of this our notebook dashboard are not working after migrating our data to Unity catalog.Following is the queryCREATE...

image image
  • 1501 Views
  • 1 replies
  • 2 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 2 kudos

unity catalog is different and Data science panel is different , in the above notebook you have to use formatted or this widget will work there and in the unity catalog you have to use get argument formatted in the query

  • 2 kudos
Jitu
by New Contributor II
  • 28804 Views
  • 6 replies
  • 3 kudos
  • 28804 Views
  • 6 replies
  • 3 kudos
Latest Reply
Chaitanya_Raju
Honored Contributor
  • 3 kudos

@Jog Giri​  I also recently encountered a similar scenario, the below code solved my purpose without any issues.import zipfile for i in dbutils.fs.ls('/mnt/zipfilespath/'): with zipfile.ZipFile(i.path.replace('dbfs:','/dbfs'), mode="r") as zip_ref:...

  • 3 kudos
5 More Replies
SanjayJanardhan
by New Contributor III
  • 2067 Views
  • 4 replies
  • 9 kudos

Unity Catalog - Metastore is it at account or workspace

I was trying to understand Unity Catalog, if metastore is at the account level or the each workspace in the account will have there own metastore? My understanding is the metastore cant be shared between two workspace.

  • 2067 Views
  • 4 replies
  • 9 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 9 kudos

@Sanjay Janardhan​ , no i have done this, you can share it in multiple workspaces

  • 9 kudos
3 More Replies
seberino
by New Contributor III
  • 1426 Views
  • 3 replies
  • 4 kudos

How prevent users from being able to create NEW databases? Possible without Unity Catalog?

How prevent users from being able to create NEW databases? Possible without Unity Catalog?I'm working on access control and I know how to remove select and modify privileges on schemas and tables but that's it.

  • 1426 Views
  • 3 replies
  • 4 kudos
Latest Reply
Vartika
Moderator
  • 4 kudos

Hi @Christian Seberino​,Hope everything is going great.Thank you for reaching out to the community with your question. Would you be happy to mark an answer as best so that other members can find the solution more quickly?Cheers!

  • 4 kudos
2 More Replies
apingle
by Contributor
  • 4234 Views
  • 3 replies
  • 4 kudos

Resolved! DLT not working with Unity Catalog

Hey guys,I am trying to create a delta live table in Unity Catalog as follows:CREATE OR REFRESH STREAMING LIVE TABLE <catalog>.<db>.<table_name> AS SELECT ... However, I get the error: org.apache.spark.sql.AnalysisException: Unsupported SQL statemen...

  • 4234 Views
  • 3 replies
  • 4 kudos
Latest Reply
Harun
Honored Contributor
  • 4 kudos

@Anant Pingle​  As of right now, DLT doesn't support Unity Catalog yet. We can expect this feature in upcomig days.

  • 4 kudos
2 More Replies
190809
by Contributor
  • 1673 Views
  • 2 replies
  • 2 kudos

Getting Access forbidden error to AWS when trying to follow the unity catalog quickstart notebook.

HiI have just created a metastore to allow unity catalog to be enabled for my workspace. However I am getting the following error when trying to create a table as per the unity catalog demo notbook:Error in SQL statement: UnauthorizedAccessException:...

  • 1673 Views
  • 2 replies
  • 2 kudos
Latest Reply
190809
Contributor
  • 2 kudos

Thanks for your answer to the second part of my question, super helpful! Can you confirm the first part of my question as well please?

  • 2 kudos
1 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels