cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Governance

Forum Posts

Garik
by New Contributor III
  • 3865 Views
  • 7 replies
  • 10 kudos

Issue with Delta Live tables access from workflow ( dlt pipelines )

Hi all,I am trying to write data from external s3 bucket to delta live tables at unity catalog( not dbfs ) from a workflow. I am seeing the following error ( saying catalog namespace is not supported. Please check full error below ) Error Detailspy4...

  • 3865 Views
  • 7 replies
  • 10 kudos
Latest Reply
Sulfikkar
Contributor
  • 10 kudos

How long should we wait to use DLT on top of the Unity catalog?

  • 10 kudos
6 More Replies
Robin_LOCHE
by New Contributor II
  • 2653 Views
  • 2 replies
  • 0 kudos

Error 'Missing Credential Scope' when using R Sparklyr on runtime > 10.4

We have multiple processing chains that uses R notebooks with Sparklyr, and we try to migrate them from runtime 10.4 to 12.2. unfortunately, there seems to be an incompatibility with Sparlyr with runtimes > 10.4:Steps to reproduce:1) Create a noteboo...

  • 2653 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Robin LOCHE​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 0 kudos
1 More Replies
ashish577
by New Contributor III
  • 2663 Views
  • 3 replies
  • 0 kudos

Is it possible to query databricks unity catalog/glue catalog delta tables from presto?

Hi,I am trying to setup presto in aws emr to be able to read delta tables. For this, as per my understanding, I need to setup the delta.properties file. Does anybody know how do I find the hive.metastore.uri that is needed for this file? For either u...

  • 2663 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

To find the hive.metastore.uri needed for the delta.properties file, you can follow the below steps:Login to your AWS Glue consoleClick on the "Crawlers" tab and select the Glue crawler that you are using for your Delta tablesIn the "Crawler details"...

  • 0 kudos
2 More Replies
Guilherme_Jorda
by New Contributor
  • 1283 Views
  • 3 replies
  • 2 kudos

Unable to link/assign a unity catalog metastore to a workspace

We are following these instructions to link/assign a unity catalog metastore to a workspace using the UI. When we apply the option to enable the unity metastore in the workspace configuration, after hitting "Update", the option gets disabled. When we...

  • 1283 Views
  • 3 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Guilherme Jordan​ :If the option to enable the Unity Catalog metastore in the workspace configuration is getting disabled after hitting "Update", it could be a sign of an issue with the workspace or the Unity Catalog metastore.Here are some steps yo...

  • 2 kudos
2 More Replies
kmartin62
by New Contributor III
  • 1438 Views
  • 2 replies
  • 0 kudos

Manage Account redirects me to /workspace-select instead of /data for Unity Catalog

I've been trying to set up a Unity Catalog and play around with it. For that purpose, I created a new workspace with premium tier. As a tutorial, I follow Dustin Vannoy's video (https://www.youtube.com/watch?v=-RwzDRVgjLc) where he goes to his userna...

  • 1438 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Martin Kostadinov​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your fee...

  • 0 kudos
1 More Replies
vlad_proex
by New Contributor
  • 1426 Views
  • 2 replies
  • 0 kudos

Cannot activate Unity Catalog on Databricks AWS Premium

Hello. Despite having Databricks Premium, I cannot find the options to work with Unity Catalog. I login as administrator, go to the Data section, but cannot find the option to create a Unity Metastore. What am I missing?

  • 1426 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Vlad Gheorghe​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...

  • 0 kudos
1 More Replies
jmill
by New Contributor
  • 2342 Views
  • 4 replies
  • 3 kudos

What is best practice for organising simple desktop-style analytics workflows in Databricks?

Apologies in advance for the soft question, but I'm genuinely struggling with this.We're a small data science unit just setting up in Databricks. While we do run some intensive ETL and analytics jobs, a non-trivial part of the team's BAU is explorato...

  • 2342 Views
  • 4 replies
  • 3 kudos
Latest Reply
Vartika
Moderator
  • 3 kudos

Hi @Jason Millburn​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...

  • 3 kudos
3 More Replies
Vivek_Singh
by New Contributor III
  • 951 Views
  • 2 replies
  • 0 kudos

How to update change data UnityCatalog table after migrating hive metastore delta table

If I will migrate the Hive_metastore table into Unity Catalog table and subsequently hive metastore table is target of curation process. how I will migrate the delta record from Hive_metastore table to Unity Catalog table

  • 951 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vartika
Moderator
  • 0 kudos

Hi @Vivek Singh Banafar​ Hope everything is going great.Just checking in. Does @Debayan Mukherjee​'s response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to he...

  • 0 kudos
1 More Replies
Lp6440
by New Contributor II
  • 1746 Views
  • 5 replies
  • 0 kudos

Unity catalog - access control to DB and workspace

After implementing Unity catalog, assuming a user has access to more than one DB. Can that user access all the DB in a single workspace? Can we restrict any specific DB in a workspace?

  • 1746 Views
  • 5 replies
  • 0 kudos
Latest Reply
Vartika
Moderator
  • 0 kudos

Hi @Prashant Lalaiya​,Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.Cheers!

  • 0 kudos
4 More Replies
Barney
by New Contributor II
  • 1794 Views
  • 2 replies
  • 0 kudos

Metastore down events in Databricks Workspace

Hi all,I see a number of metastore down events in databricks workspace: do any one know what's the reason why this happens?Thanks,Barney

  • 1794 Views
  • 2 replies
  • 0 kudos
Latest Reply
pvignesh92
Honored Contributor
  • 0 kudos

Hi @Ron Barneveld​ Could you kindly help with more details? Is this is a production metastore that you are mentioning or any metastore created by you for test environment. Also where it is hosted?

  • 0 kudos
1 More Replies
venkad
by Contributor
  • 4678 Views
  • 3 replies
  • 3 kudos

dbutils.fs.ls doesn't work on Unity catalog managed location

Hi,dbutils.fs.ls() is a most useful utility to list the files in remote cloud storage. I cannot use this function on top of the location managed by the Unity catalog.I have a managed external location created on `'abfss://xyz@abcd.dfs.core.windows.ne...

  • 4678 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Venkadeshwaran K​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...

  • 3 kudos
2 More Replies
munjal_prerna
by New Contributor II
  • 3156 Views
  • 3 replies
  • 3 kudos

Problem in enabling unity catalog

I have databricks premium and I want to enable unity catalog on it, I am not able to do so. I followed the following steps which are attached in the attachments. I am also attaching the screenshot of my cluster configuration. When I run this command:...

  • 3156 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Prerna Munjal​ Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!Regards

  • 3 kudos
2 More Replies
Chris_Shehu
by Valued Contributor III
  • 6681 Views
  • 5 replies
  • 2 kudos

Resolved! Are their plans to support init scripts within shared compute resources?

Currently we have an Init script that specifies the location of different network resources by updating the hosts file. When we moved to the Unity catalog all of our clusters had to be upgraded to support it. Now our scripts have stopped working due ...

  • 6681 Views
  • 5 replies
  • 2 kudos
Latest Reply
youssefmrini
Honored Contributor III
  • 2 kudos

We are bringing soon init script and Scala to shared Cluster

  • 2 kudos
4 More Replies
Avvar2022
by Contributor
  • 6217 Views
  • 4 replies
  • 1 kudos

Resolved! I am new to Data bricks. Setting up Data bricks Unity Catalog, in terms of best practice i have few questions.

Is it best practice to separate unity catalog meta store ADLS Gen2 separate from ADLS Gen 2 to store data ?Since per region only one meta store can be created, will there be a separate meta store for PROD, and NON-PROD(QA and DEV)? If yes they need t...

  • 6217 Views
  • 4 replies
  • 1 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 1 kudos

@Ashok Zubrewar​  coming to your 3 rd question, if you are using any external tables then non uc ADLS GEN 2 is mandatory, you can not use UC ADLS GEN2. as it hosts metadata and managed table data. there is no restriction in terms of your external buc...

  • 1 kudos
3 More Replies
ossinova
by Contributor II
  • 12247 Views
  • 4 replies
  • 3 kudos

Resolved! Best practice for organizing data objects using Unity Catalog

I was wondering if there is a best practice or recommended way to organize data objects (tables) in Unity Catalog.Following the medallion architecture, how should one organize and name these three namespaces:Catalog(s)Schema(s)Table(s)Example:Catalog...

  • 12247 Views
  • 4 replies
  • 3 kudos
Latest Reply
Avvar2022
Contributor
  • 3 kudos

In Real time do we have 1 meta store for PROD and NON-PROD ? if Yes in the backend there would be a one storage account (ADLS Gen2) > Container for all Environments.

  • 3 kudos
3 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels