Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
no information on character_maximum_length and is_identity in unity catalog table system.information_schema.columnstable definition exampleCREATE TABLE tyable_name ( UniqueID bigint not null GENERATED ALWAYS AS IDENTITY , ID varchar(256))looks like ...
Hello Team,Still i am facing same issue of value zero in character_maximum_length. Is there any other way we can get character_maximum_length of columns?When we can expect to resolve this bug?
We are attempting to create a comprehensive Data Governance solution using Unity Catalog & INFA CDGC tool. The objective is to onboard the business assets on the Informatica DG platform and use the unity catalog to trace technical assets and lineage ...
Yes, Informatica now has native integration to Unity Catalog with its cloud data catalogue and governance service. You can scan Unity catalog Metadata and propagate this in CDGC and apply policies, business terms, overlay data quality scores, display...
We have been successfully managing access for our unity catalogs using the databricks_grant resources in Terraform. Now we want to enable the Rudderstack integration for Databricks, but that does not support unity catalog and instead put files inside...
@Mattias P :Unfortunately, it is not currently possible to manage access to the Hive Metastore catalog (or other external metastores) using the databricks_grant resource in Terraform. This is because the databricks_grant resource is specifically des...
Hi!So I've been looking into trying Unity Catalog since it seems to add many great features.But one thing I cant get my head around is the fact that we cant (shouldn't?) use multiple metastores in the same region in UC.Let me explain my usecase:We ha...
Hi guys, new to we have a unity catalog setup already supporting 4 of our workspaces and we would like to rename the metastore. What I would like to confirm is whether or not would I require any configurational or code changes after renaming?
Hi @Syed Ismail Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
In my workspace, I'm able to see the unity catalogs my team has created. I'm able to see the schemas and even query data in there and create views using the SQL Editor.When I go to write the same sql queries from a notebook using spark.sql or %sql I ...
Hi @Ani Madurkar Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
Databricks-connect 11.3 works for unity catalog enabled cluster or not ? I am trying to utilize the databricks-connect in Azure DevOps pipeline using Microsoft hosted agent. But getting the below error when running the databricks-connect test command...
Hi @Rey Jhon Sadly, databricks-connect for DBR versions < 13 does not support UC, as outlined in the limitations section: https://docs.databricks.com/dev-tools/databricks-connect-legacy.html#limitations.To use UC, we recommend using DBR version >= 1...
My org is considering a transition from hive metastore to unity catalog. We currently have a workspace for each of dev/uat/production and each of those provide access to their respective blob storage account data. Unity Catalog sits at the account-le...
I did find this document which indicates that you can set the initial catalog on cluster start:: https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/hive-metastore#differences-from-table-access-control
Hi, i try to used Unity Catalog in my workspace but i don´t get it!I have followed all the steps in this video and this is my group resource in Azure: After, i change Access control (IAM):Now i have the rol as Administrator and Owner in workspace.But...
Hi @Kaniz Fatma i´m owner to the workspace of Databricks but i can´t used Unity Catalog,I tried install dbdemo about Unity Catalog:%pip install dbdemos
import dbdemos
dbdemos.install('uc-03-data-lineage')it created a notebook and cluster, but the ...
I have managed to create my catalog and schema on unity catalog. Added Storage Blob Contributor role to the unity catalog metastore in ADLS for the Access Connector for Azure Databricks.however, when trying to create a table i received this error:Err...
Hi @adam wong Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback wi...
Feel free to watch a recording I made showing how to use the sync to upgrade tables from Hive metastore to Unity Catalog https://www.youtube.com/watch?v=HTiwJH9nvr8
We are following these instructions to link/assign a unity catalog metastore to a workspace using the UI. When we apply the option to enable the unity metastore in the workspace configuration, after hitting "Update", the option gets disabled. When we...
@Guilherme Jordan :If the option to enable the Unity Catalog metastore in the workspace configuration is getting disabled after hitting "Update", it could be a sign of an issue with the workspace or the Unity Catalog metastore.Here are some steps yo...
Hi,dbutils.fs.ls() is a most useful utility to list the files in remote cloud storage. I cannot use this function on top of the location managed by the Unity catalog.I have a managed external location created on `'abfss://xyz@abcd.dfs.core.windows.ne...
Hi @Venkadeshwaran K Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...
I have databricks premium and I want to enable unity catalog on it, I am not able to do so. I followed the following steps which are attached in the attachments. I am also attaching the screenshot of my cluster configuration. When I run this command:...
Hi @Prerna Munjal Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!Regards
Hi All,I am trying to manage access to objects created in a Unity Catalog.I want the owner/creator of the object(me) to not have access to the object and only the specified users/groups to be able to select/perform other actions on the objects.But th...
Hi @Akash Sivadas, It seems there is a syntax error here. Please try the below syntax:-%sql
DENY EXECUTE ON FUNCTION catalogName.schemaName.functionTest TO username@email.com;
