Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
Is it possible to provide fine grained control (folder level/file level) for a given volume?I have two SCIM integrated groups who have read volume access at the catalog level, but those two groups need different permissions on a lower level. Preferab...
Can you define the external location at the Landing level and create two Volumes one for PDF and other for CSV and provide access to the respective groups 1 and 2.
All data finish in Excel, but do it with some dignity. Official #databricks dbsqlcli allows it to be done in the old-style mysqldump way.https://docs.databricks.com/en/dev-tools/databricks-sql-cli.html
Thank you for sharing this @Hubert-Dudek
I saw a really nice article (https://www.databricks.com/blog/announcing-public-preview-volumes-databricks-unity-catalog) on the incorporation of various elements of data lineage within Unity Catalog. In my own exploration, I've been able to replicate...
I saw a really nice article (https://www.databricks.com/blog/announcing-public-preview-volumes-databricks-unity-catalog) on the incorporation of various elements of data lineage within Unity Catalog. In my own exploration, I've been able to replicate...
I wanted to set description of a schema in Unity Catalog.Description was greyed out and on pencil icon I got: "Requires Ownership of SCHEMA 'xxxxx'."Owner is set to the Azure Active Directory group that I am member of.I was able to change the owner t...
AAD group membership cannot be used as the object owner in UC to edit object description directly. Object ownership typically refers to individual user accounts, not group memberships, in most systems.
We're thrilled to introduce UCX v0.15.0, packed with cutting-edge features, enhancements, and deprecations that'll take your upgrading experience to new heights! Release Highlights: AWS S3 support: We've added AWS S3 support to the migrate-location...
We are attempting to create a comprehensive Data Governance solution using Unity Catalog & INFA CDGC tool. The objective is to onboard the business assets on the Informatica DG platform and use the unity catalog to trace technical assets and lineage ...
Yes, Informatica now has native integration to Unity Catalog with its cloud data catalogue and governance service. You can scan Unity catalog Metadata and propagate this in CDGC and apply policies, business terms, overlay data quality scores, display...
I noticed in the Catalog Explorer of an Unity Catalog integrated workspace that there is a default catalog named 'system' owned by 'System user'. Who is this system user? It is not listed in the admin dashboard of all workspace users.
Hi all,I try to create an external volume in a catalog/schema on a subpath of the external location.This works fine, however for a certain path I cannot create an external volume. I always getInput path url overlaps with managed storage within 'Crea...
I have the same issue. I have created a Catalog using External Location, deleted it and now i can't create Volume for this Location even thought Catalog is deleted long ago
Hi All, Can anyone point me to either documentation or personally tried and tested method of backing up (and restoring) Unity Catalog and its associated managed tables? We're running on Azure and using ADLS Gen2.Regards,Ashley
Our UC managed tables are stored on prod ADLS storage which is different from UC root storage account. So what's the best way to backup and restore UC managed tables into different region? One option is deep clone tables, copy ADLS folders to an...
Hi,If I update a Tag on a table within a Noetbook as follows:sql = f"""ALTER TABLE {silver_catalog}.public.tablename SET TAGS ('Is_published' = 'Y');"""spark.sql(sql)Then the User ID and Username, nor Notebook is populated when I look in the History ...
Hi, This might be a silly question to ask but is it possible to enforce schema on a table created from external location, I am trying to create a table from a s3 location and my command looks like thisquery = """CREATE TABLE IF NOT EXISTS {} ...
define columns when create table https://community.databricks.com/t5/data-engineering/external-table-from-parquet-partition/td-p/24689
I faced with error "[RequestId=b567c3bb-05fc-4ac8-b4db-2d826b2c00be ErrorClass=QUOTA_EXCEEDED] Cannot create 1 Table(s) in Metastore 95157916-5539-4dc4-a7d6-e9a854ba0591 (estimated count: 100088, limit: 100000). " but I didn't see here this limit ...
@Yevhen Here is the doc for quota limit: https://docs.databricks.com/en/data-governance/unity-catalog/index.html#resource-quotas
Hi, Databricks CommunityI am currently plan to do migrate of existing metastore (not in a desired account and name) to a new one (a different desired account) within the same region. I understand it is not as straightforward it might be and complicat...
hi, @Retired_mod . Thank for the recommendations and links, they are helpful and I am going through them one by one.
SELECT position_no, position_function, work_function, job_profile_id, pos_cat as position_category, pos_cat_desc, job_posting_title as pos_title, employee_status as emp_status, emp_status_desc, clevel, subs...
In Azure, what are the limits of Unity Catalogue? One UC per region per subscription, or is it one UC per region per tenant?
Does it mean we will have DEV, TEST, PROD catalog in a single Unity catalog?
