Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
Hi, I have a couple of questions 1) For Unity Catalog can we sync 2 Metastores which are in different regions?2) If latency is not an issue, can a single metastore handle 2 different regions (and their workspaces)
1. There is no way to sync 2 metastores specifically but you can use Delta Sharing to create a read only view of a source metastore on a target metastore. 2. Metastores are associated to the region, only workspaces in the same region can be assigned
In our Production tenant we have majority of our workspaces in the North Europe region. We have disaster recovery set up and these workspaces are in West Europe, so will require 2 MetastoresIf Unity Catalog was to get implemented, is there a way to s...
I dont see the reply form KanizCan you please share what was the reply you received to your question? Thanks in advanceSaurabh
What is the pricing of Unity Catalog? Does it come without additional costs in premium workspace? If I upgrade my standard workspace to a premium workspace, is Unity Catalog added by default?
HI Shivam2, Unity Catalog features are provided to you at no charge once you upgrade to Premium or Enterprise tier. If your Databricks account is created after November 8, 2023, then Unity Catalog is enabled by default. You can use following instru...
I have been using "rdd.flatMap(lambda x:x)" for a while to create lists from columns however after I have changed the cluster to a Shared acess mode (to use unity catalog) I get the following error: py4j.security.Py4JSecurityException: Method public ...
Hi All,I faced the same issue. When i updated the cluster to DBR 15.5 it got resolved, it is not working in DBR 13.3 (my previous cluster version).Well, it is also working in DBR 10.4. I stayed with 15.5 as being the latest. Just try out with differe...
Hi, I am working with a firm that uses Databricks in order to analyze their system tables for cost visualizations and forecasting. I am trying to gain access to just the system tables without being able to view their other sensitive notebooks, artifa...
Hi @szymon_dybczak Thank you very much for the clarification! Turns out, the people I am working with are using the regular plan and not the premium so that might not be an option. Would the only other way be to send the system tables to an S3 bucket...
Hi there,We are using Databricks and are in the early stages of adopting it. Recently, I noticed something in Databricks that caught my attention.I implemented column-level security, which works such that if you are a member of a particular group, yo...
Hi @Arivoli ,Probably you don't have permission to create tokens. Ask you workspace administrator to grant you sufficient permission. For more information refer to documentation page:Manage personal access token permissions - Azure Databricks | Micro...
Hi all,I have a registered model deployed via DAB as part of our MLOps architecture. I'm looking to deploy this model, along with other resources (workflows, schemas, experiments, etc.), in our development environment, allowing team members to easily...
@Tapestry2508 please share your script for full bundle destroy. Maybe a github gist?
Can we Move delta live tables amongst schemas within a catalog
Moving Delta Live Tables (DLT) among schemas within a catalog is a feature that is currently not directly supported.
Hi Databricks Community,I'm currently trying to create a feature table under the Hive metastore in the source workspace and migrate it later to Unity Catalog. Below is the code I am using:from pyspark.sql.types import StructType, StructField, StringT...
As of version 0.17.0, databricks-feature-store has been deprecated. All existing modules from this package are now available in databricks-feature-engineering version 0.2.0 and later. For information about migrating to databricks-feature-engineering,...
I am trying to update permissions of an object using Rest API. Endpoint I am using is "/api/2.1/unity-catalog/permissions/catalog/catalog_name"Payload is : payload = { "changes": [ { "princi...
Hi @mayank_gupta, The error you are encountering, "INVALID_PARAMETER_VALUE," indicates that the privilege "ALL PRIVILEGES" is not applicable to the entity you are trying to modify. This error message suggests that the privilege version of the metasto...
How to edit the bucket path for the Unity Catalog metastore when it is set up laterRegistered on 2024/10 from AWS marketplace.We have created a customer management VPC and manually created the workspace.I create S3 for metastore storage from workspac...
Okay on carefully reviewing your image, doc and my internal repro > You cannot modify this path once you set it. https://docs.databricks.com/en/data-governance/unity-catalog/manage-metastore.html#step-3-add-the-storage-location-to-the-metastore
We have been using the feature MLflow Model Registry Webhooks on Azure Databricks . Webhooks are in public preview, but oddly are deprecated at the same time with the introduction of Unity Catalog. The documentation says "For an alternative, see Can ...
Do you need to upgrade your Hive tables and Views to Unity Catalog?You have the below options available which you can select based on the requirements:UCX: A command-line utility suite for comprehensive Unity Catalog migrations, ideal for large Hive ...
Thank you for sharing these upgrade options @Sourav-Kundu! It’s helpful to have a clear understanding of the different approaches available for migrating Hive tables to Unity Catalog.
Hi - I would like to hide function definition from user, what is the best way to do it in unity catalog
You can revoke the user's USAGE permission on that function.
