Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
can catalog owner delete tables created by another user.
Hi meenaliNo, it is not possible. As example, I have a catalog "dspro20", where I am the owner: In this catalog, I have a schema elt and a table "target_cities":The owner is User dspro20@nona.company. When I try to delete the table, I got this error ...
Hello! Newbie here, so apologies if this is a super basic question. Trying to figure out if we can connect Purview to an AWS instance of Databricks (vice Azure instance), but I have only seen articles on connecting Azure Databricks to Purview. I r...
¡Hola! Gracias por tu respuesta y por la información. Es útil saber que Azure Purview tiene una integración principal con Azure Databricks en consejos android, aunque la falta de soporte directo para Databricks en AWS es un punto a considerar. La ide...
Hello Briksters, i am looking for a giving a grand to developer being able execute with SHOW GROUPS WITH USER '***@***' without admin permission on UC.Could you give any tips to do it? UC Admin is able to see result query, but developer is not.
Hello all,I am looking for a way to load a persistent query history table from the query history REST API (including metrics). I do know about the system table system.query.history, but I do not have access to it.I am looking to create a python scrip...
Hi, Can you check if you have access to this? https://github.com/dbsys21/lakehouse-tacklebox/tree/main/40-observability/dbsql-query-history-sync Ideally your organization should enable the system tables and provide the necessary access, so each team ...
What permissions are actually needed in order to create a delta share?I have read that you either need to be metastore admin, or aUser with delegated permissions or ownership on both the share and the recipient objects ((USE SHARE + SET SHARE PERMISS...
Hi, Hope the doc link helps: https://docs.databricks.com/en/delta-sharing/create-share.html#requirements Specifically this statement should alleviate your immediate point. Have the USE CATALOG and USE SCHEMA privilege on the catalog and schema that...
we are switching over to Unity Catalog and attempting to confirm the ability to run our existing notebooks. I have created a new Shared Unity Catalog Cluster and ran the notebook using the new cluster. Ran into an error attempting to execute a prin...
To replace dbutils.notebook.entry_point.getDbutils().notebook().getContext().toJson() and related APIs try:from dbruntime.databricks_repl_context import get_context token = get_context().apiToken host = get_context().browserHostName get_context().__d...
Currently, we have implemented Unity Catalog in Databricks and we are facing an issue with lineage. We execute a notebook that fills a table, and upon completion, the lineage is created correctly; all tables are mapped properly, and we can visualize ...
What does your notebook do? Does it drop the existing tables and create a new one? Give more details about your process preferably the source code.
I am excited to share that I recently published a whitepaper on how to leverage Databricks to build a robust data governance framework. This paper explores key strategies and best practices for implementing a strong governance model using Databricks'...
I am using Unity Catalog to store my models. My Unity Catalog meta-store is hosted in an Azure Storage Account, which has public network access disabled. Access is restricted to certain IP ranges and private endpoint connections only.Recently, my dev...
Got issue fixed:The issue was with Private Endpoint: we had PE with dfs sub-resource only, but mlflow uses blob, so when I created PE with blob sub-resource - the issue was fixed. (also blob DNS zone and network link is needed)
I have run into an issue with creating an External Location on an ASLS storage container with the following error :"Hierarchical Namespace (HNS) not enabledAzure storage account does not have hierarchical namespace (HNS) enabled." The Storage accoun...
I ran into the same issue, after enabling the HNS, still getting this error.
I would like to set up GitLab and Grafana servers using Databricks Container Services (DCS). The reason is that our development team is small, and the management costs of using EKS are not justifiable. We want to make GitLab and Grafana accessible in...
Yes, it’s possible to set up GitLab and Grafana servers using Databricks Container Services (DCS). Here’s a high-level overview of how you can achieve this:Prepare Your Docker Images:Create Docker images for GitLab and Grafana. You can use official ...
I would like to create an external location on a publicly available S3 bucket, for which I don't have credentials. I get a syntax error unless I include credentials. Is there a way to do this? CREATE EXTERNAL LOCATION public_bucket URL 's3://public_b...
Hi @jerickson ,I have tested this on Databricks Runtime 14.3 LTS:Install on the cluster the below Maven packages:com.amazonaws:aws-java-sdk-bundle:1.12.262org.apache.hadoop:hadoop-aws:3.3.4Run the below code to read your csv file into dataframe: spar...
Hi everyone!I would love to know whether Databricks has the following capabilities and what are the core functionalities of the tool:Data glossary: provides clear definitions of business terms, ensuring that everyone in the organization uses the same...
Hi everyone,I need to transfer tables present in the Hive metastore from one workspace to another. My ultimate goal is to migrate these tables to Unity Catalog. I already have a code that will handle the migration to Unity Catalog, but I need help wi...
Are these tables managed tables or external tables? This is very important step for your migration.
