Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
I've been working on creating a schema in the Hive Metastore using the following command:spark.sql(f'CREATE DATABASE IF NOT EXISTS {database}')The schema or database is successfully created, but I encountered an issue where it's only accessible for m...
I want to use an event trigger to start a job. -The MI has the Storage Blob Data Contributor role-Test connection is successful at the level of the external location-I have read permission on the external location-I have owner permission on the job-O...
for referencehttps://stackoverflow.com/a/75906376/2842348seems this could be made to work by allowing connectivity from Databricks' private vnets, the same way it is needed currently done for serverless setups if you have an environment that blocks p...
We extensively use Databricks (DBX) for creating Tableau visuals. Whenever DBX data sources are published on our self hosted Tableau Server, we have to add the connection creds for the data source. These creds can be both personal email id-password o...
Thanks for your response. While I understood most of it, I still have some doubts to clear. Therefore explaining it again. Problem: We want users to create data sources using Databricks via their personal credentials in the local env (tableau desktop...
Hi, I have a new workspace that was converted to be controlled via Unity Catalog.There is not data stored here as this is not in use yet, so I wanted to ask if it was safe to remove this without breaking the workspace?Thank you
As things gravitate towards Unity Catalog, it would indeed be nice to have the the ability to completely remove Hive from existing setups.
Hi TeamI am looking into implementing a functionality which allows me to create a ServicePrincipal and I want to create a personal token for this ServicePrincipal using java sdk. While trying to do this I am getting this error ```com.databricks.sdk.c...
Hi Thank you for your response how would the databricks know for which service principal I want to grant permission?
I am attempting to execute the following command from a notebook on a runtime 10.4 cluster, but I'm encountering an error: "current user does not have privilege USAGE on Catalog".To provide some context, I am using DBR 10.4 specifically because I nee...
@drii_cavalcanti From the error, we could see that you do not have permission to use the catalog. I suppose you are using Unity Catalog. You should 3 levels of permissions to access the Catalog, schema and then tables.You can refer to this document -...
Hi everyone,Just wondering if I copy the catalog folders (under the metastore storage) to a different s3 bucket, once I have a new metastore with a new ID I can restore the tables as they were <catalog_name>.<schema_name>.<table_name> into the new me...
We are in the process of integrating the Unity Catalog with the workspaces across our organization. As a preliminary step, we need to designate a Databricks Account Admin.Our Azure tenant encompasses multiple subscriptions, each hosting distinct work...
Hello,I want to grant 'select' option on some columns in my table.So i created view, and now people can access my view but cannot access the table.However, they cannot do 'select' on the view, because they do not have privileges to the table.What can...
To sum up,It is not possible to create a view based on a table that someone does not have permission to.It's a pity, because this option would be very useful to limit access to batch-fed tables.Thanks all for the help and your time!Regards,Łukasz
I am trying to create an external table using csv file which is stored in ADLS gen2 My account owner has created a storage credential and an external location I am a databricks user who all privileges on external location when trying to create a tabl...
HiUnity Metastore can currently only be provisioned once for each Azure region. I have three environments in Azure each consisting of a separate Subscription and Resource Group (RG). Each RG contains a Databricks Workspace and a Storage Account holdi...
Ask and ye shall receive: https://github.com/databricks/terraform-provider-databricks/releases/tag/v1.24.0They've implemented account-level UC API in terraform now!
In my Databricks regular Clusters, Our clusters are configured to an External Hive Meta Store (Azure SQL Server). I am able to set the External Hive configuration in the Advanced options Spark config setting within the cluster setting. In our setting...
@RafaelGomez61 I have the some problem and after some exploring, I've configured it and also wrote a blog for the whole steps. You can have a look here: https://medium.com/@judy3.yang/how-to-configure-external-hive-meta-store-for-databricks-sql-wareh...
I'm looking to migrate onto unity catalog but a number of my data ingestion notebooks throw a securityexception/whitelist errors for numerous spark. functionsIs there some configuration setting I need to enable to whitelist the spark.* methods/functi...
Hi @Jakub K I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest provid...
Hi, Getting "Access validation failed" when trying to create a metastore. Anyone noticed anything extra needed to be done except for what's written in the azure guides below? Any help on how to succeed with creating a metastore would be deeply apprec...
I was told this was a bug that was fixed on Monday. We’re using AWS, and I was able to create the metastore on Tuesday without error.
Hi guys, how are you ?How can I access tables outside the Databricks ? (hive metastore)I have a python script in my local structure but I need to access are tables stored in Databricks (hive metastore) , how ???? Have any idea ?Thank you guys
