cancel
Showing results for 
Search instead for 
Did you mean: 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Governance

Forum Posts

paniz_asghari
by New Contributor
  • 2214 Views
  • 1 replies
  • 0 kudos

SparkException: There is no Credential Scope.

Hi I am new to databricks and trying to connect to Rstudio Server from my all-purpose compute cluster.Here are the cluster configuration:Policy: Personal ComputeAccess mode: Single userDatabricks run time version: 13.2 ML (includes Apache Spark 3.4.0...

  • 2214 Views
  • 1 replies
  • 0 kudos
Latest Reply
kunalmishra9
New Contributor III
  • 0 kudos

Running into this issue as well. Let me know if you found a resolution, @paniz_asghari!  

  • 0 kudos
carlafernandez
by New Contributor III
  • 30921 Views
  • 10 replies
  • 6 kudos

Resolved! Databricks-connect version 13.0.0 throws Exception with details = "Missing required field 'UserContext' in the request."

I'm trying to connect to a cluster with Runtime 13.0 and Unity Catalog through databricks-connect version 13.0.0 (for Python). The spark session seems to initialize correctly but anytime I try to use it, I get the following error:{SparkConnectGrpcExc...

  • 30921 Views
  • 10 replies
  • 6 kudos
Latest Reply
redperiabras
New Contributor II
  • 6 kudos

I have the same error up to DBR 13.3 LTS. When I upgraded it to 14.0, I was then able to connect with my databricks compute from my local environment.

  • 6 kudos
9 More Replies
BMex
by New Contributor III
  • 3081 Views
  • 1 replies
  • 0 kudos

Resolved! Hide VIEW definition in Unity-Catalog

Hi,I am trying to set up Unity-Catalog for my company and ran into a problem today. Basically, for each new source of data we ingest, we create a view-layer on top of the "tables". We do that because we have pseudonymized information in our datalake ...

BMex_1-1695113477735.png BMex_0-1695113280990.png
  • 3081 Views
  • 1 replies
  • 0 kudos
Latest Reply
BMex
New Contributor III
  • 0 kudos

One solution I found is, creating a function which does the decryption of the column, and from the view creation, I simply call the function and pass the column.This solution however pushes me to put the decryption key inside the function in plain-te...

  • 0 kudos
drii_cavalcanti
by New Contributor III
  • 1004 Views
  • 0 replies
  • 0 kudos

Permission on Hive Metastore DBX 10.4

I've been working on creating a schema in the Hive Metastore using the following command:spark.sql('CREATE DATABASE IF NOT EXISTS {database}')The schema or database is successfully created, but I encountered an issue where it's only accessible for me...

Data Governance
clusters
hive_metastore
legacy
permission
  • 1004 Views
  • 0 replies
  • 0 kudos
drii_cavalcanti
by New Contributor III
  • 741 Views
  • 0 replies
  • 0 kudos

Hive Metastore permission on DBX 10.4 Cluster

I've been working on creating a schema in the Hive Metastore using the following command:spark.sql(f'CREATE DATABASE IF NOT EXISTS {database}')The schema or database is successfully created, but I encountered an issue where it's only accessible for m...

  • 741 Views
  • 0 replies
  • 0 kudos
drii_cavalcanti
by New Contributor III
  • 375 Views
  • 0 replies
  • 0 kudos

Hive Metastore Schema Permissions

I've been working on creating a schema in the Hive Metastore using the following command:spark.sql(f'CREATE DATABASE IF NOT EXISTS {database}')The schema or database is successfully created, but I encountered an issue where it's only accessible for m...

  • 375 Views
  • 0 replies
  • 0 kudos
AsphaltDataRide
by New Contributor III
  • 25754 Views
  • 10 replies
  • 10 kudos

job event trigger - Invalid credentials for storage location

I want to use an event trigger to start a job. -The MI has the Storage Blob Data Contributor role-Test connection is successful at the level of the external location-I have read permission on the external location-I have owner permission on the job-O...

image.png
  • 25754 Views
  • 10 replies
  • 10 kudos
Latest Reply
adriennn
Contributor
  • 10 kudos

for referencehttps://stackoverflow.com/a/75906376/2842348seems this could be made to work by allowing connectivity from Databricks' private vnets, the same way it is needed currently done for serverless setups if you have an environment that blocks p...

  • 10 kudos
9 More Replies
Shubhanshu
by New Contributor II
  • 3213 Views
  • 1 replies
  • 0 kudos

Unable to create external table

I am trying to create an external table using csv file which is stored in ADLS gen2 My account owner has created a storage credential and an external location I am a databricks user who all privileges on external location when trying to create a tabl...

  • 3213 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Shubhanshu, To overcome the error and create the table, ensure that the client secret token associated with the Azure Active Directory (Azure AD) application service principal is not expired or invalid. Here are the steps you can follow: 1. Open...

  • 0 kudos
kaustubhgupta
by New Contributor II
  • 3053 Views
  • 2 replies
  • 0 kudos

How to setup oauth for Databricks connection in Tableau Server?

We extensively use Databricks (DBX) for creating Tableau visuals. Whenever DBX data sources are published on our self hosted Tableau Server, we have to add the connection creds for the data source. These creds can be both personal email id-password o...

  • 3053 Views
  • 2 replies
  • 0 kudos
Latest Reply
kaustubhgupta
New Contributor II
  • 0 kudos

Thanks for your response. While I understood most of it, I still have some doubts to clear. Therefore explaining it again. Problem: We want users to create data sources using Databricks via their personal credentials in the local env (tableau desktop...

  • 0 kudos
1 More Replies
TonyUK
by New Contributor II
  • 6641 Views
  • 7 replies
  • 4 kudos

Can I delete hive_metastore on a Unity Catalog Workspace?

Hi, I have a new workspace that was converted to be controlled via Unity Catalog.There is not data stored here as this is not in use yet, so I wanted to ask if it was safe to remove this without breaking the workspace?Thank you

  • 6641 Views
  • 7 replies
  • 4 kudos
Latest Reply
JasonThomas
New Contributor III
  • 4 kudos

As things gravitate towards Unity Catalog, it would indeed be nice to have the the ability to completely remove Hive from existing setups.

  • 4 kudos
6 More Replies
Shubhanshu
by New Contributor II
  • 3372 Views
  • 1 replies
  • 0 kudos

Resolved! Error while describing table created in unity catalog

I have created an external table in unity catalog and when describing its data getting the following errororg.apache.hadoop.fs.FileAlreadyExistsException: Operation failed: "The specified path, or an element of the path, exists and its resource type ...

  • 3372 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Shubhanshu , The error message you're seeing is a FileAlreadyExistsException from the Hadoop File System (HDFS). This error typically occurs when there's an attempt to create a file or directory that already exists, and the operation being perfor...

  • 0 kudos
shkelzeen
by New Contributor II
  • 3258 Views
  • 4 replies
  • 0 kudos

Create token for ServicePrincipal using java SDK

Hi TeamI am looking into implementing a functionality which allows me to create a ServicePrincipal and I want to create a personal token for this ServicePrincipal using java sdk. While trying to do this I am getting this error ```com.databricks.sdk.c...

  • 3258 Views
  • 4 replies
  • 0 kudos
Latest Reply
shkelzeen
New Contributor II
  • 0 kudos

Hi Thank you for your response how would the databricks know for which service principal  I want to grant permission?

  • 0 kudos
3 More Replies
sandeephenkel23
by New Contributor II
  • 3696 Views
  • 3 replies
  • 1 kudos

Resolved! SELECT SCHEMA/USE_SCHEMA WHICH IS MOST SUITABLE FOR ACCESS

Hi,As part of the DataGovernance or Authorization topic we are working on automation of the code for granting the access CATALOG LEVEL,SCHEMA LEVEL and TABLE LEVEL in Unity CatalogAs USE CATALOG Provides access at the Catalog level to user/group(whic...

Data Governance
Select Schema and Use_Schema
  • 3696 Views
  • 3 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @sandeephenkel23, Based on the information provided, if you want to grant a user access to only a particular table from a database/schema and not to all the tables, you should use the SELECT privilege on that particular table. Using USE SCHEMA or ...

  • 1 kudos
2 More Replies
cayisi
by New Contributor
  • 2935 Views
  • 1 replies
  • 1 kudos

Resolved! Scope of Databricks Account Admin Permissions within Azure Tenant

We are in the process of integrating the Unity Catalog with the workspaces across our organization. As a preliminary step, we need to designate a Databricks Account Admin.Our Azure tenant encompasses multiple subscriptions, each hosting distinct work...

  • 2935 Views
  • 1 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @cayisi, The Databricks Account Admin role does not grant authority over individual subscriptions but extends control over all subscriptions within the tenant. This is because the Databricks Account represents a single entity that can include mult...

  • 1 kudos
drii_cavalcanti
by New Contributor III
  • 3849 Views
  • 2 replies
  • 2 kudos

Resolved! Privilege Issue When Running Delta Table SaveAsTable Command in Databricks Runtime 10.4

I am attempting to execute the following command from a notebook on a runtime 10.4 cluster, but I'm encountering an error: "current user does not have privilege USAGE on Catalog".To provide some context, I am using DBR 10.4 specifically because I nee...

Data Governance
cluster
driver
permission
Unity Catalog
  • 3849 Views
  • 2 replies
  • 2 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 2 kudos

Hi @drii_cavalcanti, The error you're encountering is related to the new USAGE privilege that is enforced on Databricks clusters running Databricks Runtime 7.3 LTS and above. This privilege is required to access an object in the database. To resolve ...

  • 2 kudos
1 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels