Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
Hello,I am working with Unity Catalogue in Azure Databricks. I have enabled the system schemas for my workspace but unable to figure out a way to send these system table logs to ADLS which I have mounted using Azure Databricks Connector. Can someone ...
@Amber-26 you can try this approach, also if you want to have graphical representation of everything you can use lakehouse monitoring feature, where after enabling these tables, you can consume them into dashboards and run them and analyze them
Hello,I activated the "Table access control" option and changed the cluster access mode to Shared with the aim of giving access rights to the tables without using the unity catalog.Since this change I can't access dbfs files with python:
Databricks users can now easily set their data catalog to read-only mode. Just click "Manage Access Level" and choose "Change access to read only".
As a way to minimize storage costs, my team and I want to create views instead of tables in the Gold layer.We always try to improve the experience of our users by adding comments to the columns. the problem is that views do not inherit comments from ...
Hi @isabelgontijo ,had you found any workaround this?You can use create view statement as mentioned belowIn addition to this,i am not able to add comment on column which is encrypted and in views we are decrypting that column. I am using create view...
We have been successfully managing access for our unity catalogs using the databricks_grant resources in Terraform. Now we want to enable the Rudderstack integration for Databricks, but that does not support unity catalog and instead put files inside...
@Mattias P​ :Unfortunately, it is not currently possible to manage access to the Hive Metastore catalog (or other external metastores) using the databricks_grant resource in Terraform. This is because the databricks_grant resource is specifically des...
Hi,I was wondering if it would be possible to execute a dlt command such as @dlt.create_table against the execute endpoint API and then materialize the asset?Thank you,Adriana Cavalcanti
Ref.: The Endpoint API that I am referring to is: https://docs.databricks.com/api/workspace/commandexecution/execute
Hi I am new to databricks and trying to connect to Rstudio Server from my all-purpose compute cluster.Here are the cluster configuration:Policy: Personal ComputeAccess mode: Single userDatabricks run time version: 13.2 ML (includes Apache Spark 3.4.0...
Running into this issue as well. Let me know if you found a resolution, @paniz_asghari!
I'm trying to connect to a cluster with Runtime 13.0 and Unity Catalog through databricks-connect version 13.0.0 (for Python). The spark session seems to initialize correctly but anytime I try to use it, I get the following error:{SparkConnectGrpcExc...
I have the same error up to DBR 13.3 LTS. When I upgraded it to 14.0, I was then able to connect with my databricks compute from my local environment.
I've been working on creating a schema in the Hive Metastore using the following command:spark.sql('CREATE DATABASE IF NOT EXISTS {database}')The schema or database is successfully created, but I encountered an issue where it's only accessible for me...
I've been working on creating a schema in the Hive Metastore using the following command:spark.sql(f'CREATE DATABASE IF NOT EXISTS {database}')The schema or database is successfully created, but I encountered an issue where it's only accessible for m...
I've been working on creating a schema in the Hive Metastore using the following command:spark.sql(f'CREATE DATABASE IF NOT EXISTS {database}')The schema or database is successfully created, but I encountered an issue where it's only accessible for m...
I want to use an event trigger to start a job. -The MI has the Storage Blob Data Contributor role-Test connection is successful at the level of the external location-I have read permission on the external location-I have owner permission on the job-O...
for referencehttps://stackoverflow.com/a/75906376/2842348seems this could be made to work by allowing connectivity from Databricks' private vnets, the same way it is needed currently done for serverless setups if you have an environment that blocks p...
We extensively use Databricks (DBX) for creating Tableau visuals. Whenever DBX data sources are published on our self hosted Tableau Server, we have to add the connection creds for the data source. These creds can be both personal email id-password o...
Thanks for your response. While I understood most of it, I still have some doubts to clear. Therefore explaining it again. Problem: We want users to create data sources using Databricks via their personal credentials in the local env (tableau desktop...
Hi, I have a new workspace that was converted to be controlled via Unity Catalog.There is not data stored here as this is not in use yet, so I wanted to ask if it was safe to remove this without breaking the workspace?Thank you
As things gravitate towards Unity Catalog, it would indeed be nice to have the the ability to completely remove Hive from existing setups.
Hi TeamI am looking into implementing a functionality which allows me to create a ServicePrincipal and I want to create a personal token for this ServicePrincipal using java sdk. While trying to do this I am getting this error ```com.databricks.sdk.c...
Hi Thank you for your response how would the databricks know for which service principal I want to grant permission?
