Hi community, Azure databricks recently announced a new user management feature (now in public preview) called automatic-identity-management , which allows Azure databricks to access Azure Entra ID directly and grant users and groups permissions and ...
Exception: mlflow.exceptions.MlflowException: An API request to https://canada.cloud.databricks.com/api/2.0/mlflow/model-versions/list-artifacts failed due to a timeout. The error message was: HTTPSConnectionPool(host='canada.cloud.databricks.com', p...
Hi Databricks Community,I'm working through some networking challenges when connecting Databricks clusters to various data sources and wanted to get advice or best practices from others who may have faced similar issues.Current Setup:I have four type...
I have a Unity Catalog enabled workspace and I have full privileges including Account Admin. I would like to be able to remove the "CAN_MANAGE" privilege from the "users" group. According to the documentation, this should be possible. According to...
Hi, Can you please share the entire stacktrace to the cause of Cannot modify permissions of directory. Thanks!
In the documentation about serverless compute release notes, it is stated that "Serverless compute for notebooks and jobs uses environment versions", and that "Serverless compute always runs using the most recently released version listed here.". At ...
It seems the configuration aligns with doc https://docs.databricks.com/aws/en/release-notes/serverless/environment-version-two which specifies that the Python version in 3.11.10
Dear all,I wanted to check if anyone implemented the solution of capturing information from Databricks status page in real-time 24x7 and load that into a log or table...https://learn.microsoft.com/en-us/azure/databricks/resources/statuswhat is the be...
It seems that the webhook is the way!There is nothing about system status in Databricks REST API.There is nothing about system status in the System Tables schema.
We currently have an older instance of Azure Databricks that i migrated to Unity Catalog. Unfortunately i ran into some weird issues that don't seem fixable so i created a new instance and pointed it to the same metastore. The setting at the metastor...
Hi @KIRKQUINBAR, if you enable Predictive Optimization at the metastore level in Unity Catalog, it automatically applies to all Unity Catalog managed tables within that metastore, no matter which workspace is accessing them. PO runs centrally, so the...
Subject:Hello,My name is Priya, and my organization is a Databricks partner. We've been given access to the Partner Academy, and my company has set up an AWS sandbox environment to support our certification preparation.I'm currently trying to set up ...
I do not see anything created under network configuration. Private access settings and vpc endpoints both show 403 status when clicking on them, and there is a default policy under network policies. I will reach out to the account executive.
I have a publicly accessible SQL database that is protected by a firewall. I am trying to connect this SQL database to Databricks, but I'm encountering an authentication error. I have double-checked the credentials, port, and host, and they are all c...
The issue occurs because the Databricks cluster's outbound traffic isn't routed through the NAT Gateway due to misconfigured network settings or conflicting outbound connectivity configurations. This should be mostly resolved by your Network team but...
DearsI wanted to have a mindshare around delta sharing - how do you decide how many shares to be created and share with other departments if you are maintaining an enterprise wide data warehoouse/lakehouse using Azure Databricks. I see from the docum...
Hi @noorbasha534 ,Let me share a bit about our use case and how we’re handling Delta Sharing.Delta Sharing is indeed a simple and lightweight solution, and one of its main advantages is that it’s free to use. However, it still has several limitations...
Hi all,I am looking to capture events of permissions assigned on catalog/schemas/tables/views from the workspaces UI; example, someone gave another user USE CATALOG permission from the UI.Is it possible to capture all such events?appreciate the minds...
@Advika can you kindly please let me know the action name that I should filter upon...
In the last few days, I've encountered in Azure (and before that also in AWS, but a bit different) this message about failing to start a cluster"run failed with error message Cluster '0410-173007-1pjmdgi1' was terminated. Reason: INVALID_ARGUMENT (CL...
I see "Fleet instances do not support GPU instances" so in this case it's a no-op
Hi TeamAccidentally, we removed one of the NCC private endpoints from our storage account that was created using Terraform. When I tried to destroy and recreate it, I encountered the following error. According to some articles, the private endpoint w...
Any documentation available to connect from the Azure SQL database to Azure Databricks SQL workspace. We created a SQL warehouse personal access token for a user in a different team who can connect from his on-prem SQL DB to Databricks using the conn...
To connect an Azure SQL Database to an Azure Databricks workspace for initiating a relationship where the SQL Database interacts with Databricks (rather than the typical reverse flow), no standard methodology is outlined in public Databricks document...
When using the Databricks SDK to retrieve metadata—such as catalogs, schemas, or tables—through its built-in API endpoints, does this incur any cost similar to running SQL queries?Specifically, executing SQL queries via the API spins up a compute clu...
Hi there @Skully, You are right since you are just fetching the metadata information from catalog, tables etc instead of directly interacting or running any SQL queries, it doesn't cost same as creating a compute. when we retrieve the metadata inform...
