- 3234 Views
- 2 replies
- 3 kudos
Query has been timed out due to inactivity.
Hi,We're experiencing an issue with SQL Serverless Warehouse when running queries through the dbx-sql-connector in Python. The error we get is: "Query has been timed out due to inactivity."This happens intermittently, even for queries that should com...
- 3234 Views
- 2 replies
- 3 kudos
- 3 kudos
Getting the same error while trying to run Tableau flow on Databricks. Is there a solution for this issue?
- 3 kudos
- 1212 Views
- 1 replies
- 2 kudos
Service Principal Authentication / Terraform
Hello Databricks Community,I'm encountering an issue when trying to apply my Terraform configuration to create a Databricks MWS network on GCP. The terraform apply command fails with the following error: Error: cannot create mws networks: failed duri...
- 1212 Views
- 1 replies
- 2 kudos
- 2 kudos
Databricks account-level APIs can only be called by account owners and account admins and can only be authenticated using Google-issued OIDC tokens.In Terraform 0.13 and later, data resources have the same dependency resolution behavior as defined fo...
- 2 kudos
- 4735 Views
- 7 replies
- 2 kudos
Exact cost for job execution calculation
Hi everybody,I want to calculate the exact cost of single job execution. In all examples I can find on the internet it uses the tables system.billing.usage and system.billing.list_prices. It makes sense to calculate the sum of DBUs consumed and multi...
- 4735 Views
- 7 replies
- 2 kudos
- 1392 Views
- 2 replies
- 2 kudos
Impossible to access Terraform created external location?!
Hi all,There seems to be an external location created that nobody within the organization can actually see or manage, because it has been created with a Google service account in Terraform.Here is the problem:DESCRIBE EXTERNAL LOCATION `gcsbucketname...
- 1392 Views
- 2 replies
- 2 kudos
- 2 kudos
I would agree that the metastore admin(s) should be able to see the external location. This issue can happen with terraform scripts if the script doesn't grant additional rights on the external location.
- 2 kudos
- 957 Views
- 1 replies
- 0 kudos
Unexpected Behavior with Azure Databricks and Entra ID SCIM Integration
Hi everyone,I'm currently running some tests for a company that uses Entra ID as the backbone of its authentication system. Every employee with a corporate email address is mapped within the organization's Entra ID.Our company's Azure Databricks is c...
- 957 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @antonionuzzo, This behavior is occurring because Azure Databricks allows workspace administrators to invite users from their organization's Entra ID directory into the Databricks workspace. This capability functions independently of whether th...
- 0 kudos
- 1531 Views
- 3 replies
- 1 kudos
Monitor workspace admin activities
Hello everyone,I am conducting tests on Databricks AWS and have noticed that in an organization with multiple workspaces, each with different workspace admins, a workspace admin can invite a user who is not mapped within their workspace but is alread...
- 1531 Views
- 3 replies
- 1 kudos
- 1 kudos
You do have some control over what workspace admins can do. Databricks allows account admins to restrict workspace admin permissions by enabling the RestrictWorkspaceAdmins setting. Have a look here: https://docs.databricks.com/aws/en/admin/workspace...
- 1 kudos
- 1559 Views
- 1 replies
- 2 kudos
Resolved! Predictive Optimization with multiple workspaces
We currently have an older instance of Azure Databricks that i migrated to Unity Catalog. Unfortunately i ran into some weird issues that don't seem fixable so i created a new instance and pointed it to the same metastore. The setting at the metastor...
- 1559 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @KIRKQUINBAR, if you enable Predictive Optimization at the metastore level in Unity Catalog, it automatically applies to all Unity Catalog managed tables within that metastore, no matter which workspace is accessing them. PO runs centrally, so the...
- 2 kudos
- 1324 Views
- 2 replies
- 0 kudos
Restrict serverless options to a subset of users
Hi,It seems as if there is no way to restrict serverless options to only only a subset of users. If a user has no budget policy, I assumed he could not run a serverless workload. Unfortunately, this is not the case and it will become a cost governanc...
- 1324 Views
- 2 replies
- 0 kudos
- 0 kudos
Any news on this? This is a major blocker for us to enable serverless as we only have a handful of expert users
- 0 kudos
- 2453 Views
- 3 replies
- 0 kudos
Resolved! Help with Databricks SQL Queries
Hi everyone,I’m relatively new to Databricks and trying to optimize some SQL queries for better performance. I’ve noticed that certain queries take longer to run than expected. Does anyone have tips or best practices for writing efficient SQL in Data...
- 2453 Views
- 3 replies
- 0 kudos
- 0 kudos
When working with large datasets in Databricks SQL, here are some practical tips to boost performance:Leverage Partitioning: Partition large Delta tables on columns with high cardinality and frequent filtering (like date or region). It helps Databric...
- 0 kudos
- 2381 Views
- 3 replies
- 1 kudos
Resolved! How does reported billing in Azure relate to Databricks?
Hi,I'm confused by how costs in Azure relate to costs in Databricks. I'm currently on Azure Pay-as-you-Go and Databricks Trial. There's nothing on my Azure account going on apart from Databricks.This is the costs bar chart on Azure (€):This is the co...
- 2381 Views
- 3 replies
- 1 kudos
- 1 kudos
Thanks.I can't find a documentation on how DBU translates to $/€. The pricing calculator only works for AWS/GCP. Where would I find that info?
- 1 kudos
- 1588 Views
- 3 replies
- 0 kudos
Disable 'Allow trusted Microsoft services to bypass this firewall' for Azure Key Vault
Currently even when using vnet injected Databricks workspace, we are unable to fetch the secrets from AKV if the 'Allow trusted Microsoft services to bypass this firewall' is disabled.The secret is used a AKV backed secret scope and the key vault is ...
- 1588 Views
- 3 replies
- 0 kudos
- 0 kudos
Any update on this? Is it possible to disable the ' Allow Trusted services....' rule if you are using a private endpoint or whitelist certain IPs? Or is it required no matter what?
- 0 kudos
- 1725 Views
- 5 replies
- 0 kudos
Issue creating a workspace with Databricks Partner Account & AWS Sandbox Environment
Subject:Hello,My name is Priya, and my organization is a Databricks partner. We've been given access to the Partner Academy, and my company has set up an AWS sandbox environment to support our certification preparation.I'm currently trying to set up ...
- 1725 Views
- 5 replies
- 0 kudos
- 0 kudos
I do not see anything created under network configuration. Private access settings and vpc endpoints both show 403 status when clicking on them, and there is a default policy under network policies. I will reach out to the account executive.
- 0 kudos
- 1068 Views
- 1 replies
- 0 kudos
Metastore consolidated-northeuropec2-prod-metastore-2.mysql.database.azure.com
Hi,One of network requirments on Databricks site is to allow connection to one of these public addressesconsolidated-northeuropec2-prod-metastore-2.mysql.database.azure.com, do you know if this address is for the legacy hive metastore or it's used by...
- 1068 Views
- 1 replies
- 0 kudos
- 0 kudos
The answear is that this is used for the legacy hive and if you disable it, you don't need to create an exception in your firewall.
- 0 kudos
- 958 Views
- 1 replies
- 0 kudos
Why is Databricks Using Private IP Instead of NAT Gateway's Public IP to Connect with Source System
I have a publicly accessible SQL database that is protected by a firewall. I am trying to connect this SQL database to Databricks, but I'm encountering an authentication error. I have double-checked the credentials, port, and host, and they are all c...
- 958 Views
- 1 replies
- 0 kudos
- 0 kudos
The issue occurs because the Databricks cluster's outbound traffic isn't routed through the NAT Gateway due to misconfigured network settings or conflicting outbound connectivity configurations. This should be mostly resolved by your Network team but...
- 0 kudos
- 2560 Views
- 4 replies
- 2 kudos
How to delete or clean Legacy Hive Metastore after successful completion of UC migration
Say we have completed the migration of tables from Hive Metastore to UC. All the users, jobs and clusters are switched to UC. There is no more activity on Legacy Hive Metastore.What is the best recommendation on deleting or cleaning the Hive Metastor...
- 2560 Views
- 4 replies
- 2 kudos
- 2 kudos
Sometime in the last couple of days, this setting was pushed to my account, it looks like what you want:To see if you've been added, go to your Account Console and look under Previews.
- 2 kudos
