- 1939 Views
- 1 replies
- 0 kudos
Databricks delta sharing design
DearsI wanted to have a mindshare around delta sharing - how do you decide how many shares to be created and share with other departments if you are maintaining an enterprise wide data warehoouse/lakehouse using Azure Databricks. I see from the docum...
- 1939 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @noorbasha534 ,Let me share a bit about our use case and how we’re handling Delta Sharing.Delta Sharing is indeed a simple and lightweight solution, and one of its main advantages is that it’s free to use. However, it still has several limitations...
- 0 kudos
- 973 Views
- 4 replies
- 0 kudos
get permissions assignment done from the workspaces UI
Hi all,I am looking to capture events of permissions assigned on catalog/schemas/tables/views from the workspaces UI; example, someone gave another user USE CATALOG permission from the UI.Is it possible to capture all such events?appreciate the minds...
- 973 Views
- 4 replies
- 0 kudos
- 0 kudos
@Advika can you kindly please let me know the action name that I should filter upon...
- 0 kudos
- 874 Views
- 3 replies
- 1 kudos
misbehavior of spots with fallback to on demand on job clusters
In the last few days, I've encountered in Azure (and before that also in AWS, but a bit different) this message about failing to start a cluster"run failed with error message Cluster '0410-173007-1pjmdgi1' was terminated. Reason: INVALID_ARGUMENT (CL...
- 874 Views
- 3 replies
- 1 kudos
- 1 kudos
I see "Fleet instances do not support GPU instances" so in this case it's a no-op
- 1 kudos
- 501 Views
- 2 replies
- 0 kudos
Access locked out with SSO
We were locked out of our account (expired secret for login via Azure Entra ID and password-based login disabled).How can i add a new secret in databricks if i'm only able to login with SSO and this is broken?
- 501 Views
- 2 replies
- 0 kudos
- 1836 Views
- 0 replies
- 0 kudos
Unable to destroy NCC private endpoint
Hi TeamAccidentally, we removed one of the NCC private endpoints from our storage account that was created using Terraform. When I tried to destroy and recreate it, I encountered the following error. According to some articles, the private endpoint w...
- 1836 Views
- 0 replies
- 0 kudos
- 1150 Views
- 1 replies
- 0 kudos
Implementing Governance on DLT pipelines using compute policy
I am implementing governance over compute creation in the workspaces by implementing custom compute policies for all-purpose, job and dlt pipelines. I was successfully able to create compute policies for all-purpose and jobs where I could restrict th...
- 1150 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @DeepankarB, To enforce compute policies for DLT pipelines, make sure your policy JSON includes policy_family_id: dlt and set apply_policy_default_values: true in the pipeline cluster settings. This helps apply the instance restrictions correctly ...
- 0 kudos
- 1393 Views
- 4 replies
- 0 kudos
Databricks Predictive optimization
If we want to enable Databricks Predictive Optimization, then is it also mandatory to enable serverless Job/Notebook Compute in our account. We already have Serverless SQL warehouse available in our workspaces.
- 1393 Views
- 4 replies
- 0 kudos
- 0 kudos
The documentation states this:Predictive optimization identifies tables that would benefit from ANALYZE, OPTIMIZE, and VACUUM operations and queues them to run using serverless compute for jobs.If I don't have serverless workloads enabled how does pr...
- 0 kudos
- 1603 Views
- 0 replies
- 0 kudos
Azure Databricks with VNET injection and SCC
Hi,Azure databricks with VNET injection and SCC need to communicate with Azure endpoints for following,Metastore, artifact Blob storage, system tables storage, log Blob storage, and Event Hubs endpoint IP addresses.https://learn.microsoft.com/en-us/a...
- 1603 Views
- 0 replies
- 0 kudos
- 2786 Views
- 2 replies
- 0 kudos
Resolved! Migrate to a new account
Hey Team,We're looking into migrating our correct Databricks solution from 1 AWS account (us-east-1 region) to another (eu-central-1 region). I have no documentation left on/about how the corrent solution was provisioned, but I can see CloudFormation...
- 2786 Views
- 2 replies
- 0 kudos
- 0 kudos
I ended up using the terrafrom-databricks-provider tool to perform an export and import of the old workspace into the new one. All that was needed was a PAT in each, export from the old, sed the region, account and PAT and apply. This got me about 7...
- 0 kudos
- 3862 Views
- 22 replies
- 4 kudos
Resolved! Unable to enable Serverless Notebooks
Hello there,I have a Databricks Premium Subscription but am not able to enable Serverless Notebooks (as that option does not seem to exist). I have gone through DB documentation and have Unity Catalog Enabled. I even opened a ticket (00591635) but it...
- 3862 Views
- 22 replies
- 4 kudos
- 440 Views
- 1 replies
- 0 kudos
Does using SDK API calls cost money?
When using the Databricks SDK to retrieve metadata—such as catalogs, schemas, or tables—through its built-in API endpoints, does this incur any cost similar to running SQL queries?Specifically, executing SQL queries via the API spins up a compute clu...
- 440 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi there @Skully, You are right since you are just fetching the metadata information from catalog, tables etc instead of directly interacting or running any SQL queries, it doesn't cost same as creating a compute. when we retrieve the metadata inform...
- 0 kudos
- 1548 Views
- 4 replies
- 1 kudos
Resolved! Enable Databricks system error
Hi,We want to enable some system system tables in our databricks workspace using this command:curl -v -X PUT -H "Authorization: Bearer <PAT token>" "https://adb-0000000000.azuredatabricks.net/api/2.0/unity-catalog/metastores/<metastore-id>/systemsche...
- 1548 Views
- 4 replies
- 1 kudos
- 1 kudos
While disabling some system schemas we disabled billing system schema and now we cannot enable it again due to this error: billing system schema can only be enabled by Databricks.How can I re-enable billing schema?
- 1 kudos
- 1167 Views
- 1 replies
- 0 kudos
Collation problem with df.first() when different from UTF8_BINARY
I'm getting a error when I want to select the first() from a dataframe when using a collation different than UTF8_BINARYThis works :df_result = spark.sql(f""" SELECT 'en-us' AS ETLLanguageCode""")display(df_result)print(df_resu...
- 1167 Views
- 1 replies
- 0 kudos
- 0 kudos
other example :
- 0 kudos
- 1235 Views
- 1 replies
- 0 kudos
Resolved! Can a SQL Warehouse Pro be shared across multiple workspaces
I'm currently using a SQL Warehouse Pro in one of my Databricks workspaces, and I’m trying to optimize costs. Since the Pro Warehouse can be quite expensive to run, I’d prefer not to spin up additional instances in each workspace.Is there any way to ...
- 1235 Views
- 1 replies
- 0 kudos
- 0 kudos
hi @jfid A SQL Warehouse Pro instance cannot be shared directly across multiple Databricks workspaces. Each workspace requires its own SQL Warehouse instance, even if the compute and data access needs are similar. This is because compute resources li...
- 0 kudos
- 2387 Views
- 2 replies
- 0 kudos
Convert Account to Self-managed
I am in the process of setting up a new Databricks account for AWS commercial. I mistakenly setup the account with the email: databricks-external-nonprod-account-owner@slingshotaerospace.com to not be self-managed and I would like for this new accoun...
- 2387 Views
- 2 replies
- 0 kudos
