Step followed - Step1: To add init script that will copy the keystore file in the tmp location.Step2: To add spark config in cluster advance options - spark.driver.extraJavaOptions -Djavax.net.ssl.keyStore=/tmp/keystore.jks -Djavax.net.ssl.keyStorePa...
Hello Databricks Community,I am facing an issue with changing the ownership of an ETL pipeline to a service principal. Despite being the Databricks administrator with workspace and metastore admin permissions, I receive the following error: "Only adm...
Are you saying that you can't create a PAT token in the Databricks Workspace, or you can create a PAT token but don't have a secure way of using it with the API request? Please explain a bit more. Thanks, Lou.
How have you leveraged Databricks Serverless to streamline your data workflows? We invite you to share your real-world experiences-which workloads have benefited most, what challenges you’ve encountered, and any best practices or tips you have for ma...
Is it possible to remove Tableau as a option from the Catalog Exproler (BI tools) when you want to open or explore the data?
It is currently not possible to remove the Tableau option from the "BI tools" integration menu in the Databricks Catalog Explorer. Hope this helps. Cheers, Lou.
Hi Everyone,Hope all is well with you!I'm reaching out to the community for some advice on customizing our workspace settings. Specifically, I have two things I'm trying to figure out:Disabling the "Playground" Option: Is there a way to turn off the ...
Hello, According to the databricks_job resource terraform documentation, the description argument is supported at the task level. However, when I update the Terraform configuration to include a task description, the terraform plan reflects the change...
Hi @ChinuLee, the task-level description appears to be hidden in the UI, both during manual creation and when viewing tasks after creation. However, it is visible when you choose the "View as Code" option, as shown below:So it is supported, just hidd...
Hi all,When I use Terraform in an Azure DevOps pipeline to create Databricks Storage Credential, I got the following error. Has anybody met the same error before? Or is there any idea how to debug it? Error: cannot create storage credential: failed d...
How exactly do you need to configure auth_type in this case? I tried different options but nothing seems to work. I also would like to use the Service Connection from Azure DevOps Pipeline to deploy Databricks via TerraformTaskV4@4.
The documentation here doesn't give much information about how to use `external_id` when creating a new group. If I reference the object_id for an Azure AD Group, the databricks group gets created but the members from the AD group are not added, nor ...
Greetings from the future! Now it is clear that external_id, which IS Azure's ObjectID, comes from the internal sync mechanism, that can be enabled in your account under previews:I was able to reference my security group in Terraform and create that ...
I'm working with large Delta tables in Databricks and noticing slower performance during read operations. I've already enabled Z-ordering and auto-optimize, but it still feels sluggish at scale. Are there best practices or settings I should adjust fo...
Hi @gardenmap, if possible can you detail more?For example, in my case what I've done:For tables above 1TB as it's can segregated by date, we've decided to enable a partition by the date column;Independent if it's partitioned or not, we decided to ma...
Hi folks!I'm sure I'm not the only one, but our users have the tendency to click the big Refresh button on all dashboards every time they open them.Using resources efficiently is something I value deeply, so our team came up with a schedule policy - ...
Hey @leo-machado , yes I did request it formally and shared this post with the team - heads up, once something has been prioritised it can still take between 6-18 months to build.
Hello,I would like to restrict workspace admins from creating service principals and leave this privilege only to the account admin. Is this possible? I am aware of the RestrictWorkspaceAdmins command, but it does not meet my needs. Additionally, I h...
Hello @antonionuzzo! Based on the documentation and my understanding, there isn’t a built-in way to restrict the creation of service principals exclusively to account admins. And as you mentioned, the RestrictWorkspaceAdmins setting doesn’t cover thi...
I tried configure / setup DBT to authenticate with OAuth to Databricks, following the tutorials https://community.databricks.com/t5/technical-blog/using-dbt-core-with-oauth-on-azure-databricks/ba-p/46605 and https://docs.databricks.com/aws/en/partner...
Here are some things to consider: Debugging and Setting Up OAuth Authentication with dbt and Databricks Steps to Analyze and Setup Authentication: 1. Verify Credentials and Configuration: - Double-check the auth_type: oauth is correctly specified, as...
I have a Unity Catalog enabled workspace and I have full privileges including Account Admin. I would like to be able to remove the "CAN_MANAGE" privilege from the "users" group. According to the documentation, this should be possible. According to...
Hi, Can you please share the entire stacktrace to the cause of Cannot modify permissions of directory. Thanks!
I am using Databricks Asset Bundles to deploy Databricks workflows to all of my target environments (dev, staging, prod). However, I have one specific workflow that is supposed to be deployed only to the dev target environment.How can I implement tha...
Hi, also curious about this update, this would be a really helpful feature for us since we have humongous job specifications that are specific for dev and test environments. Adding these to the top level databricks.yml file is really cluttering up ou...
In the documentation about serverless compute release notes, it is stated that "Serverless compute for notebooks and jobs uses environment versions", and that "Serverless compute always runs using the most recently released version listed here.". At ...
It seems the configuration aligns with doc https://docs.databricks.com/aws/en/release-notes/serverless/environment-version-two which specifies that the Python version in 3.11.10
