cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
mkieszek
by • New Contributor II
  • 1163 Views
  • 4 replies
  • 1 kudos

set access control list for job cluster

Hello,currently, we are using an init script that calls the DBW API to add "can_attach_to" permissions for a specific group to initialize the job cluster. How can we set an access control list for job clusters? Is it possible to add it to a policy?

Administration & Architecture
Reply
  • 1163 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 1 kudos

Hi @mkieszek ,You can use REST API to set access control list for job clusters. The endpoint is for doing that you can find below/api/2.0/permissions/clusters/{cluster_id}  (https://docs.databricks.com/api/workspace/clusters/setpermissions)And it's n...

  • 1 kudos
3 More Replies
funsjanssen
by • New Contributor III
  • 1783 Views
  • 0 replies
  • 7 kudos

Azure Databricks Multi Tenant Solution

Hello Everyone,For the past few months, we’ve been extensively exploring the use of Databricks as the core of our data warehousing product. We provide analytics dashboards to other organizations and are particularly interested in the Column-Level Sec...

Administration & Architecture
Reply
  • 1783 Views
  • 0 replies
  • 7 kudos
AlbertWang
by • Valued Contributor
  • 1242 Views
  • 5 replies
  • 2 kudos

Resolved! Databricks bundle custom variables feature does not work for workspace host

Hi all,I am not sure whether this is a bug. I followed the document.My databricks.yml file: bundle: name: test variables: DATABRICKS_HOST: description: The host of the Databricks workspace. DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_I...

Administration & Architecture
Reply
  • 1242 Views
  • 5 replies
  • 2 kudos
View Replies
Latest Reply
filipniziol
Esteemed Contributor
  • 2 kudos

Hi @AlbertWang ,The documentation states you cannot map workspace host: 

  • 2 kudos
4 More Replies
Janpie
by • New Contributor
  • 718 Views
  • 1 replies
  • 0 kudos

Installation of python module nevergrad with R package reticulate fails

I'm trying to run Robyn in Databricks but it fails since the python module nevergrad fails to install properly using the r package reticulate. This code still worked properly about 1,5 months ago so maybe the issue is on some new update in databricks...

Administration & Architecture
Reply
  • 718 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
macromill
New Contributor II
  • 0 kudos

My same issue soleve this answer, thank you Kaniz!

  • 0 kudos
6502
by • New Contributor III
  • 2332 Views
  • 5 replies
  • 0 kudos

In Shared cluster with unity catalog Python process can't connect

The story is that I can access a service listening on port 80 using a single node cluster, but can't do the same using a shared node cluster. I read about the `spark.databricks.pyspark.iptable.outbound.whitelisted.ports` , however, setting that : `sp...

Administration & Architecture
Reply
  • 2332 Views
  • 5 replies
  • 0 kudos
View Replies
Latest Reply
adriennn
Valued Contributor
  • 0 kudos

@6502 you first need to allow the script in the metastore configuration (navgiate to catalog, on the top of the page there's a small cogwheel, click on your catalog name), then:

  • 0 kudos
4 More Replies
PabloCSD
by • Valued Contributor
  • 1058 Views
  • 1 replies
  • 0 kudos

Resolved! Databricks Asset Bundle deploy a job: Python wheel could not be found

I created a repository for testing DAB, and I'm getting this error: run failed with error message Python wheel with name dab_test_repo could not be found. Please check the driver logs for more details This is my databricks.yml: bundle: name: dab_t...

Administration & Architecture
Reply
  • 1058 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
PabloCSD
Valued Contributor
  • 0 kudos

Fixed it:Uncommenting out: # libraries: # - whl: ./dist/*.whl 

  • 0 kudos
Behwar
by • New Contributor III
  • 1794 Views
  • 1 replies
  • 0 kudos

Databricks Azure Cross-Tenant connection to storage account

I'm currently facing a challenge with establishing a cross-tenant connection between Azure Databricks in Tenant A and a Storage Account in Tenant B. Below is the detailed setup of both tenants:| Tenant A | Tenant B || -------- | -------- || Azure Dat...

Administration & Architecture
Reply
  • 1794 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Kaib
New Contributor II
  • 0 kudos

Have you tried connecting to the storage account from the compute plane itself?I've used this in a notebook to confirm that i can access a storage account over the private endpoint, even when i can't add it as an external data source in the UI.from a...

  • 0 kudos
RossMacrae777
by • New Contributor II
  • 807 Views
  • 1 replies
  • 1 kudos

Connecting to Databricks from Workato. JDBCDriver 500593 Communication Link failure

I'm trying to connect to Databricks from Workato, to pull data in as part of a Workato Recipe.  I'm getting the following error when I test the connection:"Database bridge error: Failed since could not connect to the database - Failed to initialize p...

Administration & Architecture
Reply
  • 807 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
RossMacrae777
New Contributor II
  • 1 kudos

This turned out to be location-related.  It looks as though the location of the Workato instance, along with the particular way that our application is integrated with Workato, combine to cause the connection to fail.  When I switch to a US host, fro...

  • 1 kudos
DenJackson
by • New Contributor II
  • 747 Views
  • 1 replies
  • 1 kudos

Resolved! Can't Link Git and Databricks Community Edition

I am trying to link Git and Databricks but keep running into the error: "Error fetching credentials." I am on the Settings > Linked Accounts tab, and I cannot select anything except the "Retry" button at the bottom of the page, which does not resolve...

Administration & Architecture
Reply
  • 747 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 1 kudos

Hi @DenJackson ,Community edition it pretty limite and it seems that the option to integrate repository is disabled. If you need more capabalities you can sign up for 14-day free trial. 

  • 1 kudos
eric-c
by • New Contributor
  • 828 Views
  • 1 replies
  • 0 kudos

Alert ACL's not available

The list of available ACL's for alerts suggests there's a way to set "can run" and "can manage" permissions but neither the REST API nor the Databricks CLI show options for setting permissions. Is there a way to set acl's on alerts? If so, how?Alert ...

Administration & Architecture
Reply
  • 828 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

The API call needed to set the ACLs for an alert and other object types as (alerts | dashboards | data_sources | queries) is:https://docs.databricks.com/api/workspace/dbsqlpermissions/set

  • 0 kudos
Databrixer
by • New Contributor II
  • 1013 Views
  • 2 replies
  • 4 kudos

Changes in Databricks rest api

Hi all, Just wanted to ask if there is an announcement regarding changes on Databricks REST API?  Because I had weird experiences in using the REST API. Specifically for Query History API Last Aug 29, 2024. I created a script to pull get request in t...

Administration & Architecture
Reply
  • 1013 Views
  • 2 replies
  • 4 kudos
View Replies
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 4 kudos

Yeah, totally agree with you. It should be documented/mentioned somewhere. Or maybe the API should be versioned, so if they introduce a new version, it won't break existing workflows.Anyway, thanks for sharing!

  • 4 kudos
1 More Replies
jacobholland
by • New Contributor II
  • 683 Views
  • 2 replies
  • 0 kudos

Removing Admin Users

Hello, I am trying to deal with something I thought would be straightforward, but am hitting some walls. Basically the original user associated with my Databricks account has left my organisation and, when trying to remove / disable this user, I am m...

Administration & Architecture
Reply
  • 683 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
jacobholland
New Contributor II
  • 0 kudos

Just to follow-up; I can administer & disable OTHER account admins and do the needful in every other case, but this one user account is particularly privileged.

  • 0 kudos
1 More Replies
ragonzalez
by • New Contributor
  • 663 Views
  • 2 replies
  • 0 kudos

CLI jobs create issue

I am trying to create a job using the CLI and getting the below error attachedThe command itself was grabbed from the UI so I know it is ok 

ragonzalez_0-1726182063032.png
Administration & Architecture
Reply
  • 663 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 0 kudos

Hi @ragonzalez ,You can try to escape quotation marks and see if that helps. Something similar to below:{ \"custom_tags.test_tag\": { \"type\": \"fixed\", \"value\": \"test_value\" } }\n"And you don't have to provide whole request body to command lin...

  • 0 kudos
1 More Replies
PiotrU
by • Contributor II
  • 672 Views
  • 2 replies
  • 1 kudos

No possibility to schedule DLT once per minute

HelloI wanted to setup DLT to run every minute, previously it was possible with setting up in JSON "schedule": "1 * * * *"  - but now I see that is not accepted - is there any other workaround to make it working ?

Administration & Architecture
Reply
  • 672 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
PiotrU
Contributor II
  • 1 kudos

in that case we can attach dlt to job ... but in json for dlt - I was not able to do that still

  • 1 kudos
1 More Replies
bubbybedi
by • New Contributor II
  • 518 Views
  • 1 replies
  • 0 kudos

Databricks Workspace deleted - Need to recover

Hi ,Have created Databricks workspace on GCP. It was somehow deleted which Iam not sure . Is there anyway to recover database.It was deleted on 30-Jun-2025. Is there any retention period to recover.Thanks,Balbeer

Administration & Architecture
Reply
  • 518 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
bubbybedi
New Contributor II
  • 0 kudos

sorry typo 30-jun-2024

  • 0 kudos

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Top