cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Lakehouse Architecture


Forum Posts

Behwar
by New Contributor II
  • 1234 Views
  • 1 replies
  • 0 kudos

Databricks Azure Cross-Tenant connection to storage account

I'm currently facing a challenge with establishing a cross-tenant connection between Azure Databricks in Tenant A and a Storage Account in Tenant B. Below is the detailed setup of both tenants:| Tenant A | Tenant B || -------- | -------- || Azure Dat...

  • 1234 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaib
New Contributor II
  • 0 kudos

Have you tried connecting to the storage account from the compute plane itself?I've used this in a notebook to confirm that i can access a storage account over the private endpoint, even when i can't add it as an external data source in the UI.from a...

  • 0 kudos
RossMacrae777
by New Contributor II
  • 508 Views
  • 1 replies
  • 1 kudos

Connecting to Databricks from Workato. JDBCDriver 500593 Communication Link failure

I'm trying to connect to Databricks from Workato, to pull data in as part of a Workato Recipe.  I'm getting the following error when I test the connection:"Database bridge error: Failed since could not connect to the database - Failed to initialize p...

  • 508 Views
  • 1 replies
  • 1 kudos
Latest Reply
RossMacrae777
New Contributor II
  • 1 kudos

This turned out to be location-related.  It looks as though the location of the Workato instance, along with the particular way that our application is integrated with Workato, combine to cause the connection to fail.  When I switch to a US host, fro...

  • 1 kudos
DenJackson
by New Contributor II
  • 508 Views
  • 1 replies
  • 1 kudos

Resolved! Can't Link Git and Databricks Community Edition

I am trying to link Git and Databricks but keep running into the error: "Error fetching credentials." I am on the Settings > Linked Accounts tab, and I cannot select anything except the "Retry" button at the bottom of the page, which does not resolve...

  • 508 Views
  • 1 replies
  • 1 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 1 kudos

Hi @DenJackson ,Community edition it pretty limite and it seems that the option to integrate repository is disabled. If you need more capabalities you can sign up for 14-day free trial. 

  • 1 kudos
eric-c
by New Contributor
  • 593 Views
  • 1 replies
  • 0 kudos

Alert ACL's not available

The list of available ACL's for alerts suggests there's a way to set "can run" and "can manage" permissions but neither the REST API nor the Databricks CLI show options for setting permissions. Is there a way to set acl's on alerts? If so, how?Alert ...

  • 593 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

The API call needed to set the ACLs for an alert and other object types as (alerts | dashboards | data_sources | queries) is:https://docs.databricks.com/api/workspace/dbsqlpermissions/set

  • 0 kudos
Databrixer
by New Contributor II
  • 836 Views
  • 2 replies
  • 4 kudos

Changes in Databricks rest api

Hi all, Just wanted to ask if there is an announcement regarding changes on Databricks REST API?  Because I had weird experiences in using the REST API. Specifically for Query History API Last Aug 29, 2024. I created a script to pull get request in t...

  • 836 Views
  • 2 replies
  • 4 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 4 kudos

Yeah, totally agree with you. It should be documented/mentioned somewhere. Or maybe the API should be versioned, so if they introduce a new version, it won't break existing workflows.Anyway, thanks for sharing!

  • 4 kudos
1 More Replies
jacobholland
by New Contributor II
  • 440 Views
  • 2 replies
  • 0 kudos

Removing Admin Users

Hello, I am trying to deal with something I thought would be straightforward, but am hitting some walls. Basically the original user associated with my Databricks account has left my organisation and, when trying to remove / disable this user, I am m...

  • 440 Views
  • 2 replies
  • 0 kudos
Latest Reply
jacobholland
New Contributor II
  • 0 kudos

Just to follow-up; I can administer & disable OTHER account admins and do the needful in every other case, but this one user account is particularly privileged.

  • 0 kudos
1 More Replies
ragonzalez
by New Contributor
  • 529 Views
  • 2 replies
  • 0 kudos

CLI jobs create issue

I am trying to create a job using the CLI and getting the below error attachedThe command itself was grabbed from the UI so I know it is ok 

ragonzalez_0-1726182063032.png
  • 529 Views
  • 2 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @ragonzalez ,You can try to escape quotation marks and see if that helps. Something similar to below:{ \"custom_tags.test_tag\": { \"type\": \"fixed\", \"value\": \"test_value\" } }\n"And you don't have to provide whole request body to command lin...

  • 0 kudos
1 More Replies
PiotrU
by Contributor II
  • 446 Views
  • 2 replies
  • 1 kudos

No possibility to schedule DLT once per minute

HelloI wanted to setup DLT to run every minute, previously it was possible with setting up in JSON "schedule": "1 * * * *"  - but now I see that is not accepted - is there any other workaround to make it working ?

  • 446 Views
  • 2 replies
  • 1 kudos
Latest Reply
PiotrU
Contributor II
  • 1 kudos

in that case we can attach dlt to job ... but in json for dlt - I was not able to do that still

  • 1 kudos
1 More Replies
bubbybedi
by New Contributor II
  • 379 Views
  • 1 replies
  • 0 kudos

Databricks Workspace deleted - Need to recover

Hi ,Have created Databricks workspace on GCP. It was somehow deleted which Iam not sure . Is there anyway to recover database.It was deleted on 30-Jun-2025. Is there any retention period to recover.Thanks,Balbeer

  • 379 Views
  • 1 replies
  • 0 kudos
Latest Reply
bubbybedi
New Contributor II
  • 0 kudos

sorry typo 30-jun-2024

  • 0 kudos
unalluri9999
by New Contributor
  • 867 Views
  • 0 replies
  • 0 kudos

We are getting AttributeError: module 'numpy.typing' has no attribute 'NDArray' with tidal jobs

File "/databricks/python/lib/python3.8/site-packages/PIL/_typing.py", line 10, in <module>NumpyArray = npt.NDArray[Any]AttributeError: module 'numpy.typing' has no attribute 'NDArray' we are using 10.4 runtime version , please suggest 

  • 867 Views
  • 0 replies
  • 0 kudos
jannikj
by New Contributor III
  • 3923 Views
  • 3 replies
  • 0 kudos

Resolved! Notebook cells stuck on "waiting to run" when using Cluster Libraries

Hey,we're observing the following problem when trying to run a notebook on a cluster with libraries installed:All notebook cells are stuck in "Waiting to run" (also ones containing only a '1+1' statement).When running the cluster without installing t...

  • 3923 Views
  • 3 replies
  • 0 kudos
Latest Reply
BhawaniD
New Contributor II
  • 0 kudos

Did you manage to fix this issue? I am facing a similar situation while running a notebook to read the XML files from the storage account. 

  • 0 kudos
2 More Replies
BhawaniD
by New Contributor II
  • 600 Views
  • 3 replies
  • 2 kudos

Notebook is stuck and cluster goes into waiting state while using spark libraries

Hey,We have installed the com.databricks:spark-xml_2.12:0.18.0 library in our VNET-injected Databricks workspace to read XML files from a storage account. The notebook runs successfully for text files when the cluster is started without the library i...

  • 600 Views
  • 3 replies
  • 2 kudos
Latest Reply
Witold
Honored Contributor
  • 2 kudos

Since it's a maven dependency it should be simply HTTP and port 80/443.Besides, are you aware that native XML support is included since runtime 14.3? This replaces the library spark-xml.

  • 2 kudos
2 More Replies
Redfordml
by New Contributor II
  • 543 Views
  • 1 replies
  • 1 kudos

Resolved! Do Databricks Update the Default Python Libraries in Cluster Runtimes?

Hi all,I’ve been trying to find information about whether Databricks regularly updates the default Python libraries in their cluster runtimes. I checked two different sources but didn’t find clear details.Default python libraries in runtime 11.3 LTS ...

  • 543 Views
  • 1 replies
  • 1 kudos
Latest Reply
Witold
Honored Contributor
  • 1 kudos

Yes, Databricks does that when releasing new versions of the runtime. Just compare the libraries of the other runtimes.

  • 1 kudos
tdk
by New Contributor III
  • 392 Views
  • 0 replies
  • 0 kudos

jar cluster install invalid authority

Hi allFirst time user - hope this is the right spot.My goal is to import and export to an onPrem Oracle database. To this end, I am attempting to upload the ojdbc8.jar to the cluster. This does not, however, go well. The error is:The error text: "Lib...

  • 392 Views
  • 0 replies
  • 0 kudos
ph1l0s0ph3r
by New Contributor II
  • 1608 Views
  • 4 replies
  • 0 kudos

Resolved! NAT Gateway with Azure Databricks

Hi all, what are the steps to enable external communication through a NAT Gateway? Our Databricks instance was created through VNET injection and did not have a NAT gateway created by default. We now want to pass external traffic through NAT gateway ...

  • 1608 Views
  • 4 replies
  • 0 kudos
Latest Reply
ph1l0s0ph3r
New Contributor II
  • 0 kudos

I made the changes and it works! I was able to communicate to the NAT Gateway IP in our external on-prem application.I noticed though that Databricks instance is not showing the NAT Gateway within Parameters. Is this an issue?   

  • 0 kudos
3 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels