cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

jacobholland
by New Contributor II
  • 1462 Views
  • 2 replies
  • 0 kudos

Removing Admin Users

Hello, I am trying to deal with something I thought would be straightforward, but am hitting some walls. Basically the original user associated with my Databricks account has left my organisation and, when trying to remove / disable this user, I am m...

  • 1462 Views
  • 2 replies
  • 0 kudos
Latest Reply
jacobholland
New Contributor II
  • 0 kudos

Just to follow-up; I can administer & disable OTHER account admins and do the needful in every other case, but this one user account is particularly privileged.

  • 0 kudos
1 More Replies
ragonzalez
by New Contributor
  • 1213 Views
  • 2 replies
  • 0 kudos

CLI jobs create issue

I am trying to create a job using the CLI and getting the below error attachedThe command itself was grabbed from the UI so I know it is ok 

ragonzalez_0-1726182063032.png
  • 1213 Views
  • 2 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 0 kudos

Hi @ragonzalez ,You can try to escape quotation marks and see if that helps. Something similar to below:{ \"custom_tags.test_tag\": { \"type\": \"fixed\", \"value\": \"test_value\" } }\n"And you don't have to provide whole request body to command lin...

  • 0 kudos
1 More Replies
PiotrU
by Contributor II
  • 1200 Views
  • 2 replies
  • 1 kudos

No possibility to schedule DLT once per minute

HelloI wanted to setup DLT to run every minute, previously it was possible with setting up in JSON "schedule": "1 * * * *"  - but now I see that is not accepted - is there any other workaround to make it working ?

  • 1200 Views
  • 2 replies
  • 1 kudos
Latest Reply
PiotrU
Contributor II
  • 1 kudos

in that case we can attach dlt to job ... but in json for dlt - I was not able to do that still

  • 1 kudos
1 More Replies
bubbybedi
by New Contributor II
  • 871 Views
  • 1 replies
  • 0 kudos

Databricks Workspace deleted - Need to recover

Hi ,Have created Databricks workspace on GCP. It was somehow deleted which Iam not sure . Is there anyway to recover database.It was deleted on 30-Jun-2025. Is there any retention period to recover.Thanks,Balbeer

  • 871 Views
  • 1 replies
  • 0 kudos
Latest Reply
bubbybedi
New Contributor II
  • 0 kudos

sorry typo 30-jun-2024

  • 0 kudos
jannikj
by New Contributor III
  • 6424 Views
  • 3 replies
  • 0 kudos

Resolved! Notebook cells stuck on "waiting to run" when using Cluster Libraries

Hey,we're observing the following problem when trying to run a notebook on a cluster with libraries installed:All notebook cells are stuck in "Waiting to run" (also ones containing only a '1+1' statement).When running the cluster without installing t...

  • 6424 Views
  • 3 replies
  • 0 kudos
Latest Reply
BhawaniD
New Contributor III
  • 0 kudos

Did you manage to fix this issue? I am facing a similar situation while running a notebook to read the XML files from the storage account. 

  • 0 kudos
2 More Replies
BhawaniD
by New Contributor III
  • 1754 Views
  • 3 replies
  • 2 kudos

Notebook is stuck and cluster goes into waiting state while using spark libraries

Hey,We have installed the com.databricks:spark-xml_2.12:0.18.0 library in our VNET-injected Databricks workspace to read XML files from a storage account. The notebook runs successfully for text files when the cluster is started without the library i...

  • 1754 Views
  • 3 replies
  • 2 kudos
Latest Reply
Witold
Honored Contributor
  • 2 kudos

Since it's a maven dependency it should be simply HTTP and port 80/443.Besides, are you aware that native XML support is included since runtime 14.3? This replaces the library spark-xml.

  • 2 kudos
2 More Replies
Redfordml
by New Contributor II
  • 1348 Views
  • 1 replies
  • 1 kudos

Resolved! Do Databricks Update the Default Python Libraries in Cluster Runtimes?

Hi all,I’ve been trying to find information about whether Databricks regularly updates the default Python libraries in their cluster runtimes. I checked two different sources but didn’t find clear details.Default python libraries in runtime 11.3 LTS ...

  • 1348 Views
  • 1 replies
  • 1 kudos
Latest Reply
Witold
Honored Contributor
  • 1 kudos

Yes, Databricks does that when releasing new versions of the runtime. Just compare the libraries of the other runtimes.

  • 1 kudos
tdk
by New Contributor III
  • 763 Views
  • 0 replies
  • 0 kudos

jar cluster install invalid authority

Hi allFirst time user - hope this is the right spot.My goal is to import and export to an onPrem Oracle database. To this end, I am attempting to upload the ojdbc8.jar to the cluster. This does not, however, go well. The error is:The error text: "Lib...

  • 763 Views
  • 0 replies
  • 0 kudos
ph1l0s0ph3r
by New Contributor II
  • 6204 Views
  • 4 replies
  • 0 kudos

Resolved! NAT Gateway with Azure Databricks

Hi all, what are the steps to enable external communication through a NAT Gateway? Our Databricks instance was created through VNET injection and did not have a NAT gateway created by default. We now want to pass external traffic through NAT gateway ...

  • 6204 Views
  • 4 replies
  • 0 kudos
Latest Reply
ph1l0s0ph3r
New Contributor II
  • 0 kudos

I made the changes and it works! I was able to communicate to the NAT Gateway IP in our external on-prem application.I noticed though that Databricks instance is not showing the NAT Gateway within Parameters. Is this an issue?   

  • 0 kudos
3 More Replies
kaihuGFS
by New Contributor II
  • 1484 Views
  • 3 replies
  • 1 kudos

Question regarding permission of azure databricks account console

We are facing an issue while onboarding pipelines to Azure Databricks.Our Databricks environment is deployed on an Azure subscription where all of us have owner permissions. However, we're encountering problems when trying to access the Azure Databri...

  • 1484 Views
  • 3 replies
  • 1 kudos
Latest Reply
MaximeGendre
New Contributor III
  • 1 kudos

Hello, Databricks admin account console is not linked to your Azure subscriptions.If you are owner of your sub, you will be admin of the workspaces instanciated in the sub. That's all.You need to find the identified Databricks admin for your company ...

  • 1 kudos
2 More Replies
singhals
by New Contributor II
  • 1041 Views
  • 1 replies
  • 1 kudos

Spark event log frequently missing rollover logs

Hi,We are currently using the event log to gain some visibility to what spark is doing under the hood but we frequently find that rollover logs are missing and we are left with a partial set. Is there a reason for this or a bug being tracked? Is ther...

  • 1041 Views
  • 1 replies
  • 1 kudos
Latest Reply
alpacas
New Contributor II
  • 1 kudos

I'm also having this issue  , it seems to occur much more frequently on azure than aws, but even on aws jobs sometimes some of the logs will be missing

  • 1 kudos
dbrixr
by New Contributor II
  • 2397 Views
  • 2 replies
  • 1 kudos

Reverse colors in dark mode

It seems that Databricks implements its dark mode by applying the invert filter so that all colors are reversed. This is problematic if one wants to create some sort of html widget or rich output since this filter is passed down to the result of disp...

  • 2397 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anushree_Tatode
Databricks Employee
  • 1 kudos

Hi @dbrixr   

  • 1 kudos
1 More Replies
rdvc
by New Contributor
  • 1042 Views
  • 1 replies
  • 0 kudos

Features tab under Machine learning shows error

Cannot resolve metastore id, go back. FeatureStoreError- Workspace Feature Store has been deprecated in the current workspace. Databricks recommends using Feature Engineering in Unity Catalog. Please reach out to system admin if you have any more que...

  • 1042 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 0 kudos

Hi @rdvc,Could it be that this one workspace that has this problem is not configured to use Unity Catalog? 

  • 0 kudos
unalluri9999
by New Contributor
  • 1050 Views
  • 1 replies
  • 0 kudos

How to access smtplib with ACL enables cluster

I am having a tough time to connect to smtp server from databricks with ACL enabled cluster , I am able to access with out ACL enabled cluster what can be done to access it via ACL enabled cluster import smtplibfrom email.mime.multipart import MIMEMu...

  • 1050 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 0 kudos

Hi @unalluri9999 ,What error do you get? By ACL you mean IP access lists or something different? Your workspace is vnet injected? 

  • 0 kudos