cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

TalY
by New Contributor II
  • 7726 Views
  • 5 replies
  • 0 kudos

Python notebook crashes with "The Python kernel is unresponsive"

While using a Python notebook that works on my machine it crashes on the same point with the errors "The Python kernel is unresponsive" and "The Python process exited with exit code 134 (SIGABRT: Aborted).",  but with no stacktrace for debugging the ...

  • 7726 Views
  • 5 replies
  • 0 kudos
Latest Reply
TalY
New Contributor II
  • 0 kudos

I am using the following DBR 12.2 LTS (includes Apache Spark 3.3.2, Scala 2.12).Fatal error: The Python kernel is unresponsive.--------------------------------------------------------------------------- The Python process exited with exit code 134 (S...

  • 0 kudos
4 More Replies
Hani4hanuman
by New Contributor II
  • 2143 Views
  • 2 replies
  • 1 kudos

Databricks notebook issue

Hi,I'm trying to run ADF pipeline.However, it is getting fail at Notebook activity with below error.Error :NoSuchMethodError: com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.writeToServer(Lcom/microsoft/sqlserver/jdbc/ISQLServerBulkRecord;)V I think i...

  • 2143 Views
  • 2 replies
  • 1 kudos
Latest Reply
Hani4hanuman
New Contributor II
  • 1 kudos

@shan_chandra   Thanks for your reply as per your suggetion changed Databricks version from 9.1LTS to 12.2LTSBut after change this when i check library which you provided(i.e com.microsoft.azure:spark-mssql-connector_2.12:1.3.0) under Maven it is not...

  • 1 kudos
1 More Replies
lightningStrike
by New Contributor III
  • 2934 Views
  • 3 replies
  • 0 kudos

unable to install pymqi in azure databricks

Hi,I am trying to install pymqi via below command:pip install pymqi However, I am getting below error message:Python interpreter will be restarted. Collecting pymqi Using cached pymqi-1.12.10.tar.gz (91 kB) Installing build dependencies: started Inst...

lightningStrike_0-1688995466000.png
  • 2934 Views
  • 3 replies
  • 0 kudos
Latest Reply
sean_owen
Databricks Employee
  • 0 kudos

I don't think so, because it won't be specific to Databricks - this is all a property of the third party packages. And, there are billions of possible library conflicts. But this is not an example of a package conflict. It's an example of not complet...

  • 0 kudos
2 More Replies
alejandrofm
by Valued Contributor
  • 4285 Views
  • 1 replies
  • 1 kudos

Resolved! Configure job to use one cluster instance to multiple jobs

Hi! I have several tiny jobs that run in parallel and I want them to run on the same cluster:- Tasks type Python Script: I send the parameters this way to run the pyspark scripts.- Job compute cluster created as (copied JSON from Databricks Job UI)Ho...

Community Platform Discussions
cluster
job
job cluster
  • 4285 Views
  • 1 replies
  • 1 kudos
Latest Reply
KoenZandvliet
New Contributor III
  • 1 kudos

Unfortunately, running multiple jobs in parallel using a single job cluster is not supported (yet). New in databricks is the possibility to create a job that orchestrates multiple jobs. These jobs will however still use their own cluster (configurati...

  • 1 kudos
div19882021
by New Contributor
  • 937 Views
  • 1 replies
  • 1 kudos

Is there a solution that we can display the worker types based on spark version selection using api?

Is there a solution that allows us to display the worker types or driver types based on the selection of Spark version using an api?

  • 937 Views
  • 1 replies
  • 1 kudos
Latest Reply
sean_owen
Databricks Employee
  • 1 kudos

Can you clarify what you mean? Worker and driver types are not related to Spark version.

  • 1 kudos
pabloanzorenac
by New Contributor II
  • 1963 Views
  • 2 replies
  • 2 kudos

Resolved! Reduce EBS Default Volumes

By default Databricks creates 2 volumes: one with 30GB and the other one with 150GB. We have a lot of nodes in our pools and so a los of Terabytes of Volumes, but we are not making any use of them in the jobs. Is there any way to reduce the volumes? ...

  • 1963 Views
  • 2 replies
  • 2 kudos
Latest Reply
sean_owen
Databricks Employee
  • 2 kudos

Yes, EBS vols are essential for shuffle spill for example. You are probably using them!

  • 2 kudos
1 More Replies
KrishZ
by Contributor
  • 5676 Views
  • 1 replies
  • 0 kudos

Uninstalling a preinstalled python package from Databricks

[Datasets](https://pypi.org/project/datasets/) python package comes preinstalled on databricks clusters. I want to uninstall it or completely prevent it's installation when I create/start a cluster.I couldn't find any solution on stackoverflow.And I ...

  • 5676 Views
  • 1 replies
  • 0 kudos
Latest Reply
sean_owen
Databricks Employee
  • 0 kudos

@Retired_mod note that you can't actually uninstall packages in the runtime with pip.

  • 0 kudos
Chris_Shehu
by Valued Contributor III
  • 1766 Views
  • 0 replies
  • 0 kudos

Databricks Assistant HIPPA? Future Cost?

With the Public Preview of Databricks Assistant, I have a few questions. 1) If the Azure Tenet is HIPPA compliant does that compliance also include the Databricks Assistant features? 2) Right now the product is free but what will the cost be? Will we...

  • 1766 Views
  • 0 replies
  • 0 kudos
LiliL
by New Contributor
  • 1486 Views
  • 1 replies
  • 2 kudos

create UDF in pyspark

Hi, Need the help of this community, unfortunately creating udfs is not my strongest skill set.I need to create UDF that will join two tables together, the problem is that one table has two id columns Name Table has id1 and id2 Transaction Table has ...

LiliL_0-1690464091302.png LiliL_1-1690464169973.png
  • 1486 Views
  • 1 replies
  • 2 kudos
Latest Reply
Siebert_Looije
Contributor
  • 2 kudos

Hi,I am not sure if I understand your question directly but let me give it a try:- The constraint is if id2 in name table populated then join with id2: So I think you can also could first make a column called 'id' in which you get id2 if it is popula...

  • 2 kudos
Mahajan
by New Contributor II
  • 6829 Views
  • 3 replies
  • 1 kudos

Want to disable cell scrollers.

There are two scrollers visible in my notebook, 1 for cell and another is for notebook. How can i disable cell scroller sicne i am having a hard time to navigate to my code scrolling the cell every time.

  • 6829 Views
  • 3 replies
  • 1 kudos
Latest Reply
UmaMahesh1
Honored Contributor III
  • 1 kudos

Hi @Mahajan What exactly do you mean by disabling the cell scroll ?  If and all there is an option as such, then it basically means you can't scroll the cell at all and the cell view is fixed. This makes the cell redundant as at any given point of ti...

  • 1 kudos
2 More Replies
DineshKumar
by New Contributor III
  • 4108 Views
  • 6 replies
  • 0 kudos

Databricks Cluster is going down after installing the external library

 I have created a Databricks cluster with below configurations.Databricks Runtime Version13.2 ML (includes Apache Spark 3.4.0, Scala 2.12)Node typei3.xlarge30.5 GB Memory, 4 CoresI created a notebook and trying to load the Mysql table which resides i...

  • 4108 Views
  • 6 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, The below error describes that there is an issue connecting to the host from Databricks, you can find more details about the network configurations here at https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed...

  • 0 kudos
5 More Replies
KoenZandvliet
by New Contributor III
  • 1324 Views
  • 1 replies
  • 2 kudos

Resolved! Setting cluster permissions with API

I would like to update the permissions of a cluster using the API. Documentation mentions the following: patch api/2.0/permissions/{request_object_type}/{request_object_id}.Which {request_object_type} to use? ‘cluster’, ‘cluster’ and ‘compute’ are no...

  • 1324 Views
  • 1 replies
  • 2 kudos
Latest Reply
daniel_sahal
Esteemed Contributor
  • 2 kudos

@KoenZandvliet clusters is the one you should be looking for.

  • 2 kudos
NewContributor
by New Contributor III
  • 4329 Views
  • 5 replies
  • 3 kudos

Resolved! Databricks Certified Data Engineer Associate (Version 2) Exam got suspended

Hi Team,My Databricks Certified Data Engineer Associate (Version 2) exam got suspended today and it is in suspended state.I was there continuously in front of the camera and suddenly the alert appeared and support person asked me to show the full tab...

  • 4329 Views
  • 5 replies
  • 3 kudos
Latest Reply
Rob_79
New Contributor II
  • 3 kudos

Hi @Retired_mod ,I've been into the same situation as Shifa and I've also raised ticket with Databricks but no feedback yet!Can you please help on that?Cheers,Rabie

  • 3 kudos
4 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors