cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
dng
by • New Contributor III
  • 5865 Views
  • 8 replies
  • 11 kudos

Databricks JDBC Driver v2.6.29 Cloud Fetch failing for Windows Operating System

Hi everyone, I've been stuck for the past two days on this issue with my Databricks JDBC driver and I'm hoping someone can give me more insight into how to troubleshoot. I am using the Databricks JDBC driver in RStudio and the connection was working ...

Data Engineering
Reply
  • 5865 Views
  • 8 replies
  • 11 kudos
View Replies
Latest Reply
Prabakar
Esteemed Contributor III
  • 11 kudos

@Debbie Ng​ From your message I see there was a windows update and this failure started. based on the conversation you tried latest version of the driver and still you face the problem. I believe this is something related to the Java version compatib...

  • 11 kudos
7 More Replies
J_
by • New Contributor II
  • 8899 Views
  • 7 replies
  • 6 kudos

Resolved! Clusters stuck on pending indefinitely (community edition)

From yesterday, suddenly clusters do not start and are in the pending state indefinitely (more than 30 minutes). From a previous post, I tried to add 443 port to the firewall but it doesn't help. In the clusters page, the message says: Finding instan...

Data Engineering
Reply
  • 8899 Views
  • 7 replies
  • 6 kudos
View Replies
Latest Reply
Reet
New Contributor II
  • 6 kudos

I am also having same issue and there seems to be no outage.....

  • 6 kudos
6 More Replies
cralle
by • New Contributor II
  • 5216 Views
  • 7 replies
  • 2 kudos

Resolved! Cannot display DataFrame when I filter by length

I have a DataFrame that I have created based on a couple of datasets and multiple operations. The DataFrame has multiple columns, one of which is a array of strings. But when I take the DataFrame and try to filter based upon the size of this array co...

image image
Data Engineering
Reply
  • 5216 Views
  • 7 replies
  • 2 kudos
View Replies
Latest Reply
-werners-
Esteemed Contributor III
  • 2 kudos

strange, works fine here. what version of databricks are you on?What you could do to identify the issue is to output the query plan (.explain).And also creating a new df for each transformation could help. Like that you can check step by step where...

  • 2 kudos
6 More Replies
Carneiro
by • New Contributor II
  • 6464 Views
  • 3 replies
  • 3 kudos

Resolved! Stuck in "Running Command ..."

Hi, Since yesterday, without a known reason, some commands that used to run daily are now stuck in a "Running command" state. Commands as:dataframe.toPandas() dataframe.show(n=1) dataframe.description() dataframe.write.format("csv").save(location) ge...

Data Engineering
Reply
  • 6464 Views
  • 3 replies
  • 3 kudos
View Replies
Latest Reply
jose_gonzalez
Moderator
  • 3 kudos

Hi @Luiz Carneiro​ ,Could you split your Spark's actions into more CMD (paragraphs) and run one at a time to check where it could be taking the extra time. Also, Pandas only runs on your driver. Have you try to use Python or Scala APIs instead? in ca...

  • 3 kudos
2 More Replies
Labels