cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

owen1
by New Contributor
  • 726 Views
  • 2 replies
  • 2 kudos

workflow cluster was create error

I set the workflow to run at 12:00 every day in the workflow, but the workflow failed with the error message below, and I don't know why.Run result unavailable: run failed with error message Unexpected failure while waiting for the cluster (0506-0233...

  • 726 Views
  • 2 replies
  • 2 kudos
Latest Reply
Murthy1
Contributor II
  • 2 kudos

Hello @Sangwoo Lee​ ,As mentioned by vignesh, it seems like an infra related issue. > Does the user (which executes the job) has access to start a cluster?> Incase if it is not an access issue and Incase if you are starting a lot of workflow jobs tog...

  • 2 kudos
1 More Replies
rdobbss
by New Contributor II
  • 918 Views
  • 2 replies
  • 0 kudos

RPC Disassociate error due to container threshold exceeding and garbage collector error when reading 23 gb multiline JSON file.

I am reading 23 gb multi line json file and flattening it using udf and writing datframe as parquet using psypark.Cluster I am using is 3 node (8 core) 64gb memory with limit to go upto 8 nodes.I am able to process 7gb file with no issue and takes ar...

  • 918 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Ravi Dobariya​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 0 kudos
1 More Replies
JohanRex
by New Contributor II
  • 3449 Views
  • 4 replies
  • 5 kudos

Resolved! IllegalArgumentException: requirement failed: Result for RPC Some(e100cace-3836-4461-8902-80b3744fcb6b) lost, please retry your request.

I'm using databricks connect to talk to a cluster on Azure. When doing a count on a dataframe I sometimes get this error message. Once I've gotten it once I don't seem to be able to get rid of it even if I restart my dev environment. ----------------...

  • 3449 Views
  • 4 replies
  • 5 kudos
Latest Reply
Kaniz
Community Manager
  • 5 kudos

Hi @Johan Rex​ , We haven't heard from you on my last response, and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will respond with ...

  • 5 kudos
3 More Replies
User16826994223
by Honored Contributor III
  • 278 Views
  • 0 replies
  • 0 kudos

Avro fileJune 11, 2021Apache Avro is a data serialization system. Avro provides:Rich data structures.A compact, fast, binary data format.A container f...

Avro fileJune 11, 2021Apache Avro is a data serialization system. Avro provides:Rich data structures.A compact, fast, binary data format.A container file, to store persistent data.Remote procedure call (RPC).Simple integration with dynamic languages....

  • 278 Views
  • 0 replies
  • 0 kudos
Labels