by
owen1
• New Contributor
- 1342 Views
- 2 replies
- 2 kudos
I set the workflow to run at 12:00 every day in the workflow, but the workflow failed with the error message below, and I don't know why.Run result unavailable: run failed with error message Unexpected failure while waiting for the cluster (0506-0233...
- 1342 Views
- 2 replies
- 2 kudos
Latest Reply
Hello @Sangwoo Lee ,As mentioned by vignesh, it seems like an infra related issue. > Does the user (which executes the job) has access to start a cluster?> Incase if it is not an access issue and Incase if you are starting a lot of workflow jobs tog...
1 More Replies
- 1683 Views
- 2 replies
- 0 kudos
I am reading 23 gb multi line json file and flattening it using udf and writing datframe as parquet using psypark.Cluster I am using is 3 node (8 core) 64gb memory with limit to go upto 8 nodes.I am able to process 7gb file with no issue and takes ar...
- 1683 Views
- 2 replies
- 0 kudos
Latest Reply
Hi @Ravi Dobariya Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
1 More Replies
- 6437 Views
- 3 replies
- 5 kudos
I'm using databricks connect to talk to a cluster on Azure. When doing a count on a dataframe I sometimes get this error message. Once I've gotten it once I don't seem to be able to get rid of it even if I restart my dev environment. ----------------...
- 6437 Views
- 3 replies
- 5 kudos
Latest Reply
Hi @Johan Rex We checked with databricks connect team, this issue can happen when the library is too large to upload, Databricks recommends that you use dbx by Databricks Labs for local development instead of Databricks Connect. Databricks plans no ...
2 More Replies
- 524 Views
- 0 replies
- 0 kudos
Avro fileJune 11, 2021Apache Avro is a data serialization system. Avro provides:Rich data structures.A compact, fast, binary data format.A container file, to store persistent data.Remote procedure call (RPC).Simple integration with dynamic languages....
- 524 Views
- 0 replies
- 0 kudos
