cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
killjoy
by • New Contributor III
  • 5745 Views
  • 2 replies
  • 0 kudos

Unexpected failure while fetching notebook - What can we do from our side?

Hello!We got some pipelines running in Azure Data Factory that call Databricks Notebooks to run data transformations. This morning at 6:21 AM (UTC) we got an error " Unexpected failure while fetching notebook" inside a notebook that calls another one...

Data Engineering
Reply
  • 5745 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Rita Fernandes​ :Based on the error message you provided, it seems like the issue might be related to the version mismatch between the ANTLR tool used for code generation and the current runtime version. Additionally, the error message suggests that...

  • 0 kudos
1 More Replies
Toy
by • New Contributor II
  • 2768 Views
  • 3 replies
  • 0 kudos

Pipeline Error [Py4JJavaError] com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILED

I have a pipeline that used used to run successfully and now all of a sudden is returning this error that I cannot resolve: [Py4JJavaError] 

image
Data Engineering
Reply
  • 2768 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Toy
New Contributor II
  • 0 kudos

Hi Guys, You're right the problem is with the child notebook.All my notebooks are failing at this point. I can't seem to be wining with solving this error

  • 0 kudos
2 More Replies
AK98
by • New Contributor II
  • 4645 Views
  • 3 replies
  • 0 kudos

Py4JJavaError when trying to write dataframe to delta table

I'm trying to write a dataframe to a delta table and am getting this error.Im not sure what the issue is as I had no problem successfully writing other dataframes to delta tables. I attached a snippet of the data as well along with the schema:

image.png image image
Data Engineering
Reply
  • 4645 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
ramravi
Contributor II
  • 0 kudos

Does your delta tables contains all columns what your dataframe contains. If there is schema mismatch it might be a reason for failure.df.write.format("delta") \ .option("mergeSchema", "true") \ .mode("append") \ .sav...

  • 0 kudos
2 More Replies
laus
by • New Contributor III
  • 35082 Views
  • 3 replies
  • 2 kudos

Resolved! How to solve Py4JJavaError: An error occurred while calling o5082.csv. : org.apache.spark.SparkException: Job aborted. when writing to csv

Hi ,I get the error: Py4JJavaError: An error occurred while calling o5082.csv.: org.apache.spark.SparkException: Job aborted. when writing to csv.Screenshot below with detail error.Any idea how to solve it?Thanks!

Screenshot 2022-03-31 at 17.33.26
Data Engineering
Reply
  • 35082 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
Noopur_Nigam
Databricks Employee
  • 2 kudos

Please try output.coalesce(1).write.option("header","true").format("csv").save("path")It seems to be same to https://community.databricks.com/s/topic/0TO3f000000CjVqGAK/py4jjavaerror

  • 2 kudos
2 More Replies
laus
by • New Contributor III
  • 21654 Views
  • 3 replies
  • 2 kudos

Resolved! get a "Py4JJavaError: An error occurred while calling o5082.csv." when trying to save to csv file.

Hi, I'm trying to save a dataframe to csv with the code below:output.coalesce(1).write.mode('overwrite').option('header', 'true').csv(tmp_file_path) But it get "Py4JJavaError: An error occurred while calling o5082.csv." error. Any idea how to solve...

Screenshot 2022-03-31 at 17.33.13
Data Engineering
Reply
  • 21654 Views
  • 3 replies
  • 2 kudos
View Replies
Labels