Re: Databricks execution failed with error state: ...

dataengineer17 · ‎07-16-2023

I am receiving this error Databricks execution failed with error state: InternalError, error message: failed to update run GlobalRunId(xx,RunId(yy))

This is appears as an error message in azure data factory when I use it to schedule a databricks notebook. When I click the notebook run, I see this message as well (Please refer to screenshots)

Appreciate any help on this as this is appearing as a false error on ADF as the dababricks notebook still run smoothly.

Ajay-Pandey · ‎07-16-2023

Hi @dataengineer17 ,

It looks like databricks internal error that you are getting, please create a support ticket with databricks .

Ajay Kumar Pandey

dataengineer17 · ‎07-17-2023

Thanks for your responses.

@Retired_mod it is not code or configuration issue. The job runs successfully for other runs, that run with issue was an exceptional case.

@ajay yes it sure looks like databricks internal issue. Was wondering if anyone has faced similiar issue

jose_gonzalez · ‎07-18-2023

Are you able to check the driver's logs for more details?

dataengineer17 · ‎07-19-2023

@jose_gonzalez yup I checked all driver logs (stdout, stderr, log4j) but nothing unusual

saipujari_spark · ‎07-19-2023

@dataengineer17

It could be coming from the internal jobs service, If the issue persists I would recommend creating a support ticket.

Thanks,
Saikrishna Pujari
Sr. Spark Technical Solutions Engineer, Databricks