Databricks Community

jeremy98

Hi community,My team and I are facing an issue with the Parameter Values (see the title of this discussion) being passed through each task of a job. Unfortunately, this causes our job run to fail.Do you have any suggestions on how to handle parameter...

jeremy98

Hi community,How to overwritten the job parameter inside the job task? Because seems that the job parameter has a higher priority than a task parameter although it is overwritten

jeremy98

Hi community,I’d like to handle the case when one of our Databricks jobs fails. From the documentation, I understand that the HTTP response from Databricks will look like this:{ "event_type": "jobs.on_failure", "workspace_id": "your_workspace_id"...

jeremy98 · 08-21-2025

Hi community,As the title suggests, I'm looking for a smart way to determine which runs in a for-loop task succeeded and which didn’t, so I can use that information in the next task.Summary:I have a for-loop task that runs multiple items (e.g., run1,...

jeremy98 · 08-12-2025

Hi community,My team and I have been occasionally experiencing INTERNAL_ERROR events in Databricks. We have a job that runs on a schedule, but the start times vary. Sometimes, when the job is triggered, the underlying cluster fails to start for some ...

jeremy98

Hi Pilsner,Thanks for your response, the issue is that I need to know it before. In this case, we need to set inside a notebook for example the task values. I want to be able to set it at task value. I think it is not provided from Databricks.

jeremy98

Hi syz:),Yes ahah it's weird in my opinion, but yes your solution is fine!

jeremy98

Hi,Thanks for your answer but I mean, I have 4 tasks of ingestion that took a key parameter from the job parameter definition level. But now I want that 3 of these tasks consider to take the same key parameter but the last task a new overwritten..Som...

jeremy98

bc this:is not correct I suppose, right?

jeremy98

Hi, thanks for your answer! But this couldn't cause some problems on API limits? Because sometimes we are iterating over 100 inputs.And how do you set the key and value? And how to retrieve them in the next task in batch?

Databricks Community

User Stats

User Activity

Run failed with error message Failed to resolve references: Parameter values exceed the size limit

How to overwritten job parameter task inside a job task

how to manage the databricks failure notifications to slack webhook?

Is there a way to discover in the next task if the previous for loop task has some...

how to manage a dynamic scheduled job if an INTERNAL_ERROR occurs?

Re: How to overwritten job parameter task inside a job task

Re: How to overwritten job parameter task inside a job task

Re: How to overwritten job parameter task inside a job task

Re: how to manage the databricks failure notifications to slack webhook?

Re: Is there a way to discover in the next task if the previous for loop task has some...