<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: Retries not triggered for a Run Job task in Data Engineering</title>
    <link>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176706#M57192</link>
    <description>&lt;P&gt;Hi&amp;nbsp; &lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/272497"&gt;@alejandrorr&lt;/a&gt;,&lt;/P&gt;&lt;P&gt;Adding to ashishshivatare's replies: yes, the docs say retries are supported on Run Job tasks. Step 6 on the Run Job task page says "To configure retries, run-duration or streaming-backlog thresholds, or notifications, see Advanced task settings." The bundle docs list max_retries, min_retry_interval_millis and retry_on_timeout under "Other task settings", which "allow you to configure behaviors for all tasks". The only task type the bundle docs say "does not support retries" is the condition task. Your 12-hour interval (43200000 ms) is within the int32 range, and the API docs list no maximum for that field, so the docs give no reason to reject it.&lt;BR /&gt;So the fields missing from jobs get, and the missing Retries option in the UI, are not what the docs describe. Please check two things. First, run databricks -v. If the CLI is old, update it and deploy again (this is a guess, not a documented cause). Then run databricks jobs get JOB_ID again and look at the finance task. If the fields are still missing, open a support ticket with the bundle validate JSON and the jobs get JSON that you posted here.&lt;BR /&gt;Until then, you can put the retry settings on the tasks of the child job (p06_pipeline_monthly_finance) itself. You can also repair the failed parent run. The repair docs say a repair re-runs "only the subset of unsuccessful tasks and any dependent tasks", and repair works only for jobs with two or more tasks. Note that a repair of the Run Job task runs the whole child job again.&lt;/P&gt;&lt;P&gt;Thanks, Sundar&lt;/P&gt;&lt;P&gt;Docs: &lt;A href="https://docs.databricks.com/aws/en/jobs/tasks/run-job" target="_blank"&gt;https://docs.databricks.com/aws/en/jobs/tasks/run-job&lt;/A&gt;&lt;/P&gt;</description>
    <pubDate>Wed, 18 Nov 2026 02:39:00 GMT</pubDate>
    <dc:creator>Yougasundar</dc:creator>
    <dc:date>2026-11-18T02:39:00Z</dc:date>
    <item>
      <title>Retries not triggered for a Run Job task</title>
      <link>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176643#M57185</link>
      <description>&lt;P&gt;&lt;SPAN&gt;Is it possible to configure retries for a Databricks task of type &lt;/SPAN&gt;&lt;STRONG&gt;&lt;SPAN&gt;Run Job&lt;/SPAN&gt;&lt;/STRONG&gt;&lt;SPAN&gt;?&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;I configured &lt;/SPAN&gt;max_retries&lt;SPAN&gt; and &lt;/SPAN&gt;min_retry_interval_millis&lt;SPAN&gt; on the task in a Databricks Asset Bundle, as I would for a notebook task. However, when the called job fails, the Run Job task does not retry. Also, the retry fields do not appear in the Jobs API &lt;/SPAN&gt;get&lt;SPAN&gt; response.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;Is this configuration supported for Run Job tasks? If so, how should retries be configured?&lt;/SPAN&gt;&lt;/P&gt;</description>
      <pubDate>Tue, 17 Nov 2026 10:15:17 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176643#M57185</guid>
      <dc:creator>alejandrorr</dc:creator>
      <dc:date>2026-11-17T10:15:17Z</dc:date>
    </item>
    <item>
      <title>Re: Retries not triggered for a Run Job task</title>
      <link>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176644#M57186</link>
      <description>&lt;P&gt;Hi&amp;nbsp;@alejandorr&lt;/P&gt;&lt;P&gt;Yes, retries are supported on Run Job tasks. The Run Job task docs say to configure retries, run-duration or streaming-backlog thresholds, or notifications, see Advanced task settings, the same place you’d set them for a notebook task. So the setting itself isn’t the problem. &lt;SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/jobs/tasks/run-job" target="_blank" rel="noopener noreferrer"&gt;databricks&lt;/A&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;The clue is that the retry fields don’t show up in the Jobs API get response. That means they never reached the deployed job, so the parent task has no retry policy to apply. A few things to check:&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;Placement in the YAML:&amp;nbsp;max_retries and min_retry_interval_millis belong at the task level, as siblings of run_job_task. If they’re indented under run_job_task, they won’t be applied. It should look like this:&lt;/LI&gt;&lt;/OL&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&amp;nbsp;&lt;/DIV&gt;&lt;/DIV&gt;&lt;DIV&gt;yaml&lt;/DIV&gt;&lt;DIV&gt;&lt;PRE&gt;&lt;SPAN&gt;resources:
  jobs:
    parent_job:
      name: parent-job
      tasks:
        - task_key: call_child
          max_retries: 3
          min_retry_interval_millis: 60000
          retry_on_timeout: false
          run_job_task:
            job_id: ${resources.jobs.child_job.id}&lt;/SPAN&gt;&lt;/PRE&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;OL&gt;&lt;LI&gt;Check what the bundle actually sends: Run databricks bundle validate --output json and look for max_retries on that task. If it’s missing there, the problem is in the YAML. Also confirm you deployed to the target you’re checking in the API.&lt;/LI&gt;&lt;LI&gt;Continuous trigger: If the parent job uses a continuous trigger, task retry policies don’t apply. Continuous jobs use their own retry mode with exponential backoff instead.&lt;/LI&gt;&lt;LI&gt;Isolate bundle vs. platform:&amp;nbsp;Add retries to the Run Job task by hand in the UI, then call the get API. If the fields show up, the platform supports it and the issue is in the bundle config.&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;One design point to keep in mind: a retry on the Run Job task reruns the whole child job, including tasks that already succeeded. If only one task in the child job tends to fail, it’s often better to put retries on that task inside the child job, so only the failing work reruns.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Tue, 17 Nov 2026 10:30:23 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176644#M57186</guid>
      <dc:creator>ashishshivatare</dc:creator>
      <dc:date>2026-11-17T10:30:23Z</dc:date>
    </item>
    <item>
      <title>Re: Retries not triggered for a Run Job task</title>
      <link>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176659#M57187</link>
      <description>&lt;P&gt;&lt;SPAN&gt;Hi,&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;Thank you very much for your quick response.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;I checked the bundle configuration as suggested. The retry fields appear at the task level in the output of &lt;/SPAN&gt;databricks bundle validate --output json&lt;SPAN&gt;. For example, the &lt;/SPAN&gt;finance&lt;SPAN&gt; task is configured as follows:&lt;/SPAN&gt;&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;max_retries	:	10
min_retry_interval_millis	:	43200000
retry_on_timeout	:	true
	run_job_task		{2}
job_id	:	${resources.jobs.p06_pipeline_monthly_finance.id}
	job_parameters		{4}
ingestion_date	:	{{job.parameters.ingestion_date}}
reference_period_yyyymm	:	{{job.parameters.reference_period_yyyymm}}
reprocess_reason	:	{{job.parameters.reprocess_reason}}
table_suffix	:	{{job.parameters.table_suffix}}
task_key	:	finance&lt;/LI-CODE&gt;&lt;P&gt;&lt;SPAN&gt;After running databricks bundle deploy, I can see the retry attributes in the bundle files in the Databricks workspace. However, when I call the Jobs API to retrieve the deployed job, the finance task does not include those attributes:&lt;BR /&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;disabled	:	false
	email_notifications		{0}
run_if	:	ALL_SUCCESS
	run_job_task		{2}
job_id	:	513004093798668+018226-06-15T09:09:58.668Z
	job_parameters		{4}
ingestion_date	:	{{job.parameters.ingestion_date}}
reference_period_yyyymm	:	{{job.parameters.reference_period_yyyymm}}
reprocess_reason	:	{{job.parameters.reprocess_reason}}
table_suffix	:	{{job.parameters.table_suffix}}
task_key	:	finance
timeout_seconds	:	0&lt;/LI-CODE&gt;&lt;P&gt;&lt;SPAN&gt;&lt;BR /&gt;I also checked the task in the Databricks UI, but I cannot see an option to configure &lt;/SPAN&gt;&lt;STRONG&gt;&lt;SPAN&gt;Retries&lt;/SPAN&gt;&lt;/STRONG&gt;&lt;SPAN&gt; for a Run Job task. That option does appear for notebook tasks. The job is managed by a bundle and has &lt;/SPAN&gt;edit_mode: UI_LOCKED. The parent job uses a scheduled trigger, not a continuous trigger.&lt;/P&gt;</description>
      <pubDate>Tue, 17 Nov 2026 11:26:46 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176659#M57187</guid>
      <dc:creator>alejandrorr</dc:creator>
      <dc:date>2026-11-17T11:26:46Z</dc:date>
    </item>
    <item>
      <title>Re: Retries not triggered for a Run Job task</title>
      <link>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176660#M57188</link>
      <description>&lt;P&gt;Hi&amp;nbsp;&lt;SPAN class=""&gt;&lt;A class="" href="https://community.databricks.com/t5/user/viewprofilepage/user-id/272497" target="_blank" rel="noopener" aria-label="View Profile of alejandrorr"&gt;&lt;SPAN class=""&gt;alejandrorr&lt;/SPAN&gt;&lt;/A&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;Thanks for checking all of that, it rules out the bundle. bundle validate shows the retry fields at the task level, the deployed bundle files have them, and the Jobs API returns the task without them. So the backend is silently dropping retry settings on this Run Job task. The missing Retries option in the UI points the same way.&lt;/P&gt;&lt;P&gt;This doesn’t match the Run Job task documentation, which says retries can be configured under Advanced task settings. I’d raise a support ticket with the job ID, the bundle validate output and the API get response. Either the feature isn’t enabled in your workspace yet, or the docs are ahead of the platform.&lt;/P&gt;&lt;P&gt;Two quick checks before you open the ticket:&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;In a scratch job that isn’t bundle-managed, add a Run Job task in the UI and see whether Retries appears. If it doesn’t, that confirms it’s a platform limitation and not related to UI_LOCKED.&lt;/LI&gt;&lt;LI&gt;Redeploy with a small min_retry_interval_millis (say 60000) and check the API again. Your current value is 12 hours. If the fields show up with a smaller value, the long interval is being rejected rather than retries being unsupported.&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;In the meantime, the most reliable workaround is to put the retries on the tasks inside the child job (p06_pipeline_monthly_finance) rather than on the Run Job task. That’s often the better design anyway, because only the failing task reruns instead of the whole child pipeline:&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;PRE&gt;&lt;SPAN&gt;resources:
  jobs:
    p06_pipeline_monthly_finance:
      tasks:
        - task_key: &amp;lt;task_that_fails&amp;gt;
          max_retries: 10
          min_retry_interval_millis: 43200000
          retry_on_timeout: true
          # existing task settings&lt;/SPAN&gt;&lt;/PRE&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;P&gt;If you need the retry at the parent level specifically, another option is to replace the Run Job task with a small notebook task that triggers the child job through the Databricks SDK (run_now_and_wait), raises an exception if the child run doesn’t succeed, and carries the retry policy itself. Notebook tasks do support retries, so the parent retries the whole child run that way.&lt;/P&gt;</description>
      <pubDate>Tue, 17 Nov 2026 11:33:23 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176660#M57188</guid>
      <dc:creator>ashishshivatare</dc:creator>
      <dc:date>2026-11-17T11:33:23Z</dc:date>
    </item>
    <item>
      <title>Re: Retries not triggered for a Run Job task</title>
      <link>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176706#M57192</link>
      <description>&lt;P&gt;Hi&amp;nbsp; &lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/272497"&gt;@alejandrorr&lt;/a&gt;,&lt;/P&gt;&lt;P&gt;Adding to ashishshivatare's replies: yes, the docs say retries are supported on Run Job tasks. Step 6 on the Run Job task page says "To configure retries, run-duration or streaming-backlog thresholds, or notifications, see Advanced task settings." The bundle docs list max_retries, min_retry_interval_millis and retry_on_timeout under "Other task settings", which "allow you to configure behaviors for all tasks". The only task type the bundle docs say "does not support retries" is the condition task. Your 12-hour interval (43200000 ms) is within the int32 range, and the API docs list no maximum for that field, so the docs give no reason to reject it.&lt;BR /&gt;So the fields missing from jobs get, and the missing Retries option in the UI, are not what the docs describe. Please check two things. First, run databricks -v. If the CLI is old, update it and deploy again (this is a guess, not a documented cause). Then run databricks jobs get JOB_ID again and look at the finance task. If the fields are still missing, open a support ticket with the bundle validate JSON and the jobs get JSON that you posted here.&lt;BR /&gt;Until then, you can put the retry settings on the tasks of the child job (p06_pipeline_monthly_finance) itself. You can also repair the failed parent run. The repair docs say a repair re-runs "only the subset of unsuccessful tasks and any dependent tasks", and repair works only for jobs with two or more tasks. Note that a repair of the Run Job task runs the whole child job again.&lt;/P&gt;&lt;P&gt;Thanks, Sundar&lt;/P&gt;&lt;P&gt;Docs: &lt;A href="https://docs.databricks.com/aws/en/jobs/tasks/run-job" target="_blank"&gt;https://docs.databricks.com/aws/en/jobs/tasks/run-job&lt;/A&gt;&lt;/P&gt;</description>
      <pubDate>Wed, 18 Nov 2026 02:39:00 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/retries-not-triggered-for-a-run-job-task/m-p/176706#M57192</guid>
      <dc:creator>Yougasundar</dc:creator>
      <dc:date>2026-11-18T02:39:00Z</dc:date>
    </item>
  </channel>
</rss>

