ISSUE 001 · SEPTEMBER 2026

Repair the task. Not the whole pipeline.

The tip

Repair the run, not the pipeline

When a Databricks job run fails, Repair and rerun re-runs only the failed tasks and their dependents — successful tasks are skipped, not re-executed. Open the failed run, click Repair run, and the dialog shows exactly which tasks will execute before you confirm anything.

The repair creates a new run with its own version number, linked to the failed parent run — so the run page keeps a full repair history instead of burying the failure.

How to validate: after the repair, open the run's repair history. The previously failed tasks should show success, and the previously successful tasks should not have new executions — their original run entries stand.

Use this when: a mid-pipeline task failed and everything upstream already succeeded. Avoid this when: the failure is in the first task (just rerun the job), or you want the whole pipeline validated end to end after a definition change.

Implementation breakdown

The 3 a.m. failure you don't replay from scratch

Illustrative scenario: a composite built from common Jobs failure modes — not a report of one observed incident.

The failure

A nightly job runs four tasks: ingest → validate → transform → publish. At 3 a.m. transform fails on a transient cluster issue and publish is skipped. ingest took 50 minutes. Re-running the whole job replays those 50 minutes for nothing.

Step 1 — Repair only what's broken

Open the failed run and click Repair run. The dialog lists transform (failed) and publish (skipped — a downstream dependent) as the tasks to execute; ingest and validate are skipped. Confirm, and only those two run.

Step 2 — Fix the cause before you repair

Repair re-executes; it doesn't fix. If the failure is deterministic — bad SQL, a missing partition, a wrong config — fix the task first, then repair. For transient failures, set automatic retries on the task (max retries) so the next 3 a.m. failure heals itself without a manual repair at all.

Step 3 — Make repaired tasks idempotent

Repair is only safe to repeat when re-running a task produces the same result. Overwrites are naturally idempotent; appends are not — a repair after a partial append inserts the rows twice. Prefer a keyed overwrite:

(df.write
  .mode("overwrite")
  .option("replaceWhere", "dt = '2026-09-23'")
  .saveAsTable("events_daily"))

If a task must append, guard it: skip when the target already has the batch, or dedupe on a business key before writing.

The result: the pipeline recovers in the time the failed tasks take — not the whole pipeline's runtime — and the repair history shows exactly what was re-executed and why.

Dev Q&A

I repaired the run and got duplicate data. Why?

Because the failed task had already written part of its output before failing — typically an append write — and the repair wrote it again. Repair re-executes the task; it can't unwrite a partial append. Three defenses, in order: make the write idempotent (overwrite or merge on keys), make the task check-before-write, or manually clean the partial output before repairing.

Related: tasks skipped because their dependencies failed are picked up by the repair automatically — you don't select them by hand. And if a task fails transiently on a schedule, automatic task retries catch it before anyone has to click Repair run.

Recommended reading: Databricks blog: Save time with repair and rerun

About this article

Applies to: Databricks Jobs (Workflows) — notebook, Python, SQL, and other task types.
Last verified against Databricks docs: September 24, 2026.
Corrections: Sep 24, 2026 — full article restored to the archive (previously a restoration placeholder).