Hi @Malthe,

The ISOLATION_STARTUP_FAILURE.SANDBOX_STARTUP error you are seeing is a transient infrastructure-level issue where the serverless execution sandbox fails its internal liveness check before your code even starts running. Since the error message itself says "Please contact Databricks support" and includes SQLSTATE: XXKSS, this is recognized as an internal platform error rather than something caused by your code or configuration.

WHAT IS HAPPENING

On serverless compute, each task runs inside an isolated sandbox container. During startup, the platform performs a liveness check (verifying a process is listening on port 8000). When that check exceeds its deadline, the sandbox is marked as failed and the task errors out with the message you posted. Because the sandbox never fully started, the retry often hits the same transient condition, especially if the underlying infrastructure is temporarily constrained.

RECOMMENDED ACTIONS

1. Configure task-level retries with a delay: In your job configuration, add a retry policy to each task. Serverless jobs can auto-optimize retries, but you can also set explicit retries (e.g., 2-3 max retries) with a retry interval. The interval is calculated in milliseconds between the start of the failed run and the next retry. Adding a delay (e.g., 60000 ms) gives the platform time to recover before the next attempt. You can configure this in the job UI by clicking "+ Add" next to "Retries" in the task panel, or via the Jobs API retry_policy field.

2. Open a support ticket: Since the error explicitly says "Please contact Databricks support" and includes an internal exception class, Databricks Support can correlate the timestamps with backend telemetry to determine whether this was tied to a specific deployment rollout, a regional capacity event, or another root cause. Given that you are on Azure Databricks, you can open a ticket directly through the Azure portal. Include the job run IDs, task run IDs, timestamps (with timezone), and the workspace URL so support can look up the exact sandbox that failed.

3. Monitor the Databricks status page: You can check service health at https://status.databricks.com (for AWS workspaces) or https://status.azuredatabricks.net (for Azure). You can also subscribe to email, webhook, or Slack notifications for your region so you are alerted proactively when there is a platform-level incident.

4. Review your job for timeout settings: You mentioned these failures take several minutes before they complete as failed. You can set an execution timeout using the spark.databricks.execution.timeout Spark property for serverless jobs to cap how long a task waits before being marked as timed out. Combining this with a retry policy ensures that a sandbox startup stall does not block your entire ETL pipeline for an extended period.

REGARDING THE CUSTOM SPARK VERSION SUGGESTION

The suggestion from @sandy_123 about pinning to a custom Spark version applies to classic (non-serverless) compute where you control the Databricks Runtime version. On serverless compute, you cannot pin a specific runtime version because serverless is a versionless product where Databricks automatically manages the runtime. If you also experience this error on classic compute clusters, pinning to a known-good DBR version can be a valid temporary workaround while a fix is rolled out.

REGARDING THE DELTA MERGE CONTEXT

You noted this keeps happening on the same Delta Lake merge task. If the merge operation is particularly large or resource-intensive, the sandbox may be more susceptible to startup timeouts under load. Consider whether breaking that merge into smaller batches or optimizing the merge predicate could help reduce the resource pressure at startup time.

SUMMARY

- This is a transient platform-level error, not caused by your code
- Add task retries with an interval delay to make your ETL more resilient
- Open a support ticket with run IDs and timestamps for root cause analysis
- Subscribe to Databricks status notifications for your region
- On serverless, you cannot pin a custom runtime version, but retries and timeouts are your primary levers

Documentation references:
- Repair and retry failed jobs: https://docs.databricks.com/en/jobs/repair-job-failures.html
- Configure tasks: https://docs.databricks.com/en/jobs/configure-task.html
- Serverless compute overview: https://learn.microsoft.com/en-us/azure/databricks/compute/serverless/
- Serverless compute limitations: https://learn.microsoft.com/en-us/azure/databricks/compute/serverless/limitations
- Databricks status page (Azure): https://status.azuredatabricks.net

* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.

If this answer resolves your question, could you mark it as "Accept as Solution"? That helps other users quickly find the correct fix.