Honest answers first, then a debugging path — because part of this simply isn't documented, and I'd rather say so than guess.
1. Is the SSM Agent expected in DBR images? I found no Databricks documentation committing to include it, and your own finding (binary absent on 17.3) matches that. Databricks builds its own hardened Ubuntu images; assume the agent is not there unless Databricks says otherwise. Note the stock Ubuntu 24.04 AMIs ship the agent via snap — Databricks images are not stock Ubuntu, so that expectation doesn't carry over.
2. Init scripts are the supported way to install software on cluster nodes in general, yes — but I found nothing from Databricks blessing (or forbidding) the SSM Agent specifically. "Not documented" is the accurate answer.
3/4. Before fighting the install, I'd test one hypothesis that would explain everything you're seeing: does your init script actually run in the same environment as the EC2 host? Attach a notebook to the test cluster and run:
%sh
ps -p 1 -o comm= # is PID 1 systemd?
which amazon-ssm-agent
systemctl status amazon-ssm-agent 2>&1 | head
cat /var/log/amazon/ssm/amazon-ssm-agent.log 2>/dev/null | tail -50
If PID 1 isn't systemd, you're inside a containerized runtime environment — the agent has no service manager to run under and nothing that installs there manages the host, which would explain "init script succeeded, instance never registers." Also: "init script completed successfully" only means exit code 0 — if you installed via snap, snapd likely isn't functional on these nodes and the failure can be silent. If you retry, use the regional S3 .deb package with dpkg instead of snap, and check the agent log above for registration errors (it needs to reach ssm and ssmmessages endpoints).
One step back, though: what's the underlying goal? These nodes are ephemeral — autoscaling creates and destroys them constantly, so SSM Fleet Manager inventory will churn even if you get registration working. If the driver is shell access, the web terminal already covers it. If it's compliance/monitoring requirements, ask your account team about the Enhanced Security and Compliance add-on (hardened images + security monitoring agents) — that's the Databricks-native answer to most of the requirements that usually motivate SSM. And if you genuinely need SSM on managed nodes, this question is worth putting to Databricks support directly, since the public docs are silent on it.
Thomaz A. Rossito Neto
Principal Data & AI — CI&T
thomazn@ciandt.com
linkedin.com/in/thomaz-antonio-rossito-neto
