There doesn't seem to be a proper way to do this currently.

We ended up running the job a couple of times in order to figure out some upper bound for run time, and just using that in the cron. Some jobs now run every 5 minutes during office hours, which is close enough for our usecase.

This does cause issues with Skipped runs when compute is slow to spin up, so make sure you adjust any notifications accordingly.

Alternatively, one could apply a Continuous schedule to the job, then toggle the schedule state for that job to ACTIVE at the start and to PAUSED at the end of the day using the Databricks API. We added two jobs in Databricks that call the API to toggle this state. Do test this thoroughly, or you'll have some costs waiting for you by Monday morning 🙂

This all feels very hacky for functionality that feels like it should be supported by default.