Variable Compute clusters within a Job

allyallen
New Contributor III

We have 3 possible compute clusters that we can run a notebook against.
They are varying sizes and the one that the notebook uses will depend on the size of the data being processed.

We "t-shirt size" each tenant base on their data size (S, M, L) and can read this config in from Postgres in a notebook.
Once we know the t-shirt size, is there a way of setting the compute cluster dynamically in subsequent tasks? 
e.g. a tenant is size M so the rest of the tasks in the job run on the M cluster

We'd like to avoid duplicating jobs/tasks!
Thanks in advance