I am trying to pre-load a docker image to a Databricks cluster pool instance.
As per this article I used the REST API to create the cluster pool and defined a custom Azure container registry as the source for the docker images.
https://learn.microsoft.com/en-us/azure/databricks/compute/pools#dcs
Pool gets created successfully.
But now whenever I try to create a cluster using that pool, Databricks still asks me to define a docker source in the cluster creation step. Gives me the error
"'docker image' must be provided for cluster created with instance pool xxxxxxxxx"
Does this mean that I still have to define the same docker when I create the cluster. If so, what is the point of defining it in the cluster pool. I want to define the custom docker container at the cluster pool level so users who use the pool to create their clusters don't have to know about the underlying docker image used or the credentials to access the docker source.
Can anyone please help me clear this up.
