@Retired_mod The structure of the `cluster-create.json` is perfectly fine. The issue is as stated above related to the structure is that the SDK does not allow nested structures from the JSON file to be used, and instead they need to be cast to specific Python dataclasses.
Here's what I came up with to get around the situation:
def create_compute_cluster(db_client: WorkspaceClient, cluster_conf: dict) -> str:
cc = CreateCluster.from_dict(cluster_conf)
refactored_input = dict()
for field in list(cc.__dataclass_fields__.keys()):
refactored_input[field] = cc.__getattribute__(field)
return db_client.clusters.create_and_wait(**refactored_input, timeout=CLUSTER_UP_TIMEOUT)
I could also see the function reading the json file more like this:
def create_compute_cluster(db_client: WorkspaceClient, create_config_path: dict) -> str:
with open(create_config_path) as file:
create_config = json.load(file)
cc = CreateCluster.from_dict(create_config)
refactored_input = dict()
for field in list(cc.__dataclass_fields__.keys()):
refactored_input[field] = cc.__getattribute__(field)
return db_client.clusters.create_and_wait(**refactored_input, timeout=CLUSTER_UP_TIMEOUT)
What may make sense is some additional functions in ClustersAPI class unless overloading is preferred using multipledispatch. All this assumes there's a need outside my own to do this type of pattern. 🤷