Hi, as Raman says it is probably that the cluster type is not currently available. In order to see which ones are available to update the config you can run the following:

from databricks.sdk import WorkspaceClient
import json

def list_node_types():
    w = WorkspaceClient()
    response = w.clusters.list_node_types()
    return response.node_types

def filter_node_types(min_cores, max_cores, min_memory, max_memory):
    node_types = list_node_types()
    filtered_node_types = [
        node_type for node_type in node_types if 
        node_type.num_cores >= min_cores and 
        node_type.num_cores <= max_cores and
        min_memory <= node_type.memory_mb <= max_memory
    ]
    return filtered_node_types

def node_type_to_dict(node_type):
    return {
        'node_type_id': node_type.node_type_id,
        'num_cores': node_type.num_cores,
        'memory_mb': node_type.memory_mb,
        'description': node_type.description
    }

filtered_node_types = filter_node_types(4, 4, 8192, 16384)
filtered_node_types_dicts = [node_type_to_dict(node_type) for node_type in filtered_node_types]

display(json.dumps(filtered_node_types_dicts))

The default for the SQL server connect gateway on AWS is r5.xlarge. 

You can set the driver type in the pipeline config -

gateway_pipeline_spec = {
   "pipeline_type": "INGESTION_GATEWAY",
   "name": gateway_pipeline_name,
   "gateway_definition": gateway_def.as_dict(),
   "clusters":{
    "driver_node_type_id": "r5.xlarge"
   }
 }

I hope this helps