Hi @pranitha,
The instance_id
field refers to the specific instance of a node, which can include both driver and worker nodes. In your query, you are counting distinct instance_id
s, which might include instances that are not currently active or are driver nodes, leading to a higher count than the maximum number of worker nodes set in your cluster configuration.
Additionally, the node_timeline
table captures node-level resource utilization data at minute granularity, and each record contains data for a given minute of time per instance. This means that if nodes are frequently added and removed (e.g., due to autoscaling), the count of distinct instance_id
s over a period of time can exceed the maximum number of worker nodes set for the cluster at any single point in time.