Issue in system.compute.node_timeline table

[Abishrp]

For some of my clusters i can able to find data in  system.compute.node_timeline table to view utilization details even after 36 hrs since the run.

[Walter_C]

If you check at the metrics tab of the specific cluster in the UI does it actually shows data?

[Abishrp]

No, there is no data.

[Walter_C]

Ok how many time was the cluster running? Sometimes depending on the time of the run there is no data populated to the metrics tab so probably wont be on the system table.

[Abishrp]

Mostly this issue occurs in Job Compute clusters. It runs 3 times a day. Sometimes, driver details not available and sometimes both worker and driver data not available.

[Walter_C]

Does this job completes really fast, once the cluster is up how many time does it takes to complete the run?

[Abishrp]

In a particular job, I use job compute with 1 worker in a particular task, the particular task completed in 11 minutes.

[Abishrp]

Adding to my previous reply, during the same job run, i also use another job compute with single worker to run another task, this task completes in 6 minutes. For this job compute i get only worker data in system table.

[Abishrp]

[Walter_C · Accepted solution]

Yes seems that the runs are 1 minute of execution so this might be the reason why the metrics are not loaded