HI @vziog
When you run a job on a shared cluster, you're seeing PREMIUM_JOBS_SERVERLESS_COMPUTE_EU_WEST in the usage table because the system is tracking the serverless resources used to manage and coordinate your job, even when the actual compute happens on a shared cluster. This behavior occurs because:
- All Databricks jobs have an orchestration layer that runs on serverless compute
- This orchestration layer exists even when your job executes on a shared all-purpose cluster
- The job orchestration costs are tracked separately from the actual compute costs
For your second scenario, when running on Job Compute clusters, you see both SKUs because:
- PREMIUM_JOBS_COMPUTE represents the actual cluster compute resources
- PREMIUM_JOBS_SERVERLESS_COMPUTE_EU_WEST represents the orchestration layer
The NULL values for job_id and job_run_id in some records likely happen because those costs aren't directly tied to a specific job execution but rather to the overall orchestration system.
This dual-charging model ensures that both the orchestration overhead and the actual compute resources are properly accounted for. The serverless component is typically small compared to the cluster compute costs but is still necessary for job coordination.
To accurately calculate job costs, you should include both SKU types for a complete picture of the resources used.
LR
