Two things I'd love to see quantified here, since they're where the real budgeting risk hides:
1) GPU vs. CPU cost-per-token - the piece notes "GPUs excel at fine-tuning" but doesn't give a cost-per-token comparison against CPU serving for inference at low/medium QPS. In practice the crossover point (where GPU throughput gains outweigh the DBU/hour premium) is workload-specific, and teams often default to GPU too early because the sizing guide skips this tradeoff. And, some model cannot simply be run on CPU.
2) Orchestration overhead - "LangChain adds overhead" is true but unquantified. In our benchmarking, orchestration-layer overhead (retries, tool-call round-trips, intermediate reasoning steps) has been a bigger swing factor in monthly agent spend than the model-serving line item itself, especially once evaluation sampling is layered on top.
And, there are some additional cost components that we need to cater for when building Agents on Databricks.
