You're absolutely right. The implementation should definitely take restricted network scenarios into consideration, as many enterprise Databricks deployments operate in air-gapped or network-restricted environments.

Commenting out the token counting logic is a pragmatic short-term solution.