topic Re: Cant find reason but suddenly new Jobs are getting huge job id numbers. example 945270539673815 in Data Engineering

Cant find reason but suddenly new Jobs are getting huge job id numbers. example 945270539673815

JBear — Thu, 24 Feb 2022 09:38:15 GMT

Created Job ID is suddenly started to make huge numbers, and that is now making problems in Terraform plan, cause int is too big

Error: strconv.ParseInt: parsing "945270539673815": value out of range

Im new on the board and pretty new with Databricks so bare with me, if more information is needed please ask ill try to answer with my best know how.

Databricks is running on Azure.

Re: Cant find reason but suddenly new Jobs are getting huge job id numbers. example 945270539673815

JBear — Fri, 25 Feb 2022 05:41:53 GMT

Im using Terraform 1.0.10 i will try next to update providers and terraform and see if it helps

Re: Cant find reason but suddenly new Jobs are getting huge job id numbers. example 945270539673815

JBear — Fri, 25 Feb 2022 06:11:44 GMT

Got it working again i thing problem was old version of providers and terraform Now running on following version and everything is ok now

Terraform v1.1.6

on windows_amd64

+ provider registry.terraform.io/databrickslabs/databricks v0.5.0

+ provider registry.terraform.io/hashicorp/azuread v1.6.0

+ provider registry.terraform.io/hashicorp/azurerm v2.98.0

+ provider registry.terraform.io/hashicorp/random v3.1.0

Re: Cant find reason but suddenly new Jobs are getting huge job id numbers. example 945270539673815

BilalAslamDbrx — Tue, 01 Mar 2022 12:36:30 GMT

For posterity -- you are seeing large Job IDs because of an update to our internal database. We did this because our jobs service is scaling out massively, and the old ID scheme wasn't going to work for much longer.

Re: Cant find reason but suddenly new Jobs are getting huge job id numbers. example 945270539673815

Anonymous — Sun, 03 Apr 2022 19:25:16 GMT

Hi @Jere Karhu , In case you are using the Job/Run id in API, please be advised that you will need to change the client-side logic to process int64/long and expect a random number. In some cases, you just need to change the declared type in their source code to long and do not count on ids being sequential or sorted.

Previously the job/run id was sequential and could be sorted.

Current sample job URL - https://sample.cloud.databricks.com/?o=3329755207123451#job/281680268617720/run/6615691

(here the job/run id has improved to be a bigger/longer number)

Previous sample job URL - https://sample.cloud.databricks.com/?o=3329755207123451#job/5/run/6615597

It is mitigated in version 0.5.3. The below link points to the main Terraform Registry page suggests you use the latest version of the terraform provider.

https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs