topic Re: Incorrectly truncated long numbers in DS&E workspaces notebooks in Data Engineering

Incorrectly truncated long numbers in DS&E workspaces notebooks

Cosimo_F_ — Thu, 06 Oct 2022 15:13:54 GMT

Hello,

I am getting inconsistent representation of long types.

1661817599972 is the unix timestamp in milliseconds for Monday, August 29, 2022 11:59:59.972 PM GMT

when I execute:

`select 1661817599972 as t`

the result is:

166181759997 (last digit truncated)

The number is correctly cast as a bigint according to `describe select 1661817599972 as t`.

However when I execute:

select timestamp_millis(1661817599972) as t

the result is (correctly) `2022-08-29T23:59:59.972+0000`. So the internal representation of the object seems correct. This appears to happen for bigints only. For instance:

`select 16618175678 as t`

yields:

1661817567

but

`select 1661817567 as t`

also yields

1661817567

DBR 11.2

Thank you,

Cosimo.

PS: displays correctly in DBSQL

Re: Incorrectly truncated long numbers in DS&E workspaces notebooks

Hubert-Dudek — Fri, 14 Oct 2022 12:33:03 GMT

Can you share your code and screenshot? As for me, it worked ok.

Re: Incorrectly truncated long numbers in DS&E workspaces notebooks

Cosimo_F_ — Fri, 14 Oct 2022 12:46:21 GMT

Hi Hubert,

here's the screenshot:

Re: Incorrectly truncated long numbers in DS&E workspaces notebooks

Cosimo_F_ — Fri, 14 Oct 2022 12:49:35 GMT

Executed just now:

Re: Incorrectly truncated long numbers in DS&E workspaces notebooks

Hubert-Dudek — Thu, 20 Oct 2022 12:22:11 GMT

Hi, I tried again on another cluster but still working ok. Maybe try to change the runtime version. Additionally, you can check in Spark UI -> Environment -> Spark properties. Perhaps you have there some options set that change that behavior. I tried to find now in the options something related but couldn't.

Re: Incorrectly truncated long numbers in DS&E workspaces notebooks

Cosimo_F_ — Tue, 25 Oct 2022 23:27:30 GMT

Thank you both for following up, unfortunately I'm still having the issue. Seems like a notebook frontend issue to me...

@Hubert Dudek thanks for the suggestion, I will take a look.

Best,

Cosimo.

Re: Incorrectly truncated long numbers in DS&E workspaces notebooks

Cosimo_F_ — Tue, 25 Oct 2022 23:33:24 GMT

Weirdly, it seems to work if I indicate a long enough column name (or if I specify no column name). This I think supports the hypothesis of a frontend issue.

Re: Incorrectly truncated long numbers in DS&E workspaces notebooks

Anonymous — Mon, 14 Nov 2022 08:58:22 GMT

Hi @Cosimo Felline

Hope all is well!

Does @Hubert Dudek (Customer) response answer your question?

If you could resolve your issue, would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!