topic QRY Results incorrect but Exported data is OK in Data Engineering

QRY Results incorrect but Exported data is OK

AndLuffman — Thu, 17 Aug 2023 13:17:40 GMT

I ran a query "Select * from fact_Orders". This presented a lot of garbage, The correct column headers, but the contents were extremely random, e.g. blanks in the key column, VAT rates of 12282384234E-45 .

When I export to CSV , it presents fine in the CSV.

When I just select the Order_id and Vat_rate, they present correctly. When I select * a single row, it presents fine.

There seems to be a 'limit' on the number of columns but this varies , I think depending on data_types, but the column count limit seems to be a power of 2 (I have failed at the 4th and the 8th column, depending on which columns are chosen).

So, the data is being presented incorrectly in the databricks interface.

Has anybody else seen this and, more importantly, is there a fix?

Re: QRY Results incorrect but Exported data is OK

AndLuffman — Thu, 17 Aug 2023 15:07:53 GMT

Hi @Retired_mod , it does feel like something to do with a limitation somewhere but the confusions with that are 1: It has worked fine for months with much more data (I did limit the table to 500 records after the issue manifested itself, 400,000 in original) and 2: other people in the team with essentially the same setup are no suffering the same issue.
We were wondering if there was something subtle in the config that is causing this.

Re: QRY Results incorrect but Exported data is OK

AndLuffman — Fri, 18 Aug 2023 07:35:23 GMT

Thanks @Retired_mod , a few things to ponder/research. I am not knowledgeable on how databricks works, I just use it. I was hoping that this had happened to someone else and there would be simple switch to flick in a setting somewhere.