cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

QRY Results incorrect but Exported data is OK

AndLuffman
New Contributor II

I ran a query "Select * from fact_Orders".     This presented a lot of garbage,  The correct column headers, but the contents were extremely random, e.g.  blanks in the key column, VAT rates of 12282384234E-45  . 

When I export to CSV , it presents fine in the CSV.

When I just select the Order_id and Vat_rate, they present correctly. When I select * a single row, it presents fine.

There seems to be a 'limit' on the number of columns but this varies , I think depending on data_types, but the column count limit seems to be a power of 2  (I have failed at the 4th and the 8th column, depending on which columns are chosen).

So, the data is being presented incorrectly in the databricks interface.

 

Has anybody else seen this and, more importantly, is there a fix?

2 REPLIES 2

Hi @Retired_mod ,  it does feel like something to do with a limitation somewhere but the confusions with that are 1: It has worked fine for months with much more data (I did limit the table to 500 records after the issue manifested itself, 400,000 in original)  and 2: other people in the team with essentially the same setup are no suffering the same issue. 
We were wondering if there was something subtle in the config that is causing this.

Thanks @Retired_mod , a few things to ponder/research. I am not knowledgeable on how databricks works, I just use it. I was hoping that this had happened to someone else and there would be simple switch to flick in a setting somewhere. 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group