topic Re: Need a way to show the non-trimmed data while query a table ? in Data Engineering

Need a way to show the non-trimmed data while query a table ?

Rajkishore — Tue, 28 Mar 2023 10:53:20 GMT

When querying a json data , the values are getting trimmed. I need to see the full data for that field, is there any way to do so ?

Re: Need a way to show the non-trimmed data while query a table ?

pvignesh92 — Tue, 28 Mar 2023 16:24:27 GMT

@Raj Sethi Hi, I believe you are using display option. Try to use show with truncate as False so that your entire contents can be displayed

Ex. df.show(truncate = False)

Re: Need a way to show the non-trimmed data while query a table ?

Lakshay — Tue, 28 Mar 2023 18:41:25 GMT

Hi @Raj Sethi , I think if you use "df.show(truncate=False)" , this should display the complete result. But, a better option would be to write the data to a table or file to see the complete value

Re: Need a way to show the non-trimmed data while query a table ?

Anonymous — Thu, 30 Mar 2023 03:35:57 GMT

Hi @Raj Sethi

We haven't heard from you since the last response from @Lakshay Goel and @Vigneshraja Palaniraj , and I was checking back to see if her suggestions helped you.

Or else, If you have any solution, please share it with the community, as it can be helpful to others.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

Re: Need a way to show the non-trimmed data while query a table ?

Rajkishore — Thu, 30 Mar 2023 09:38:20 GMT

Thanks for the response @Vigneshraja Palaniraj ! While executing in python it works, but if I run it as sql ,do I have this option? Please let me know if you have come across any solution for this .

Re: Need a way to show the non-trimmed data while query a table ?

Rajkishore — Thu, 30 Mar 2023 09:38:55 GMT

Thanks for the response @Lakshay Goel ! While executing in python it works, but if I run it as sql ,do I have this option? Please let me know if you have come across any solution for this .

Re: Need a way to show the non-trimmed data while query a table ?

pvignesh92 — Thu, 30 Mar 2023 13:28:04 GMT

Hi @Raj Sethi , If you are expecting something like below

%sql

select * from tbl -> This internally uses .display API to display your results. So you can't really control truncation there.

But if you can try the same above command like below, it could help to solve your issue.

spark.sql("select * from tbl").show(truncate = False)

Please let me know if this helps.