โ11-15-2022 09:54 PM
I have to query a data for showing in my dashboard.
But it truncated the results and showing only first 1,000 rows.
In the dashboard view, there is no option to re-execute with maximum result limits.
I don't want to switch back to standard view and click to re-execute everytime.
So, I would like to know if I can change default from showing only 1,000 records to return all records from query.
More information
Workspace: Data Science & Engineering
Language: Python (PySpark)
Databricks Runtime: 9.1 LTS (includes Apache Spark 3.1.2, Scala 2.12)
โ11-22-2022 07:47 AM
@Thanapat Sontayasaraโ 1000 is still the default. Default of 10k rows will be made available by Jan.
โ11-16-2022 04:13 AM
Hi @Thanapat Sontayasaraโโ, Limit 1000 is selected by default for all queries to ensure that the query returns at most 1000 rows. If you want to return all rows for this query, you can unselect LIMIT 1000 by clicking the Run (1000) drop-down. If you want to specify a different limit on the number of rows, you can add a LIMIT clause in your query with a value of your choice.
Source - Read this document
โ11-19-2022 01:10 AM
โ11-19-2022 02:02 AM
Hi @Thanapat Sontayasaraโ, Here is the thread similar to your problem with the best answer. Please LMK if that helps.
โ11-19-2022 02:18 AM
Hi! @Kaniz Fatmaโ.
Yes, definitely the same question as mine.
But, still, I don't find any guideline of how to change the default like mention in that thread.
Do you have any clue about "the changes need to occur at the shard level."?
I have no idea what is the shard level. Or you have any reference document for me to work on.
โ11-16-2022 11:38 AM
@Thanapat Sontayasaraโ 10000 limit is available as an option in the notebook(which propagates to dashboard after its run in notebook) while 1000 rows still is the default. 10000 limit is experimental and it can be made as default depending on the number of use cases from customers which may take some time.
โ11-19-2022 01:13 AM
Hi! @Srihasa Reddy Akepatiโ Sorry for late reply.
Interesting! Can you share with us about how to change or set the 10,000 as default when running command like display(df) and return 10,000.
With screenshot or document guideline shared would be really appreciated!
โ11-21-2022 09:19 PM
@Thanapat Sontayasaraโ Default is still 1000. Let me check internally if there is a way to change the default.
โ11-22-2022 07:47 AM
@Thanapat Sontayasaraโ 1000 is still the default. Default of 10k rows will be made available by Jan.
โ11-22-2022 08:03 AM
Hi @Srihasa Reddy Akepatiโ
That's a useful information and also a good news.
I'm looking forward to that feature release.
Thank you so much for your support. ๐
โ11-18-2022 02:28 AM
Hi @Thanapat Sontayasaraโ, We havenโt heard from you on the last response from @Srihasa Reddy Akepatiโ and me, and I was checking back to see if our suggestions helped you.
Or else, If you have any solution, please do share that with the community as it can be helpful to others.
Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.
โ11-19-2022 01:15 AM
Hi! @Kaniz Fatmaโ Sorry for late reply.
I am still don't find any solution to change the default query limit when using display(df).
If I got the solution, I will share with the community ASAP.
Thank you for your support ๐