Facing Data Truncation Issues in Databricks Dashboards

Akshay_Petkar
Valued Contributor

‎05-31-2024 03:44 AM

I'm encountering data truncation in my Databricks dashboards. I'm working with a large dataset, and the dashboard only displays a limited number of (truncated) rows.
let's take a dataset containing 1 million sales records. The dashboard currently only displays a limited number of rows, which affects the accuracy of important metrics such as total sales. Is there a way to configure the dashboards to handle larger datasets and ensure that they display the complete picture, including all 1 million records in visualizations?

Akshay Petkar
0 Kudos

NandiniN
Databricks Employee
Databricks Employee

‎06-01-2024 04:51 AM

Hi @Akshay_Petkar ,

This is simple in Databricks SQL, just uncheck LIMIT 1000 in the drop down.

https://docs.databricks.com/en/sql/get-started/visualize-data-tutorial.html

Limit 1000 is selected by default for all queries to ensure that the query returns at most 1000 rows. If a query is saved with the Limit 1000 setting, this setting applies to all executions of the query (including within dashboards). If you want to return all rows for this query, you can unselect LIMIT 1000 by clicking the Run (1000) drop-down. If you want to specify a different limit on the number of rows, you can add a LIMIT clause in your query with a value of your choice.

The query result displays in the Results tab.

 

Same way in notebook you can click on download full results.

But it won't preview the entire data on the UI.

0 Kudos