cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Query limiting to only 10000 rows

Y2KEngineer
New Contributor

Hi I am query my Azure Databricks table using VB script/Simba Spark ODBC driver.

While querying into the DB(lets say 'Select * from table_1') it is not returning any data. However while querying a limit (lets say 'Select TOP 10000 ID from table_1'), it is giving me the first 10000 rows. The table has 50k rows and I want all them at once due to some limitation at target system

1 REPLY 1

SP_6721
New Contributor III

Hi @Y2KEngineer 

This issue is likely related to row fetching limits or buffer sizes in the driver or system settings. You can try adjusting a couple of things in your ODBC connection string:

  • Set RowsFetchedPerBlock=50000 to make sure it fetches all rows in one go.
  • Add EnableCloudFetch=0 to turn off Cloud Fetch, which sometimes causes problems with large result sets due to network or firewall restrictions.

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local communityโ€”sign up today to get started!

Sign Up Now