topic Re: Query limiting to only 10000 rows in Data Engineering

Query limiting to only 10000 rows

Y2KEngineer — Thu, 17 Apr 2025 04:55:46 GMT

Hi I am query my Azure Databricks table using VB script/Simba Spark ODBC driver.

While querying into the DB(lets say 'Select * from table_1') it is not returning any data. However while querying a limit (lets say 'Select TOP 10000 ID from table_1'), it is giving me the first 10000 rows. The table has 50k rows and I want all them at once due to some limitation at target system

Re: Query limiting to only 10000 rows

SP_6721 — Mon, 21 Apr 2025 11:56:27 GMT

Hi @Y2KEngineer

This issue is likely related to row fetching limits or buffer sizes in the driver or system settings. You can try adjusting a couple of things in your ODBC connection string:

Set RowsFetchedPerBlock=50000 to make sure it fetches all rows in one go.
Add EnableCloudFetch=0 to turn off Cloud Fetch, which sometimes causes problems with large result sets due to network or firewall restrictions.