Hi @aranjan99,

To add some detail to the earlier reply: the confusion here comes from the fact that the REST API and the Databricks CLI handle the "limit" parameter differently.

REST API BEHAVIOR

In the REST API (POST or GET to /api/2.1/jobs/runs/list), the "limit" query parameter controls the page size, meaning how many runs are returned in a single response. The default is 25 and the maximum is 25. If there are more runs than the limit, the response includes a "next_page_token" field. If you simply do not follow that token, you get exactly N runs (or fewer if fewer exist). For example:

curl -s -H "Authorization: Bearer $DATABRICKS_TOKEN" \
"https://<your-instance>/api/2.1/jobs/runs/list?job_id=<JOB_ID>&limit=5"

That request returns at most 5 runs in a single response. If you stop there and do not paginate, you have your 5 most recent runs.

DATABRICKS CLI BEHAVIOR

The Databricks CLI auto-paginates by design. When you run:

databricks jobs list-runs --job-id 716703755579168 --limit 10

The CLI treats --limit as the page size for each underlying API call, but then automatically follows every next_page_token until all runs are returned. This is why you see 1000+ results even with --limit 10. The CLI is making multiple API calls behind the scenes (each fetching 10 runs per page) and concatenating all results.

This is a known behavior difference that is being tracked for improvement in the CLI:
https://github.com/databricks/cli/issues/1459

WORKAROUNDS

1. Use the REST API directly if you need precise control over the number of results. A single API call with limit=N and no pagination gives you exactly N runs.

2. If you prefer the CLI, pipe the output through jq to trim results:

databricks jobs list-runs --job-id <JOB_ID> --output json | jq '.runs[:10]'

3. If you use the Databricks Python SDK, the list_runs method also auto-paginates by default, but you can break out of the iterator after N results:

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
runs = []
for run in w.jobs.list_runs(job_id=<JOB_ID>):
runs.append(run)
if len(runs) >= 10:
break

ABOUT THE OFFSET PARAMETER

You mentioned that setting offset > 1 causes limit to be respected. The "offset" parameter on the list runs API is deprecated in favor of token-based pagination (page_token). The interaction between offset and limit can produce unexpected results, so it is best to avoid offset entirely and use page_token-based pagination or the direct single-call approach described above.

DOCUMENTATION REFERENCES

- List Runs API reference: https://docs.databricks.com/api/workspace/jobs/listruns
- List Jobs API reference: https://docs.databricks.com/api/workspace/jobs/list
- Databricks CLI GitHub issue tracking this behavior: https://github.com/databricks/cli/issues/1459

* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.