Introduction
This is the second part of a three-part series on concurrent load testing of Databricks SQL warehouses with Apache JMeter. In Part 1, we externalized our SQL queries and parameter values and moved connection details and the concurrency schedule into a `run.properties` file.
In this part, we build the JMeter test plan itself, run it once to confirm everything works, and then shape the load to look like a real business day. By the end, you will have a plan you can point at any warehouse to measure how it behaves under realistic concurrent load. Prefer to skip the manual setup? The Summary includes a prompt that scaffolds the whole configuration with an LLM.
Installing the Ultimate Thread Group Plugin
JMeter's built-in Thread Group supports only a single, fixed concurrency level for the duration of a run, which cannot reproduce the ramp-up, peak, and ramp-down pattern of production traffic.
To model these stages, we use the Ultimate Thread Group plugin, which chains multiple concurrency windows within a single run.
The plugin is a one-time installation via the JMeter Plugins Manager. If you don’t already have the Plugins Manager, download `jmeter-plugins-manager.jar` into JMeter's `/lib/ext/` directory and restart JMeter. Then open Options → Plugins Manager → Available Plugins, select Custom Thread Groups, and click Apply Changes and Restart JMeter. Full installation notes are on the Ultimate Thread Group page.
Building the Test Plan
We'll build the plan from an empty test plan, adding each element in the order it appears in the tree. Follow along in JMeter — by the end of this section you'll have a plan you can run.
Open JMeter and save the new plan as `DBSQL_ConcurrentBenchmark.jmx` inside the `/TestPlans/` folder. Figure 1 below shows the finished test plan - the structure you will have once every element in this section has been added. Do not create anything yet; we will add each element one at a time and arrive at this layout by the end of the section.
Figure 1. This is the finished test plan — the target state. It's shown up front as a roadmap; the sections below build up to it one element at a time, so your tree won't match this until the end.
User-Defined Variables
To avoid repeating long expressions in every sampler, we define short, readable variable names once and reference them throughout. Right-click Test Plan → Add → Config Element → User Defined Variables, and add the following:
|
Variable |
Value |
|
v_tp_database_jdbc_driver |
${__P(user.database.jdbc.driver)} |
|
v_tp_folder_path_results |
${__P(user.test.plan.folder)}/testResults |
|
v_tp_test_inputs_path |
${__P(user.test.plan.folder)}/inputs |
|
v_tp_test_limit_resultset |
${__P(user.test.limit.resultset,1000)} |
JDBC Connection Configuration
Right-click Test Plan → Add → Config Element → JDBC Connection Configuration. Name the connection pool `warehouse_default` and configure it as follows:
|
Field |
Value |
|
Variable Name for pool |
warehouse_default |
|
Database URL |
${__P(user.database.url)} |
|
JDBC Driver Class |
${v_tp_database_jdbc_driver} |
|
Username |
${__P(user.database.user)} |
|
Password |
${__P(user.database.password)} |
setUp Thread Group
JMeter guarantees that a setUp Thread Group completes before any other thread group starts, which makes it the correct place to count the available SQL files before the load test begins.
This count allows a single JDBC sampler to cycle across all queries automatically, so that adding or removing a `.sql` file requires no change to the plan.
Right-click Test Plan → Add → Threads (Users) → setUp Thread Group, (a single thread is the default and all we need here). Then add a BeanShell Sampler and name it Count SQL Files. This sampler is the one piece not in the basic plan. It runs a short script once at startup that counts how many .sql files are in your /inputs/ folder and stores that number in the p_sqls_count property. Later, the JDBC Request uses that count to spread queries across every file automatically — so when you add or remove a .sql file, the plan adapts without any edits.
File directory = new File("${v_tp_test_inputs_path}");
files = directory.list();
int fileCount = 0;
for (int i = 0; i < files.length; i++) {
if (files[i].toLowerCase().endsWith(".sql")) {
fileCount++;
}
}
vars.put("v_tmp_fileCount", String.valueOf(fileCount));
props.put("p_sqls_count", vars.get("v_tmp_fileCount"));Note: BeanShell is used here because it runs once in the setUp Thread Group outside the load path, avoiding timing impacts. For workloads under load, prefer a JSR223 Sampler with Groovy for compiled performance.
Ultimate Thread Group
Right-click Test Plan → Add → Threads (Users) → jp@gc - Ultimate Thread Group. This is where you set the load. The schedule is a table, one row per stage, with five columns:
|
Column |
Meaning |
|---|---|
|
Start Threads |
Number of users in this stage |
|
Initial Delay |
How long to wait before this stage begins |
|
Startup Time |
Time spent ramping up to the user count |
|
Hold Load For |
How long to hold at the full user count |
|
Shutdown Time |
Time spent ramping back down |
For now, add a single row so we can run the plan and check it works. This starts a single user and holds them for 120 seconds:
|
Start Threads |
Initial Delay |
Startup Time |
Hold Load For |
Shutdown Time |
|
1 |
0 |
0 |
120 |
0 |
Figure 2. The Ultimate Thread Group with a single stage - 1 user held for 120 seconds - used for the first run.
Next, add a JDBC Request sampler (right-click Ultimate Thread Group → Add → Sampler → JDBC Request). A single sampler serves all SQL files: a modulo expression over the file count selects one file per thread, round-robin. Enter the expressions exactly, as the nested function calls are intentional:
| Name (query label) | sql${__BeanShell(${__threadNum} % ${__P(p_sqls_count)})}.sql |
|
Variable Name of Pool |
warehouse_default |
|
Query Type |
Prepared Select Statement |
|
SQL Query |
${__eval(${__FileToString(${__eval(${v_tp_test_inputs_path}/sql${__BeanShell(${__threadNum} % ${__P(p_sqls_count)})}.sql)},,)})} |
|
Variable Names |
row_count |
|
Limit ResultSet |
${v_tp_test_limit_resultset} |
Because the file is chosen by `${__threadNum} % p_sqls_count`, each thread is pinned to one query file for the whole run rather than being reshuffled per iteration. The modulo spreads threads across the files as evenly as possible automatically — with 50 threads and 2 files, ~25 threads always run sql0 and ~25 always run sql1. If you want some queries to run more often than others, you don't change the thread count; you change how many files map to each query — e.g. duplicate a hot query into two .sql files so twice as many threads run it. `p_sqls_count` counts whatever files are present, so the distribution adjusts on its own.
The Variable Names field captures the query result into the row_count variable. This is what `sample_variables=row_count_#` in `run.properties` picks up to add the row_count column to the CSV results — as mentioned in Part 1.
The SQL Query value reads the appropriate file from disk and evaluates any `${variable}` placeholders against the CSV row.
Add a CSV Data Set Config (right-click Ultimate Thread Group → Add → Config Element → CSV Data Set Config) to load each thread's variable values:
|
Filename |
${v_tp_test_inputs_path}/variables_sql${__BeanShell(${__threadNum} % ${__P(p_sqls_count)})}.csv |
|
Delimiter |
; |
|
Sharing Mode |
Current thread group |
Finally, add a Summary Report listener (right-click Ultimate Thread Group → Add → Listener → Summary Report) and set a timestamped filename, so that each run writes a new file and previous results are preserved:
${v_tp_folder_path_results}/testResults_${__time(MM-dd-yyyy-HH-mm-ss,)}.csvSave the test plan.
Run a Smoke Test First
Before shaping a realistic load, run the plan once with the single user row you just added. This is a smoke test: a short, low-load run whose only job is to confirm the plumbing works — the JDBC connection, the SQL files, the CSV inputs, and result capture — while the plan is still simple and easy to debug.
Run it and check that the Summary Report fills in, CSV results appear under `/testResults/`, and there are no connection or file errors. Once this passes, you know the plan is sound and can move on to shaping the load. If it fails, it is far easier to find the problem here than inside a long, multi-stage run.
Simulating a Business Day
With a working plan, we can now make the load realistic. Real BI workloads don't stay flat — a warehouse is busy in the morning and afternoon and quieter at lunch. Rather than test one steady load, we can follow that pattern in a single run:
Figure 3. Example load profile over a business day. Each step is one spawn() window with its own number of users and duration.
Instead of entering rows manually, bind the group to `run.properties`. In the Ultimate Thread Group, switch the schedule source to "Define Threads schedule with a property" and set it to: `${__P(threads_schedule)}`
The table populates at runtime from threads_schedule in run.properties. Each spawn() call creates a row with the same five ordered values:
spawn(threads, start_delay, ramp_up, hold_time, ramp_down)So the business-day profile above is written as:
spawn(15,0s,0s,3600s,0s) spawn(30,3600s,0s,7200s,0s) spawn(10,10800s,0s,3600s,0s) spawn(50,14400s,0s,10800s,0s) spawn(20,25200s,0s,3600s,0s)|
Stage |
Wall-clock |
Users |
spawn() |
|
Morning ramp |
09:00–10:00 |
15 |
spawn(15,0s,0s,3600s,0s) |
|
Steady morning |
10:00–12:00 |
30 |
spawn(30,3600s,0s,7200s,0s) |
|
Lunch lull |
12:00–13:00 |
10 |
spawn(10,10800s,0s,3600s,0s) |
|
Afternoon peak |
13:00–16:00 |
50 |
spawn(50,14400s,0s,10800s,0s) |
|
End-of-day tail-off |
16:00–17:00 |
20 |
spawn(20,25200s,0s,3600s,0s) |
Each stage's start_delay picks up exactly where the previous stage's hold ends, so the windows chain back-to-back with no gaps — an eight-hour day from 09:00 to 17:00; to try it quickly, scale the durations down (for example, use `120s` in place of each hour).
Keeping the schedule in `run.properties` rather than in the plan itself means you can change the load — more users, longer peaks, extra stages — without ever opening the test plan.
If you only care about the busiest moment, a single high-concurrency window is enough:
threads_schedule=spawn(50,0s,0s,180s,0s)Summary
In this part we built a complete, reusable JMeter test plan. It reads its queries and parameters from files, counts the SQL files at startup so it adapts as you add or remove them, and takes its load from the configuration instead of the plan. We ran a short smoke test to check the setup was working, then changed the schedule to match a normal business day — busier in the morning and afternoon, quieter at lunch — all in one run, without touching the plan.
The result is a benchmark you can reuse: point it at a different warehouse, swap the queries, or change the load pattern by editing configuration alone. That makes it practical to compare warehouse sizes, check whether an SLA holds under realistic traffic, and repeat the same test as your workload grows.
|
💡 Generate your config with an LLM The folder layout is regular enough that an assistant can scaffold it for you. Give it your queries and warehouse details: You are scaffolding a JMeter benchmark for a Databricks SQL warehouse. Inputs I'll provide: my SQL queries and the warehouse connection details. Produce:
You still review the output and run the smoke test — but the scaffolding is done in seconds. |
In Part 3, we run the full benchmark from the command line and analyze the results — measuring latency per query and calculating the exact cost of the run from Databricks system tables.
