Scenario Analyzer
July 15, 2026 · View on GitHub
The Scenario Analyzer runs several configurations of a model side by side and compares their results. Each scenario is the same or a different model with its own inputs and run settings; you run them all together, then compare a response across them statistically.
You will need: Java 21 and a model bundle (this guide uses the DriveThroughPharmacyWithQ model from the KSL Book Examples bundle,
book-examples.jar). New here? Read Common UI & concepts and the simpler Single-Model guide first.
⚠️ Screenshots and comparison figures pending refresh. This guide now uses the DriveThroughPharmacyWithQ model (varying
Pharmacy.numPharmacists). The window images and the specific comparison numbers / box plot below were captured from an earlier M/M/1 example and will be regenerated for DriveThroughPharmacyWithQ; the written steps are correct.
What you'll be able to do
- Add several scenarios bound to a model and give each its own run settings.
- Run all scenarios at once (sequentially or concurrently).
- Read a multiple-comparison report that ranks the scenarios on a response and tells you which is best with statistical confidence.
1. At a glance
You build a list of scenarios on the Scenarios tab, click Simulate, then read the Scenario Reports and Comparison Analyzer tabs.
| Use this app when… | Use a sibling app when… |
|---|---|
| You have a handful of named configurations to compare. | You want one model only → Single-Model app |
| The configurations are chosen by hand. | You want to vary inputs on a grid/design → Experiment app |
| You want a statistical "which is best?" answer. | You want the computer to search for the best inputs → Simopt app |
2. Before you begin
Like the other apps, the Scenario Analyzer loads models from bundles (use the
Bundles menu to load a JAR, or drop one in your <KSLWork>/bundles/ folder). See
Common UI → Models and bundles. Results are written
under your working directory. The Enable database toggle persists each run to
a SQLite database so you can revisit it later (for example, in the Results app);
it is not required for the in-app Comparison Analyzer, which works on the results of
your most recent run, held in memory.
3. A guided tour of the window
- Menu bar — File, Bundles, View. See Common UI.
- Run toolbar — Simulate / Cancel, the Mode toggle (Sequential or Concurrent), the Analysis name, Enable database, and the DB policy.
- Tabs — Scenarios, Scenario Reports, Comparison Analyzer.
- Scenarios table — one row per scenario (Status, Run?, Name, Model, Reps, Overrides) with Add / Edit / Clone / Delete / Move buttons above it.
- Console drawer — the run log. See Common UI.
4. Tutorial — compare three configurations
Step 1 — Add scenarios
On the Scenarios tab, click Add…, pick the DriveThroughPharmacyWithQ model, and name the scenario. Repeat to build a short list. In the screenshot above there are three:
| Name | Model | Reps | Overrides |
|---|---|---|---|
| Baseline | DriveThroughPharmacyWithQ | 30 | (none) |
| Longer horizon | DriveThroughPharmacyWithQ | 30 | run length → 40000 |
| More replications | DriveThroughPharmacyWithQ | 100 | (none) |
Select a row and click Edit… to open a scenario editor where you set run parameters, control overrides (model inputs), and RV overrides. Untick Run? to skip a scenario without deleting it; use Clone to fork a near-duplicate.
Tip. The most useful comparisons vary a model input — for DriveThroughPharmacyWithQ, the number of pharmacists (
Pharmacy.numPharmacists). You set that under Control Overrides in the scenario editor.
Step 2 — Set the analysis name and run
Type an Analysis name (it names the output folder and database), choose Sequential mode, and click Simulate. The console shows progress and each row's Status advances to Completed. (Leave Enable database ticked if you also want the run saved to disk for later analysis in the Results app; it's optional for the comparison below.)
Step 3 — Read the reports
The Scenario Reports tab collects a report per scenario plus a summary. The Comparison Analyzer tab is where the scenarios meet: pick a response and an analysis (box plot, confidence intervals, or multiple comparison with the best (MCB)).
Reading the results
The Comparison Analyzer compares a chosen response across the scenarios. The most
instructive comparison for this model is System Time under one, two, and three
pharmacists (Pharmacy.numPharmacists = 1, 2, 3) — exactly the output the analyzer
produces: each alternative's average with a 95% half-width, a box plot, and an MCB
(multiple comparison with the best) ranking.
Comparison figures pending regeneration. The averages, box plot, and MCB intervals here are being refreshed for DriveThroughPharmacyWithQ (together with the screenshots); the earlier version used the retired M/M/1 example, whose numbers don't carry over. The shape of the result is stable and worth knowing: with one pharmacist the mean System Time is about 1.0 minute (utilization ρ ≈ 0.5); adding a second pharmacist sharply cuts it, and a third adds a smaller further gain.
Which is best? MCB answers this with statistical confidence: the MCB (min) intervals identify the most-staffed configuration as the one minimizing System Time and screen out the rest at 95% confidence. When the best-vs-runner-up interval excludes zero, that improvement is statistically real — not just noise.
5. Reference — every tab explained
Scenarios
The master list. Each row is a scenario; the toolbar above runs them. Add… picks a model; Edit… opens the per-scenario editor (run parameters + control/RV overrides); Clone, Delete, Move Up/Down, and Clear Scenarios manage the list; Run? includes/excludes a row.
Scenario Reports
After a run, a standard report per scenario plus a batch summary, in the formats you enabled. See Common UI → Reports.
Comparison Analyzer
Pick a response, then run a box plot, confidence-interval, or MCB analysis across the scenarios. It works on the in-memory per-replication results of your most recent run — no database required (re-run if you've edited the scenarios since). The same analysis panel appears in the Results app, where it reads from a saved database instead.
6. Common tasks
| Task | How |
|---|---|
| Add / duplicate a scenario | Add… / Clone on the Scenarios tab |
| Skip a scenario without deleting it | Untick its Run? box |
| Run faster on a multi-core machine | Set Mode → Concurrent |
| Change a scenario's inputs | Select it, Edit…, use the override sections |
| Save / reopen the scenario set | File → Save / Open (a .toml document) |
7. Troubleshooting & gotchas
| Symptom | Cause | Fix |
|---|---|---|
| Comparison Analyzer has no data | No run yet this session (or the last run produced no per-replication data). | Simulate the scenarios first. |
| "name must be unique" when adding | Two scenarios share a name. | Rename — names are the row key and the DB key. |
| A scenario shows Skipped | Its Run? box is unticked. | Tick it to include it. |
| DB error about an existing database | A database with that analysis name exists. | Set DB → NEW, or change the Analysis name. |
8. See also
- Common UI & concepts · Single-Model app · Experiment app · Results app
- KSL Book — multiple-comparison procedures.