Skip to content

Load-Test Tool: Scenarios and Reports ​

Updated: 11/10/26, 10:51:10 AM

Built-in Scenarios ​

ScenarioWhat it loads
pingConnectivity and the minimal-overhead baseline
pay-barcodeBarcode payment where the merchant scans the payer code (the most common capacity scenario)
pay-mixedMixed payment methods
refund-flowOrder followed by refund
close-flowOrder followed by close order
gatewayGateway payment pre-order flow
preauthPre-authorization freeze / capture / unfreeze
allocSplit payment
reconcileReconciliation
callbackCallback chain
soakSoak (low TPS, long duration)
ramp-to-breakLadder to peak, finding the breaking point

Scenario syntax is documented in _common.md in the scenario pack directory. Key points:

  • Capacity scenarios must not pass notifyUrl, to keep merchant notifications out of the throughput numbers;
  • Order numbers follow PERF-${runTag}-...; amounts use integer cents;
  • Barcode amounts ending in 1 = immediate success, ending in 3 = processing (useful for close-order drills);
  • The scenario DSL is strict; unknown fields fail outright. After editing, self-check offline with perfctl scenario list|validate.

Data Generation and Volume Ladders ​

bash
./perfctl.exe datagen --dataset datasets/orders-1m.yaml --yes [--dry-run]
./perfctl.exe clean --tag PERF-1M --yes
./perfctl.exe verify --tag PERF-1M --expected 1000000
  • Data generation COPYs business orders, fund orders, refund orders, and transactions in dependency order, with support for status distribution, amount ranges, and back-dated timestamps;
  • Datasets support multiple merchants via refs.merchants[]: rows rotate by order line number, and all records derived from the same source order share one merchant;
  • Cleanup deletes in reverse by tag (refund → transaction → fund order → business order);
  • Volume ladders run with perfctl ladder --experiment experiments/volume-ladder.yaml --yes, generating data and running query benchmarks tier by tier at 100K / 1M / 5M.

Metric Definitions ​

MetricDescription
LatencyClient histogram (0.5ms~60s log buckets), reporting p50 / p90 / p95 / p99 / p999 / max
Dimensionsrun / stage / flow / step / error class; signing time and optional response signature verification are broken out separately
Time seriesSampled every second, one set overall and one per step
Step timelinesteps.jsonl: one line per step with requests per second, error count, p50/p95/p99
Resource samplingactuator on the main app and mock (CPU / GC / Hikari / Tomcat), PG connections and transactions, Redis INFO, the load generator host itself
AttributionTop 2 suspects by resource and latency signature (CPU/RSA, connection pool, Tomcat, Redis, PG, load generator host)

Three boundary concepts:

  • Stable ceiling: the highest TPS that continuously meets the SLA (P99, error rate, backpressure);
  • Breaking point: the first failing tier;
  • Recommended capacity: stable ceiling × 0.7.

Reports and Artifacts ​

bash
./perfctl.exe report --run run-20261025-153000
./perfctl.exe compare --a run-... --b run-...
./perfctl.exe results prune --keep 20

Each run gets one directory; report.html is offline and self-contained, ready to send to the accepting party. The same directory also holds:

FileContent
meta.jsonScenario snapshot, host CPU cores, mock configuration, load-test database baseline
steps.jsonlStep-level per-second metrics
timeseries.jsonlOverall per-second metrics
resources.jsonlResource samples
attribution.jsonAttribution results
verify.jsonResponse signature-verification sampling results

Ladder experiments have their own ladder report (perfctl report --ladder ... or the ladder page in the console).

Consistency Verification ​

Identify the successful request counts for /unipay/pay, /unipay/gateway/pre-pay, and /unipay/refund by step path, and compare them with the order and refund counts in the database; the same applies to combined scenarios. When they do not match, first check the signature-verification sample in verify.json, then check whether mock returned success as expected.

FAQ Quick Reference ​

SymptomResolution
Doctor reports DB-01 database does not existThe load-test database has not been created; create it per Preparing the Load-Test Database or change database.name
Doctor reports DB-02 configured database differs from the actual connectionThe app still points at the old database; restart it against the target database (the tool does not switch databases)
Doctor reports MOCK-03 insufficient capacityRaise the order capacity on the Mock Channel console, or set it to 0
Doctor reports APP-04 log level DEBUGChange to INFO/WARN and restart (the tool does not change it for you)
WARN blocks the runConfirm the risk and tick each item, or use --accept-warn ID
Data generation says the tag already existsUse another tag, or run clean --tag and regenerate; use --append to continue
Query benchmark shows Seq ScanExpected to hit pay_normal_order missing a (mch_no, create_time) index; the report flags it