Before and After Load Testing

Why Before-And-After Testing Works

A single load test can show whether a page or endpoint handled one scenario. A before-and-after test goes further: it shows whether a specific change improved, damaged, or did not affect the user-facing result.

The key is to keep the scenario consistent. If the URL, request method, headers, body, concurrency, request rate, duration, or timeout changes between runs, the reports become harder to compare fairly.

Use The Same Settings

• Same target URL and HTTP method.
• Same headers, authorization pattern, and request body when testing APIs.
• Same total request count, request rate, concurrency, and duration.
• Same timeout so timeout counts are comparable.
• Same environment when possible, especially for staging versus production checks.

What Changes Matter

Start with correctness. More 500 responses, more 429 responses, or more timeouts usually matter more than a small latency difference. After correctness looks stable, compare p95 and p99 latency to understand whether the slower side of user experience improved or regressed.

Throughput also matters. If requests per second drops while latency rises, the system may be queueing work, waiting on a dependency, or hitting a worker or connection limit.

Example Before-And-After Comparison

Imagine the same API route is tested with 50 requests, 5 requests per second, 3 concurrent users, and a 10-second timeout before and after a deploy.

• Before: p95 220ms, p99 360ms, 0 errors, 0 timeouts, 5.0 RPS.
• After: p95 780ms, p99 1,900ms, 3 errors, 2 timeouts, 3.8 RPS.

That result is a regression signal. The endpoint still responds most of the time, but slower tail latency, timeouts, new errors, and lower throughput suggest the deploy should be investigated before more traffic arrives.

Example Decision Rules

• If errors increase, investigate logs and status-code changes before tuning speed.
• If p95 increases sharply, inspect slow queries, upstream calls, caching, and cold starts.
• If p99 gets worse but p50 stays stable, look for rare stalls, retries, locks, or queue delays.
• If throughput drops, check worker limits, connection pools, rate limits, and hosting constraints.

Review a sample report or run a controlled comparison from the load test form.