API Benchmark

Overview

API Benchmark executes ordered suites of HTTP requests. Each iteration runs the entire suite sequentially, while multiple suites can overlap up to the configured concurrency limit. PHP and Swoole drive the requests; a local dashboard displays progress and results.

• HTTP(S) endpoints, including GET, POST, PUT, PATCH and DELETE.
• Unmeasured warm-up, repeatable iterations, maximum concurrency and inter-request delays.
• Throughput, average latency, response-time percentiles, completed calls and failure counts.
• JSON result exports and time-series charts.
• Linux x86_64 AppImage distribution or Docker Compose deployment.

Installation and launch

Linux AppImage

Download an AppImage from GitLab Releases. Available assets depend on the published version. PHP and Swoole are bundled, so host-side PHP is not required.

chmod +x api-benchmark-*-x86_64.AppImage
./api-benchmark-*-x86_64.AppImage
# Without FUSE:
APPIMAGE_EXTRACT_AND_RUN=1 ./api-benchmark-*-x86_64.AppImage

The dashboard listens on http://127.0.0.1:8989 and attempts to open your existing browser. A browser is not bundled.

Docker Compose

git clone https://git.small-project.dev/pub/apps/api-benchmark.git
cd api-benchmark
docker compose up --build

Open http://127.0.0.1:8989. From a container, 127.0.0.1 refers to that container, not the host: use a hostname reachable from the container when benchmarking host services.

Development

The repository also provides bin/start for Linux with PHP 8.3+ and the Swoole extension installed. Check the exact release requirements in the repository README.

Using the dashboard

1. Open JSON: choose a scenario file or paste its content into the editor.
2. Run benchmark: execute warm-up suites followed by measured iterations.
3. Stats: review throughput, response times, percentiles, concurrency and failures.
4. Export JSON: save results for reporting and comparison.

Execution files are stored in ~/.local/state/api-benchmark or $XDG_STATE_HOME/api-benchmark.

JSON scenario reference

Authenticated Actor workflow example

This scenario uses response extraction from the response-variables development branch. Older releases may not support these advanced fields.

{
  "baseUrl": "http://127.0.0.1:8000",
  "requests": [
    {
      "type": "POST",
      "route": "/api/actors",
      "headers": {
        "Authorization": "Bearer {{env:BENCHMARK_JWT}}"
      },
      "body": "{\"firstname\":\"Benchmark\",\"lastname\":\"Actor\"}",
      "expectedStatus": 201,
      "extract": {
        "actorId": "$.id"
      }
    },
    {
      "type": "GET",
      "route": "/api/actors?page=1&perPage=10",
      "headers": {
        "Authorization": "Bearer {{env:BENCHMARK_JWT}}"
      },
      "expectedStatus": 200
    },
    {
      "type": "GET",
      "route": "/api/actors/{{actorId}}",
      "headers": {
        "Authorization": "Bearer {{env:BENCHMARK_JWT}}"
      },
      "expectedStatus": 200
    },
    {
      "type": "DELETE",
      "route": "/api/actors/{{actorId}}",
      "headers": {
        "Authorization": "Bearer {{env:BENCHMARK_JWT}}"
      },
      "expectedStatus": 200
    }
  ],
  "warming": 4,
  "iterations": 1000,
  "maxConcurency": 10,
  "minDelay": 10
}

{{env:BENCHMARK_JWT}} loads a JWT from the benchmark process environment; {{actorId}} substitutes the identifier extracted from the POST response. Never commit a real token.

Concurrency applies to complete suites, not individual requests inside a suite. Eight calls repeated over 1,000 iterations yield 8,000 measured calls, excluding warm-up.

Understanding the metrics

Throughput is the number of completed requests per second. Average latency is the arithmetic mean response time. p50, p95 and p99 indicate latency thresholds containing 50%, 95% and 99% of observations. Failures include transport errors or statuses rejected by the validation rules supported by the selected version.

Time-series charts show periodic observations rather than a per-request histogram. Their axes are normalized separately: chart heights are not directly comparable.

Actor / MOB2E benchmark example

The following numbers come from a user-run Symfony/Swoole/MySQL API workload after adding indexes and paginating the actor list to 10 rows. They describe the target API and test environment, not the intrinsic performance of API Benchmark.

This setup achieves its lowest latency at concurrency 10. Raising concurrency from 100 to 500 adds almost no throughput but greatly increases latency. For comparisons, keep hardware, data volume, load pattern and configuration constant.

Quality, tests and CI/CD

The project documents max-level PHPStan checks, Pest tests for JSON validation, and functional runner tests against an actual HTTP server. CI builds Linux AppImages; SemVer tags publish GitLab Releases and attach the packaged application.

composer install
composer analyse
composer test:unit
composer test:functional

See GitLab Releases for authoritative versions and available downloads.

Security, limitations and safety

• Only test endpoints you own or are explicitly authorized to load-test.
• Use test credentials and disposable test data, not production secrets.
• POST, PUT and DELETE calls during warm-up and measurement have real side effects.
• Start with modest concurrency; monitor CPU, RAM, MySQL connections and error rates.
• The local interface can access internal HTTP resources. Do not expose it to untrusted networks.
• Compatibility with older Linux distributions is not guaranteed for bundled AppImages.

Troubleshooting

Immediate failures and transport status 0

Confirm baseUrl, the target port and network access from the benchmark process. Status 0 is not an HTTP response from the server.

HTTP 401/403

Check JWT validity, expiry and header propagation. Environment variables must exist in the benchmark worker process, not just in a different terminal.

AppImage startup

If FUSE is missing, use APPIMAGE_EXTRACT_AND_RUN=1. Consult <ID>.log in the application state directory for failed runs.

High latency

Start with low concurrency. Test each route separately, then inspect pagination, SQL indexing, CPU saturation and worker count of the target API.

Repository, releases and license

Open-source project by Sébastien Kus. Published releases and download assets may change independently from this guide.