Public betaOpen source. Built in the open.Explore the beta
OmniSwitch
QuickstartGet started
Proof before performance claims

Measurements, not marketing numbers.

Evaluate the self-hosted gateway with reproducible workloads. Cross-gateway results remain pending; protocol coverage and operating model are separate questions.

Updated · Public beta

Cross-gateway results are pending

PendingNo published measurement campaign

The cross-gateway harness is available under bench/ on the release-foundation branch. No measurement campaign has been published.

There is no competitor latency table, throughput ranking, or promised run date. Results need a pinned run manifest, reproducible configurations, and raw artifacts.

Published baseline and methodology

Frozen methodology / v1

Methodology frozen 2026-09-11. Review the harness’s canonical protocol before comparing a self-hosted process with a managed network service.

Environment

  • Dedicated vCPU instance, CPU governor performance, nothing else running
  • Load generator and gateway on the same host; mock backend on loopback
  • OS, kernel, Go, Rust and Python versions recorded in the run manifest
  • Each gateway built from a pinned tag; commit SHA recorded
  • Fresh process per repetition; idle period between runs

Workload

  • Mock OpenAI-shaped backend, fixed think time, fixed-size JSON body
  • 60s per run, warm-up discarded, 5 repetitions, median reported with min/max
  • Closed-loop concurrency for the table; open-loop stepping for the ceiling
  • Identical request bodies and headers for every gateway

Fairness controls

  • Default quickstart configs, auth on everywhere, payload logging off everywhere
  • No OmniSwitch tuning; no competitor detuning; per-gateway config diffs published
  • Vendor-published numbers are never mixed into our tables
  • Maintainers of compared projects notified before the run
  • Managed services excluded — a network hop is not comparable to a local binary

What the campaign will measure

  1. Non-streaming overhead. Added latency over direct-to-mock at 64 concurrent; throughput ceiling by rate stepping. Metrics: p50 / p95 / p99 added ms, req/s before p99 doubles.
  2. Streaming TTFT. Fixed-cadence SSE chunks; does the gateway buffer, and what does it add to first byte? Metrics: TTFT delta ms, inter-chunk jitter.
  3. Upstream failure. Primary upstream returns errors mid-run. What reaches the client, how fast does fallback engage, how does recovery look? Metrics: client error rate, time to first fallback, recovery seconds.
  4. Cache hit path. Cost of serving an exact-match cache hit, with cache scope per API key. Metrics: hit latency p99, req/s.
  5. Guardrails on. Built-in PII + injection + 8 regex rules on input and output, 8 KB and 64 KB payloads. Metrics: added ms vs scenario 1.
  6. Memory and CPU. RSS and CPU seconds over a 60s run at 64 concurrent, logging on, payload logging off. Metrics: RSS MB steady, CPU ms per request.

Published guardrail microbenchmarks

Guardrail scanning · 8 rules · single pass vs per-rule regexp

Environment: AMD Ryzen 5 5500U (12 threads), Windows, Go 1.26, wazero 1.9 — laptop-class, not the pinned benchmark rig. These are published component measurements, not cross-gateway results and not measurements from this website session.

Existing BENCHMARKS.md component results
PayloadGo pathRust-WASM pathReported ratio
8 KB~1.48 ms~1.24 ms1.19x
64 KB~12.8 ms~9.3 ms1.38x
256 KB~53.5 ms~37.8 ms1.42x
Published source and environment

A component microbenchmark does not establish end-to-end overhead, time to first token, or production throughput. Re-run on your own hardware.

Reproduce the component benchmarksShell
git clone https://github.com/omniswitch-dev/omniswitch && cd omniswitch
go test ./internal/accel/ -bench "First|Regexp" -benchtime=2s
go test ./internal/guardrail/ -bench . -benchtime=2s

Read the caveats first

  • Microbenchmarks against a mock. Real providers add hundreds to thousands of ms; a millisecond between gateways will not be visible to users. Tail behaviour under failure will.
  • One machine, one cloud, once published. Ratios travel better than absolutes — re-run on your own hardware.
  • Co-located load generator removes network noise but competes for CPU at the ceiling; ceilings will be conservative for everyone.
  • Default configs differ across gateways. We will publish per-gateway config diffs rather than normalizing features away.

What we do not infer

  • A performance ranking before measured, reproducible results exist.
  • Anything about managed services, model quality, cost savings, or cache hit rates on your traffic.
  • Kubernetes-native strengths of control-plane gateways run standalone.
  • Stability across versions — every release re-runs the harness once it exists and posts regressions here.
QuickstartInstall optionsDocumentationRelease notes