CodSpeed

CodSpeed is a performance testing and profiling platform that plugs into continuous integration pipelines. It runs benchmarks on every pull request, reports performance impact through PR comments and status checks, and can block merges that degrade performance beyond a configured threshold. The product combines benchmarking, profiling and regression checks so that performance changes are visible at review time rather than after release.

The platform supports multiple measurement modes: CPU simulation, which the company states keeps measurement variance below 1% by weighting executed instructions by measured hardware costs and accounting for L1 and last-level cache misses; memory instrumentation, which tracks heap allocation peak usage, count and size using eBPF profiling for Rust and C/C++ with libc, jemalloc and mimalloc; and walltime, which measures real execution time including I/O, system calls and multithreading, recommended on CodSpeed's hosted macro runners. Analysis features include differential flame graphs that are process- and thread-aware, environment diffs that surface toolchain, linked library and CPU/hardware differences between compared runs, and an MCP server exposing tools such as query_flamegraph and compare_runs to AI assistants. An Optimization Agent identifies a root cause, applies a fix, benchmarks the result and opens a pull request for review, while an AI setup Wizard detects or generates benchmarks and creates CI workflow configuration automatically.

CodSpeed provides language integrations for Rust (divan, criterion.rs, bencher), Python (a pytest plugin, pytest-codspeed), Node.js (vitest, tinybench, benchmark.js), Go (the builtin testing package), C/C++ (google-benchmark) and a community Zig integration, plus a CLI with its own codspeed.yml harness for benchmarking arbitrary executables. It works locally and in CI systems including GitHub Actions, GitLab CI and CircleCI.

Business model

CodSpeed offers a self-serve SaaS product that developers connect to their repositories via a GitHub App, alongside a demo-led motion for teams; the website advertises a free entry tier ("Start for Free") and a "Book a Demo" path. The company also operates hosted macro runners tuned for consistent walltime measurement, and distributes open-source CLI and language harnesses under MIT/Apache licenses.

Traction

CodSpeed's open-source page lists 123 tracked repositories and states that hundreds of open-source maintainers use the platform. Individual projects show large benchmark and run volumes, for example astral-sh/ruff with 593 benchmarks and 63K runs, oxc-project/oxc with 324 benchmarks and 58K runs, astral-sh/uv with 328 benchmarks and 34K runs, and BerriAI/litellm with 194 benchmarks and 29K runs. The main CLI repository CodSpeedHQ/codspeed had 367 stars, 31 forks and 900 commits. Public testimonials come from Charlie Marsh (Astral), Samuel Colvin (Pydantic) and Node.js performance contributor Yagiz Nizipli.

Latest developments

Recent changelog entries describe CodSpeed v5, which enables cycle estimation by default and adds opt-in allocation exclusion, and note that results from v5 are not comparable with earlier versions so cross-version comparisons are marked N/A. Other updates include walltime profiling via a samply fork on all operating systems, process and thread selection in flame graphs (also exposed through the MCP server), and environment diffs in run comparisons. CodSpeed also announced SOC 2 Type II compliance following a third-party audit, alongside a Trust Center documenting security practices and subprocessors.