Warp

Warp launches Factory Benchmarks in early access, letting teams evaluate model and harness configurations on their own coding tasks rather than public benchmarks like SWEBench. Benchmarks defined via factory.yaml replay prior runs across a configuration matrix; Scorers grade runs with an LLM-as-a-judge loop; Pareto-style views map dimensions like cost, quality, correctness, and verbosity. Warp reports about 63% lower internal cost per PR on some task types. ↗

Warp Factories enters closed beta: open infrastructure for cloud software factories — fleets of coding agents automating the SDLC from triage through verification. A foreman orchestrator routes work from tools like Slack and Jira, spawning subagents with chosen models and harnesses spanning Warp's agent, Claude Code, Codex, and open-weight models. Factories are version-controlled code with bring-your-own inference; Warp says about 30% of its own tasks run this way. ↗