Anthropic successfully migrated its Messages API to Rust, utilizing Claude to accelerate implementation while engineers maintained control over architecture and verification. This transition yielded a 40 to 50 percent reduction in service layer CPU usage and a 10 to 20 percent improvement in time to first token. The project relied on shadow traffic to validate parity between the legacy Python service and the new Rust implementation before gradual deployment. This approach demonstrates that AI-assisted coding is most effective when paired with rigorous, human-led testing protocols to ensure system stability and performance.
Anthropic’s head of engineering, Katelyn Lesse, says a Claude-assisted rewrite of the Messages API in Rust reduced CPU use by roughly 40–50% and improved time to first token by 10–20%. She described the work in an October 27, 2026, Latent Space interview with Anthropic product leader Angela Jiang.
Claude wrote much of the implementation, according to Lesse’s account, while engineers designed the system and checked its behavior using shadow traffic. A secondary reconstruction by explainx reports that the Rust version had completed its production rollout by the interview.
The useful lesson is the division of responsibility. Anthropic describes handing substantial implementation work to an AI coding system while keeping architecture, compatibility checks and deployment decisions with engineers. That is a more transferable pattern than the headline percentage alone.
What the Performance Figures Actually Mean
Lesse’s reported gains concern the Messages API service layer. The interview segment indexed by Real Alpha identifies both the CPU reduction and the improvement in time to first token.
These are estimates from an Anthropic executive, not independently validated benchmark results. The available sources provide no published benchmark methodology, workload description, latency percentiles or comparable service configurations. That limits how precisely another team can interpret the percentages.
Reported metric
Lesse’s estimate
Practical meaning
Messages API CPU use
About 40–50% lower
Less CPU consumption in the API service layer
Time to first token
About 10–20% better
A shorter wait before the response starts arriving
Model generation speed
No separate result reported
The account supplies no measurement of subsequent token generation
The distinction between API efficiency and model inference matters. An API service handles work around the model, including request processing and response delivery. Reducing that overhead can improve the customer’s wait for the first token without changing how quickly the model generates tokens afterward.
For illustration, a 10–20% reduction from a one-second baseline would mean a first-token wait of roughly 800–900 milliseconds. That is arithmetic, not an Anthropic measurement. The actual effect depends on the starting latency and workload.
Likewise, if the same workload previously consumed 100 units of CPU capacity, a 40–50% reduction would leave 50–60 units. That could materially ease a capacity constraint, but it cannot be translated directly into an equivalent reduction in Anthropic’s overall serving bill. The reported metric covers the API layer, not the whole inference stack.
I’d treat this as a promising service-efficiency result, with the cause still only partly explained. A rewrite can change data structures, concurrency and accumulated implementation choices alongside the language. Without profiling, the published account cannot tell us how much of the gain came specifically from Rust.
The Migration Playbook: Plan, Port, Shadow, Cut Over
The most detailed migration description comes from explainx’s reconstruction, which draws partly on translated reporting of Lesse’s remarks. Its procedural details are therefore secondary reporting, useful for understanding the approach but less direct than a published Anthropic engineering write-up.
Humans defined the architecture and migration plan. Claude handled much of the coding, but engineers decided what the replacement needed to do and how to verify it.
A dedicated team built a complete Rust replacement. The project was treated as a port of the existing service, with the temptation to add features deliberately constrained.
The old and new implementations stayed aligned. Other teams continued adding features to the Python service. Static analysis helped ensure that a Python change prompted consideration of the corresponding Rust work.
Production requests were mirrored to the replacement. Shadow traffic let the team compare the implementations before putting the new service in charge of customer responses.
Traffic moved over gradually. The team increased the Rust service’s production role as the comparison supported moving ahead.
These steps address different failure modes. Architecture planning controls scope. Feature tracking limits drift between implementations. Shadow traffic tests behavior against real request patterns. Gradual cutover limits how much traffic depends on the replacement at once.
The static-analysis detail is particularly important. A long-running port has two moving targets: the replacement under construction and the original system still receiving changes. A replacement that matched last month’s API can still be wrong today.
For another team, the transferable requirement is explicit change tracking. Anthropic’s reported mechanism used static analysis; a smaller service might use migration labels, required review fields or shared compatibility tests. The tool can differ, but each change to the original needs an accountable decision about the replacement.
Why Shadow Traffic Is the Crucial Detail
Shadow traffic exposes a replacement to copies of real requests while the established service continues handling the live customer response. It gives engineers a way to investigate differences before those differences become customer-facing behavior.
That is especially valuable for an API rewrite. Compatibility includes more than accepting a valid request and returning a plausible answer. Clients can depend on error behavior, streaming structure and edge-case handling that are easy to overlook in a clean implementation.
The reported Anthropic approach compared the Python and Rust implementations for consistency. The published summary does not describe the comparison machinery, coverage thresholds or treatment of side effects.
For a team adapting the method, three questions deserve explicit answers:
What must match? Define which fields, status codes, events and ordering guarantees are part of the compatibility contract.
What can legitimately vary? Timing and nondeterministic model output may require different checks from deterministic request validation.
What must never happen twice? Mirrored traffic needs controls wherever a second execution could create a customer-visible action, duplicate accounting or another side effect.
Those are implementation recommendations, not claims about Anthropic’s internal setup.
For a model API, comparing generated text byte for byte may be a poor parity test when output can vary. Teams need to isolate service behavior from permissible model variation. A request-validation mismatch and a different generated sentence are different kinds of evidence.
My view is that this validation boundary is the strongest part of the story. AI can help produce a convincing replacement quickly; confidence in a migration still depends on defining observable compatibility and investigating exceptions.
Shadow traffic is also one layer of evidence. It exercises the traffic received during the comparison period. Rare failures, resource exhaustion and recovery behavior still require deliberate tests.
Claude Wrote Code; Engineers Kept Responsibility
Lesse’s account separates implementation from engineering ownership. The interview summary describes Claude doing much of the coding while humans retained system-design responsibility and verified the result.
That distinction has practical consequences. A coding model can propose an implementation, but someone still needs to decide:
Which existing behaviors must remain compatible.
Which architectural boundaries the replacement must preserve.
What evidence is sufficient to move production traffic.
Who investigates mismatches and stops a rollout.
The account also needs care around Rust experience. According to explainx’s reconstruction, the team started without Rust specialists, learned during the project and later hired engineers with Rust experience.
Calling this a project completed with “zero Rust developers” would erase that progression. The supported lesson is narrower: AI assistance may help a team begin work in an unfamiliar language, while learning and experienced hires still matter.
I’d be cautious about using this example to justify skipping language expertise. Generated code still needs maintainers who can diagnose failures, understand performance tradeoffs and evaluate library choices. Anthropic’s reported hiring is part of the implementation story, not a footnote to it.
What Infrastructure Teams Can Borrow
A port with a clear compatibility target is easier to evaluate than a rewrite that simultaneously changes product behavior. Holding back new features makes mismatches more informative: engineers can investigate whether the replacement preserved the existing contract without also debating whether a deliberate redesign explains the difference.
That creates a useful decision test for an AI-assisted migration. Before generating large amounts of replacement code, a team should be able to describe the current contract, track ongoing changes and observe the new implementation safely.
If those foundations are missing, faster coding can simply produce a larger unverified system sooner.
Anthropic’s account supports a practical approach: assign engineers ownership of architecture and acceptance criteria, use Claude for substantial implementation work, compare against the running service, and expand production exposure in controlled steps. Teams can adopt those controls without assuming they will reproduce Anthropic’s reported savings.
Frequently Asked Questions
4 questions
1
Did Claude rewrite the API on its own?
No. Lesse says Claude wrote much of the implementation, while engineers handled architecture, planning and verification.
2
Does the CPU reduction mean Claude generates tokens faster?
The reported CPU saving concerns the API service layer. Lesse separately reported better time to first token; the account provides no measurement of subsequent token-generation speed.
3
Did the team have Rust expertise?
The team reportedly started without Rust specialists, then learned during the port and hired engineers with Rust experience.
4
How was the replacement checked?
The account describes shadowing production traffic to compare the Python and Rust implementations, followed by a gradual production cutover. It does not disclose the detailed comparison rules or acceptance thresholds.
