Anthropic explains: Why Claude Code performed worse for weeks

Redaktion · · 3 Min. Lesezeit

For six weeks users reported that Claude Code grew forgetful, picked tools oddly, and generally felt “less intelligent.” Anthropic has now published a post-mortem — the complaints were justified, and three unrelated changes had stacked on top of each other.

Three root causes, layered on top of each other

1. Reasoning effort dialed down (March 4 – April 7) Anthropic lowered the default reasoning effort from “high” to “medium” because some users were hitting extreme latencies. Internal tests showed only minor quality regressions, but in real-world use Claude Code felt noticeably dumber. Reverted on April 7.

2. Caching bug (March 26 – April 10) A mechanism was supposed to drop older reasoning segments only after an hour of inactivity. An implementation bug instead deleted them on every turn. The result: Claude continually lost context about its own prior decisions — forgetfulness, repetition, poor tool selection. The constant cache misses also burned through usage limits faster than normal. The bug slipped past code review, unit tests, and internal dogfooding.

3. Overly strict length instruction (April 16 – 20) To rein in Opus 4.7’s verbosity, Anthropic added this line to the system prompt: “Length limits: keep text between tool calls to ≤25 words. Keep final responses to ≤100 words unless the task requires more detail.” On a broader eval suite this cost about 3% in quality.

Fix and compensation

All three issues are fixed in version 2.1.116 of April 20. Anthropic has reset usage limits for every subscriber to compensate for the extra burn caused by the caching bug.

What Anthropic is changing

  • Stricter internal testing before updates, including soak periods and gradual rollouts for changes that affect model behavior.
  • Every system prompt change must now pass a broad, model-specific eval.
  • More staff will run the public build instead of internal versions so regressions surface earlier.
  • A new X account @ClaudeDevs for transparency on changes.

Takeaway

Three innocuous individual changes — a latency safety valve, a caching refactor, an anti-verbosity rule — added up to a multi-week quality dip that no single eval caught. The episode shows how hard regressions in agentic systems are to pin down: reasoning behavior, cache strategy, and output length all interact, and traditional tests don’t see it. For teams using Claude Code daily, the lesson is less the what than the how of the failure chain.

Entdecke mehr