Adding agentic scaffolding—more tools, memory, skills, and retained context—frequently degrades rather than improves performance, reframing optimization around right-sizing and behavior-aware configuration.
Evidence
- Tools, skills, and retained memory can degrade baseline reasoning and raise token cost per correct answer (AlphaDiana), and the same context-engineering technique helps one agent type while hurting another (Same Context, Different Agents)
- Practical limits come from harness budgets—visible tool-menu size and completion caps—rather than model capability (ToolBottleneckBench), and irrelevant context injection selectively breaks state tracking (Decomposing Long-Horizon Failures)
- Self-improving agents overfit to specific skill banks and break when skills evolve (Skill-Policy Co-evolution Trap); compounding per-step reliability makes smaller models sometimes more efficient per completed task (When Is Smaller Greener)
Implications
- Builders should treat scaffolding as adaptive and measured per agent/task (complexity-adaptive tool routing, context-guided model routing) rather than defaulting everything on
- Context hygiene—purging history, forgetting, source-judged compaction—becomes a first-class reliability lever, not just a cost optimization
Concepts
Agents Tool Use Context Engineering LLM Evals Open Source LLMs
Confidence
medium