Adding agentic scaffolding—more tools, memory, skills, and re-injected context—frequently degrades rather than improves performance, reframing optimization around per-completed-task reliability and right-sizing.
Evidence
- AlphaDiana finds adding tools, skills, or retained memory can degrade baseline reasoning and raise token cost per correct answer
- ToolBottleneckBench shows visible tool-menu size and completion caps, not model capability, are the binding constraints
- Context-injection and compaction studies show irrelevant history breaks sequential state tracking and compressors hallucinate derailing steps; the same context-engineering technique helps one agent yet hurts another
- The skill-policy co-evolution trap and ‘When Is Smaller Greener’ show policies overfit to skill banks and that small per-step reliability gaps compound across multi-step tasks
Implications
- Builders should treat scaffolding as a cost with diminishing or negative returns—pruning tools, gating context, and selecting models by per-task efficiency and drift-aware evaluation
- Harness budgets, tool-registry design, and context hygiene deserve as much optimization attention as model choice
Concepts
Agents Tool Use Context Engineering LLM Evals
Confidence
medium