RAG research is shifting the performance lever from bigger dense embeddings and vector databases toward structure, source-awareness, and cost-efficient hybrid retrieval—reframing ‘what/how you index’ as more decisive than model or index scale.
Evidence
- Scaling and infrastructure studies favor lean retrieval: BM25 is most cost-efficient at scale with hybrid Agent+BM25 best for accuracy, ScalableRAG drops embeddings/vector DBs entirely, brute-force numpy embeddings replace vector DBs for small corpora, and B1ade shows strong RAG with small models.
- Structure beats scale: schema-constrained causal graphs cut extraction 8x–135x with ‘what is placed in the graph matters more than how many nodes,’ and hypergraph/co-evolving graph-text memory improve multi-hop reasoning.
- Quality gains come from source and retrieval-interface design: source-aware reranking lifts precision 0.48→0.72, Harness-G shows structured action selection beats free-form queries, and late-interaction models beat dense embeddings on unseen languages.
- Enterprise/agentic RAG requires domain constraints: SCAIR shows generic agentic RAG fails on real knowledge graphs, and AST-based codebase indexing is preferred over text similarity.
Implications
- Teams can cut operational cost and complexity by defaulting to BM25/hybrid and careful schema/graph design before reaching for dense vector infrastructure.
- Reliability work will center on source credibility, structured agent-retrieval interfaces, and domain-constraint encoding rather than embedding model upgrades.
Concepts
RAG Embeddings Vector Database Hybrid Search Reranking Agents
Confidence
medium