Daily Brief — Jun 29, 2026: LLM Evals

🛰 AI Brief — Jun 29, 2026

How to read prio and sources

prio N is the radar’s practical-relevance score for this item (higher runs first; items at or below the noise threshold are filtered out as noise). Under each signal: Concepts / Entities are graph links; Source / N sources list every outbound link for that story.

🥇 Lore adds deterministic repo knowledge for coding agents via MCP · prio 13

Teams building agent workflows around repo-local knowledge: it shows a concrete MCP-based pattern for giving agents authoritative decisions without relying on fuzzy retrieval. For builders who work with Claude Code or Cursor, the main takeaway is the emphasis on typed, validated Markdown as a deterministic source of truth that can be enforced in CI. Concepts: MCP Code Agents Tool Use Context Engineering Entities: Claude Code Cursor Claude Desktop 24 sources: github.com, arxiv.org, habr.com, habr.com, arxiv.org, habr.com, qbitai.com, qbitai.com, importai.substack.com, github.com, arxiv.org, arxiv.org, habr.com, habr.com, quesma.com, arxiv.org, arxiv.org, habr.com, simonwillison.net, arxiv.org, arxiv.org, qbitai.com, vllm.ai, huggingface.co

🥈 Supersede: Training the memory-update gap in LLM agents · prio 12

For builders working on agents that carry state across sessions, this is a direct warning that memory maintenance can fail even when the underlying model is strong and full-context performance is high. It also matters because the paper turns temporal fact-currency into a trainable objective, which is a concrete direction for anyone evaluating or training agent memory systems. Concepts: Agent Memory LLM Evals Entities: GPT-5.4 Qwen2.5-3B 2 sources: arxiv.org, arxiv.org

🥉 Speculative Refinement shows where standard benchmarks can misread hybrid decoding · prio 12

For builders who rely on benchmarks to compare generation systems, this paper is a reminder that evaluation setup can change the apparent winner. Its main value is in the diagnostic failures it surfaces for code tasks and multi-stage generation, which directly affect how you should interpret scores from hybrid or non-autoregressive systems. Concepts: LLM Evals Source: arxiv.org

4️⃣ Selective quitting improves LLM agent safety in ToolEmu evaluation · prio 11

For builders of agentic systems, this is a concrete safety result: a simple quit instruction appears to reduce risky behavior in multi-turn tool-using agents without materially hurting helpfulness. It is also a useful evaluation signal for teams working on agent behavior, because the paper tests the idea across 12 models rather than treating it as a one-off anecdote. Concepts: Agents Tool Use LLM Evals Source: arxiv.org

5️⃣ CalBrief benchmarks evidence-calibrated scientific briefing in LLMs · prio 11

For builders of LLM assistants and eval harnesses, this is a concrete example of how to test not just answer quality but whether a model calibrates claims to evidence. The paper also argues that strength judgment and auditable evidence organization should be evaluated separately, which is directly useful for designing better evaluation rubrics. Concepts: LLM Evals Entities: GPT-4o Claude Sonnet Gemini Flash Source: arxiv.org

Knowledge Gaps

Topics the AI stream keeps raising that the knowledge base hasn’t sufficiently covered yet — candidates for what to learn next. Embeddings · RAG · Context Engineering · Agent Memory

🚀 Models & Releases (2)

• prio 6 DeepSeek V4 is slated for mid-July with peak-hour API pricing Entities: DeepSeek ME News BlockBeats KuCoinFlash Source: kucoin.com
• prio 6 Sber releases KVAE-Audio, a 48 kHz audio tokenizer with open code and weights Entities: Sber MMAudio Meta Stable Audio Source: habr.com

🧪 Research Papers (77)

• prio 10 ProMSA proposes a progressive multimodal search agent for KB-VQA Concepts: Agents Tool Use RAG Source: arxiv.org
• prio 10 SIGA wraps coding agents with a lightweight contract for simulator setup Concepts: Agents Code Agents Tool Use Context Engineering RAG Source: arxiv.org
• prio 10 Low-Agreeableness Persona Conditioning for Safer LLM Fine-Tuning Concepts: LLM Evals Source: arxiv.org
• prio 10 Paper argues prompt injection cannot be perfectly prevented inside shared-embedding models Concepts: Agents Tool Use Source: arxiv.org
• prio 10 SHIFT proposes gate-modulated activation steering for RAG knowledge conflicts Concepts: RAG Source: arxiv.org
• prio 10 EXPLORE-Bench tests long-horizon egocentric scene prediction for MLLMs Concepts: LLM Evals Source: arxiv.org
• prio 10 DiscoBench evaluates whether search agents should ask clarifying questions Concepts: Agents Tool Use LLM Evals RAG Source: arxiv.org
• prio 10 Dataset subset selection for preserving benchmark rankings Concepts: LLM Evals Source: arxiv.org
• prio 10 Paper proposes preregistering future LLMs to reduce p-hacking in LLM-based research Concepts: LLM Evals Source: arxiv.org
• prio 10 alphaXiv on agent-native memory systems Concepts: Agent Memory Agents Context Engineering RAG Entities: alphaXiv 2 sources: twitter.com, alphaxiv.org
• prio 10 BINEVAL breaks evaluation into atomic yes/no checks Concepts: LLM Evals Source: twitter.com
• prio 9 DysLexLens proposes a low-resource LLM framework for analyzing dyslexic learners in online forums Concepts: RAG Evaluation Entities: arXiv Reddit GitHub Source: arxiv.org
• prio 9 SHARD: a privacy-preserving transform for dense retrieval embeddings Concepts: Embeddings RAG Source: arxiv.org
• prio 9 Learning to Evict from Key-Value Cache uses RL for KV cache management Concepts: Long Context LLM Evals Source: arxiv.org
• prio 9 Hybrid fact-checking paper combines knowledge graphs, LLMs, and web search fallback Concepts: Agents Tool Use RAG Source: arxiv.org
• prio 9 Aloe-Vision releases open medical vision-language models, data, and a cleaner evaluation benchmark Concepts: LLM Evals Entities: Aloe-Vision Source: arxiv.org
• prio 9 NLL-Guided Layer Selection for Sliding-Window Long-Context Adaptation Concepts: Long Context LLM Evals Entities: Qwen3-4B Source: arxiv.org
• prio 9 Ko-WideSearch benchmarks Korean web agents on exhaustive breadth search Concepts: Agents LLM Evals Source: arxiv.org
• prio 9 Multimodal KB-VQA shows a primacy bias in retrieved context order Concepts: RAG RAG Evaluation Reranking Source: arxiv.org
• prio 9 Paper evaluates coding LLMs on execution resources, not just test results Concepts: LLM Evals Source: arxiv.org
• prio 9 A Factorized Study of Probe-Based Uncertainty Estimation in LLMs Concepts: LLM Evals Source: arxiv.org
• prio 9 Test-input generation for tensor programs: which strategy actually finds kernel bugs Concepts: LLM Evals Source: arxiv.org
• prio 8 Dialogue-to-detection pipeline for insurance fraud at FNOL Concepts: RAG Embeddings LLM Evals Source: arxiv.org
• prio 8 Multimodal self-evaluation loops can amplify evaluator preference collapse Concepts: LLM Evals Agents Entities: GPT-4o DeepSeek-Chat Source: arxiv.org
• prio 8 Adaptive Turn-Taking for Real-time Multi-Party Voice Agents Concepts: Agents Entities: ModeratorLM Source: arxiv.org
• prio 8 Benchmarking first-stage recall for large-scale code-to-code retrieval Concepts: Codebase Indexing RAG Source: arxiv.org
• prio 8 End-to-End Dynamic Sparsity for Resource-Adaptive LLM Inference Entities: LLaMA-3-8B Qwen 3-4B Source: arxiv.org
• prio 8 LLawCo proposes cooperation laws for embodied multi-agent behavior Concepts: Agents LLM Evals Source: arxiv.org
• prio 8 Retaining by Doing: RL Shows Less Forgetting Than SFT in Post-Training Entities: arXiv LLaMA Qwen Source: arxiv.org
• prio 8 Paper finds position-bias correction is not enough for one-pass attention sorting Concepts: Long Context Context Engineering Entities: LLaMA-2-7B-32K-Instruct YaRN-Llama-2-7b-64k Source: arxiv.org
• prio 8 Empirical study of factual errors in human-written text and factual error detection Concepts: LLM Evals Entities: GPT-5.4 Source: arxiv.org
• prio 8 SpaceDG benchmarks spatial reasoning under visual degradation Concepts: LLM Evals Source: arxiv.org
• prio 8 Cascaded framework for cost-aware LLM serving Source: arxiv.org
• prio 8 Mitigating Position Bias in Transformers with Layer-Specific Positional Embedding Scaling Concepts: Long Context Context Engineering Source: arxiv.org
• prio 7 SpatialUAV benchmark targets low-altitude UAV spatial intelligence Concepts: LLM Evals Source: arxiv.org
• prio 7 Delayed Verification Can Destabilize Multi-Agent LLM Belief Concepts: Agents LLM Evals Source: arxiv.org
• prio 7 Smooth MMD alignment for more accurate numeric prediction in LLMs Concepts: LLM Evals Source: arxiv.org
• prio 7 RateQuant calibrates mixed-precision KV cache allocation with rate-distortion theory Entities: Qwen3-8B Source: arxiv.org
• prio 7 USAD: Uncertainty-aware Statistical Adversarial Detection Source: arxiv.org
• prio 7 EntMTP proposes entropy-guided scheduling for multi-token prediction inference Source: arxiv.org
• prio 7 CBD proposes API-only black-box LLM unlearning with controlled behavioral divergence Source: arxiv.org
• prio 7 Personality composition changes multi-agent LLM outcomes only in some task types Concepts: Agents Source: arxiv.org
• prio 7 COOPA: A Modular LLM Agent Architecture for Operations Research Problems Concepts: Agents Tool Use LLM Evals Source: arxiv.org
• prio 7 Grounded Iterative Language Planning uses a small world-model backbone to reduce hallucinated state changes in LLM agents Concepts: Agents Entities: arXiv gpt-4o-mini Source: arxiv.org
• prio 7 Paper argues self-evaluation is not consistently easier than generation in in-context QA Concepts: LLM Evals Source: arxiv.org
• prio 7 Continual Memorization of Factoids in Language Models Source: arxiv.org
• prio 7 Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning Concepts: Agents Source: arxiv.org
• prio 7 Paper evaluates TSFM embeddings on E-Nose data Concepts: Embeddings Entities: arXiv Chronos-2 MOMENT Source: arxiv.org
• prio 7 Triadic Werewolf adds a third faction to probe theory of mind in LLMs Concepts: Agents LLM Evals Entities: GPT-4.1 DeepSeek-V3.1 Llama-3.3-70b Source: arxiv.org
• prio 7 SidConArena benchmarks LLM agents in open-ended bargaining Concepts: Agents LLM Evals Source: arxiv.org
• prio 6 SingGuard introduces a policy-adaptive multimodal guardrail with dynamic reasoning Concepts: LLM Evals Source: arxiv.org
• prio 6 Quantifying the Domain Gap in Cross-Sensor Diffusion Super-Resolution Source: arxiv.org
• prio 6 Masked Language Flow Models Concepts: LLM Evals Source: arxiv.org
• prio 6 Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining Concepts: LLM Evals Entities: arXiv Source: arxiv.org
• prio 6 Solver-driven geometry problem solving with verified theorem proposals Concepts: Agents Tool Use Entities: QwenVL3-2B Source: arxiv.org
• prio 6 Textual Belief States for World Models Under Strict Mediation Source: arxiv.org
• prio 6 KG2Cypher proposes a data-centric pipeline for enterprise text-to-Cypher systems Source: arxiv.org
• prio 6 Democratic ICAI uses persona debate to derive steering principles from preference data Concepts: LLM Evals Source: arxiv.org
• prio 6 Position paper argues that “machine unlearning” is being used too broadly in LLM research Concepts: LLM Evals Entities: arXiv Source: arxiv.org
• prio 6 The Context-Ready Transformer Concepts: Long Context Context Engineering Source: arxiv.org
• prio 6 Empirical study of allocation costs in calibration-guided LLM compression Concepts: LLM Evals Entities: Qwen3-8B Llama-3.2-1B Source: arxiv.org
• prio 6 Tandem RLVR trains a senior and junior model to improve handoff robustness Concepts: LLM Evals Entities: arXiv Qwen3-4B-Instruct Source: arxiv.org
• prio 6 Ontology-Guided Evidence Path Inference for Multi-hop KGQA Concepts: RAG Source: arxiv.org
• prio 6 Deployment-Side Adaptiveness in Multi-Horizon Volatility Forecasting Concepts: LLM Evals Entities: PatchTST Source: arxiv.org
• prio 6 Symbolic feedback-driven self-refinement for LLM planning Concepts: Agents LLM Evals Source: arxiv.org
• prio 6 Mining refactoring candidates in BDD test suites with clustering, classifiers, and LLM judges Concepts: LLM Evals Entities: arXiv SBERT XGBoost Source: arxiv.org
• prio 6 PRISON benchmarks criminal potential and anti-crime behavior in LLMs Concepts: LLM Evals Source: arxiv.org
• prio 6 Machine Learning for Coding Retail Product Names to Consumer-Price Categories Source: arxiv.org
• prio 6 Paper finds uncertainty does not change layer-wise inference dynamics much Concepts: LLM Evals Source: arxiv.org
• prio 6 Psychometric study compares LLM-based digital twins to human response patterns Concepts: LLM Evals Entities: LLMs Source: arxiv.org
• prio 6 Student-Centric Answer Selection for Distillation Concepts: LLM Evals Source: arxiv.org
• prio 6 Multilingual fine-tuning improves financial causality QA in FinCausal 2026 Concepts: LLM Evals Entities: multilingual BERT multilingual BART Llama 3.1 GPT-4.1-mini Source: arxiv.org
• prio 6 ATOD: Hybrid on-policy distillation for multi-turn autonomous agents Concepts: Agents Source: arxiv.org
• prio 6 Paper proposes a signal-coverage matrix for autoformalization evaluation Concepts: LLM Evals Entities: DeepSeek DeepSeek V4 Pro Source: arxiv.org
• prio 6 Truthfulness detection with sparse MLP value vectors Concepts: LLM Evals Source: arxiv.org
• prio 6 Formalizing latent thought evaluation in LLMs Concepts: LLM Evals Source: arxiv.org
• prio 6 Ask, Don’t Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement Concepts: LLM Evals Entities: alphaXiv Source: alphaxiv.org

🛠 Tools & Frameworks (10)

• prio 9 Connecting MQTT telemetry, TimescaleDB, and an LLM through MCP Concepts: MCP Tool Use Entities: Anthropic Google Microsoft Source: habr.com
• prio 8 DocumentDB announces an open-source MongoDB-compatible database on PostgreSQL Concepts: Vector Database Embeddings Entities: MongoDB PostgreSQL GitHub Microsoft Source: documentdb.io
• prio 7 Herdr: a terminal multiplexer for running and watching agent sessions Concepts: Agents Tool Use Source: github.com
• prio 7 Firecrawl ranks #10 in GitHub AI tools and adds web search, scrape, and interact APIs Concepts: Agents Tool Use MCP Entities: Firecrawl Source: github.com
• prio 7 A browser-based graphical shell for SSH Source: probablymarcus.com
• prio 6 Darts adds a unified FoundationModel layer for zero-shot time series forecasting Entities: Darts arXiv Chronos-2 TimesFM 2.5 Source: arxiv.org
• prio 6 OceanBase launches an AI database that unifies lakehouse storage and multimodal data in one engine Concepts: RAG Context Engineering MCP Agents Tool Use Entities: OceanBase Ant Group Ant Afu Lingguang Source: qbitai.com
• prio 6 NixOS 26.05 “Yarara” is now available Entities: NixOS Nixpkgs GNOME systemd Source: nixos.org
• prio 6 CachyOS June 2026 release adds Hyprland Noctalia, DoQ support, and performance tweaks Entities: CachyOS Hyprland GCC pacman Source: cachyos.org
• prio 6 ClinePass adds access to latest open-weight models without API key juggling Concepts: Open Source LLMs Entities: DAIR.AI Cline DeepSeek MiniMax Source: twitter.com

💬 Opinions (13)

• prio 10 How to Tie AI to Facts and Reduce Hallucinations Concepts: Context Engineering LLM Evals Source: habr.com
• prio 9 A Practical Pipeline for Turning PDFs into LMS Course Drafts Entities: WebRise LinkedIn McKinsey & Company Source: habr.com
• prio 9 Budget local AI builds under 100k rubles are benchmarked for CPU and cheap GPU inference Concepts: Long Context LLM Evals Entities: MSI ASUS AMD Tesla Source: habr.com
• prio 9 WebGL Screenshots Are Faster With ANGLE on Mesa llvmpipe Entities: Microlink Chrome ANGLE SwiftShader Source: microlink.io
• prio 9 Building a Home AI Server on a Budget Concepts: Long Context Entities: Habr AMD Ubuntu KDE Source: habr.com
• prio 8 HackerRank’s open-source ATS shows unstable LLM resume scoring Concepts: LLM Evals Entities: HackerRank GitHub LinkedIn Reddit Source: danunparsed.com
• prio 8 A deterministic support pipeline before LLM handoff Concepts: RAG Context Engineering Hybrid Search Entities: FinlogiQ Yandex DeepSeek YandexGPT Source: habr.com
• prio 7 Userscripts that add a jump-to-start button for AI chat replies Concepts: Tool Use Entities: OpenAI Anthropic Google DeepSeek Source: habr.com
• prio 7 Voice agent postmortem: three failures that were fixed in code, not prompts Concepts: Agents Entities: Yandex AIRA Postgres YandexGPT Source: habr.com
• prio 7 AI Speeds Up the Easy 80%, but Leaves the Hard 20% Untouched Entities: Bell Labs Source: jonathanbeard.io
• prio 6 ComfyUI as a prototyping workflow for image-generation pipelines Entities: NVIDIA ComfyUI Source: t.me
• prio 6 Why AI Pilot Projects Fail When Data and Processes Are Messy Entities: Gartner Source: habr.com
• prio 6 DAIR.AI shares a short intro to LLM-as-a-Judge Concepts: LLM Evals Entities: DAIR.AI X Source: twitter.com

FAQ

What is in the 2026-06-29 AI brief?

The 2026-06-29 brief selected 107 signal items for AI builders and filtered 238 items as noise, using the radar’s community-relevance scoring.