RSI × Agentic Search

更新 2026-10-30

RSI × Agentic Search Knowledge Base

最后更新：2026-10-30

这是什么

为 hao 持续追踪 RSI（recursive self-improvement / self-evolving agents）与 agentic search（deep research、multi-step retrieval）的交叉领域。目标是分类整理，看清 trend，区分已被证明有效的方案（proven）和只是 promising 的方案。

proven 的门槛（hao 定）：来自知名 AI 大厂，或者讨论度很高，并且有可复现的量化证据。Anthropic、OpenAI、Google 等的工程实践也收录在内。

File map

File 内容
trends.md 11 个主 trend（T12 于 2026-10-30 升回，标为 weaker evidence）和 1 个 watchlist trend（T9，证据较弱）。每个 trend 都有 statement / evidence（带 [src: ...]）/ evidence strength / momentum / open questions
proven.md Tier 1 proven (7)、Tier 0 proven pitfalls (4，其中 X1、X3 标为 weaker evidence)、Tier 2 主表 (12) 和 Tier 2 watchlist (6)
ideas.md 原 tracker 的 5 个 idea（A 组）、8 个组合 idea（B 组）、6 个 10-30 新增 idea（C 组），以及综合排名
relations.md 每篇 paper / blog 与 RSI、agentic search 的关系图（S→ / →S / 共享瓶颈），按 pipeline 阶段分表
log.md 按日期排列的日志（含已恢复的 09-26..10-29、空跑日和失败日，以及 GAP 标注）
papers/index.md 论文分类索引，链接到 9 个分类文件，并给出各 Source tier 的数量
papers/rsi-harness-evolution.md Harness 演化、泛化与正则化、verifier-free、evolve-the-evolver
papers/rsi-memory-evolution.md Memory 程序 / index 演化
papers/rsi-skill-evolution.md Skill library 及其失败模式
papers/rsi-training-and-theory.md 权重路线、co-training、理论、benchmark
papers/search-rl-policies.md RL search agents
papers/search-retrieval-and-query.md Retrieval program、query refinement、BM25
papers/search-context-and-deep-research.md Context 管理、deep research 架构、benchmark
papers/industry-practice.md 大厂和社区实践，每条带 [verified] / [from memory] / [unverified] 标签
papers/tech-blogs.md 每日任务发现的 lab / builder 博客（2026-10-30 新建），带置信度标签
archive/tracker-2026-09.md 原 tracker（09-08..09-25），只读归档

相关的深读笔记在 /workspace/agent/paper-digests/deep-dives/：meta-harness-vs-mesh、harness2、rrsi、rpl-retrieval-programs。

如何更新（daily task t-455e1f 应遵循）

  1. 新论文：追加到对应的 papers/<category>.md，必填字段为 id、title、Found 日期、Source（见第 7 条）、Key idea、Results、Search relevance、Sent。先 grep arXiv ID 去重。没有 ID 时写 "ID not recovered"，不要编造。
  2. log.md：每次运行追加一行带日期的记录，空跑和失败也要记（标 empty / failed）。
  3. 有新证据支持或反驳某个 trend 时，更新 trends.md 中该 trend 的 evidence 和 momentum。
  4. 升级到 proven.md Tier 1 必须满足三条：大厂或高讨论度（见第 7 条）、有 held-out 或生产数字、至少 2 个独立团队或已在生产中部署。否则只进 Tier 2。
  5. 工业实践进 industry-practice.md，必须标注置信度。
  6. 改动后更新本文件的"最后更新"日期。
  7. Inclusion rule（hao 2026-10）：只有来自大厂，或者讨论度很高的论文才算数。
    • 大厂名单（hao 2026-10-30 最终版，取代 10-29 的暂定名单）：
      • 美国：OpenAI、Anthropic、Google/Google DeepMind（含 Google Research、Google Cloud）、Meta（FAIR/GenAI/MSL）、Microsoft（MAI / Microsoft Research）。
      • 中国大厂和头部 AI lab：Alibaba（含 Qwen、Tongyi）、Tencent、ByteDance（Seed）、Baidu、Huawei、DeepSeek、Moonshot（Kimi）、Zhipu、MiniMax、Ant Group、Kuaishou、Meituan（含 LongCat）、JD。
    • 名单以外的机构一律不算大厂，只有达到下面的高讨论度门槛才算数。例如 Amazon、Salesforce、Sakana、NVIDIA、Apple、xAI、Mistral、Shanghai AI Lab、BAAI、Cohere，以及所有学术机构（包括 Stanford/Berkeley/CMU/MIT）。
    • 高讨论度：X 上互动很多、HF Daily Papers trending（2026-10-29 回填时按 ≥50 upvotes 计）、被广泛写博客讨论或被知名研究者转发、GitHub ≥1K stars，或引用很多（回填时按 Semantic Scholar ≥100 计）。
    • 每篇论文都要写 - Source: 字段：lab 名称（必须从 arXiv affiliations 核实，不要猜；只有 1 个次要合作者不算）、high-discussion（写出证据），或 other / unverified。
    • other / unverified 的条目放到该文件底部的 "Watchlist (low signal)"，不删除。名单以外机构的论文 Source 写成 other (<机构>)；如果达到高讨论度，写成 high-discussion (...)，并注明机构不在名单内。
    • trends.md 和 proven.md 的主证据只能来自 lab / high-discussion；只靠 other 支撑的 trend 或方案要降级（标 weaker evidence 或移入 watchlist）。
  8. Daily report 规则：每天给 hao 的报告只呈现 lab 或 high-discussion 的工作。other 的论文静默写入对应文件的 watchlist 和 log.md，不在消息里提及。
  9. 路径：本目录是 /workspace/agent/rsi-agentic-search/。旧任务 prompt 写的是 /workspace/group/... 路径，该路径不存在，任务 prompt 需要改为本目录。

Proven Solutions vs Promising-but-Unproven

最后更新：2026-10-30。

分级标准（严格执行）

Tier 1: Proven：必须同时满足以下三条：

  1. 来源可信：来自知名 AI 大厂（Anthropic / OpenAI / Google / DeepMind / Meta 等，完整名单见 index.md 的 inclusion rule），或讨论度很高（这是 hao 2026-10 设定的门槛）。Source 为 other / unverified 的论文只能作为辅助证据，不能单独支撑 Tier 1、Tier 0 或 Tier 2 主表。
  2. 有量化证据：在 held-out 或 production 中有数字，而不只是 evolve set 上的结果。
  3. 可复现：至少 2 个独立团队复现或延伸，或已在生产环境部署。

Tier 2: Promising but unproven：只有单篇论文或单一团队、只有 LLM-judged eval、数字只来自 abstract，或者只在非 search 任务上验证过。

Tier 0: Proven pitfalls：已被多个独立来源证实的失败模式，设计时必须规避。

注：标 [from memory] 的数字是回忆所得，引用前需要重新核对。

[src: ...] 标注（2026-10-29 回填）取自 papers/*.md 中每篇论文的 Source 字段。

2026-10-30 重新分级：按 hao 的最终大厂名单（index.md 第 7 条），Amazon、Salesforce、Sakana、Shanghai AI Lab、BAAI 不再算大厂，只有 high-discussion 时才算主证据；JD 等中国大厂改为算大厂。结果：Tier 1 的 7 条全部保留（P4 变薄）；X1 标为 weaker evidence；MASS 从 Tier 2 主表移到 Tier 2 watchlist。

Tier 1: Proven

P1. Broad agentic search 交给 subagent 做 context isolation；其他搜索作为主循环里的 tool

  • What：大范围探索型搜索（读很多网页或文件、大部分最后会被丢弃）由 subagent 并行执行，只返回结论。目标明确的搜索，或后端已经排好序的搜索，作为主循环的 tool/skill。
  • Evidence：
    • Anthropic multi-agent research system (2025-06)：多 agent 比单 agent Opus 好约 90% [from memory]，token 用量约 15x。 [src: Anthropic]
    • Claude Code Explore subagent（生产环境）。 [src: Anthropic]
    • Anthropic "fold subagents" article (2026-10)：合并为单循环后约 2x 速度。 [src: Anthropic]
    • Anthropic commerce agents article (2026-10)：search 作为 tool 写进 system prompt。 [src: Anthropic]
  • When to use：deep research、代码库探索、需要读大量原始内容的调研。
  • Caveats：
    • token 成本高（约 15x）。
    • 交接时会丢细节。
    • 文章建议一次只拆一个 subagent，并用 eval 验证。

P2. 关键词 / BM25 检索 + agent loop 作为默认检索底座

  • What：不急于上 dense / graph RAG；让 agent 迭代写 keyword query，底层用 BM25/grep，必要时再做 hybrid。
  • Evidence：
    • BM25 Wins at Scale 2607.26497：在 512K docs 上 BM25 胜出，agent 用 39x tokens。 [src: high-discussion]
    • SAGE benchmark 2602.05975 (10-06)：deep-research agent 的 query 天然是 keyword 形状。 [src: other]
    • Meta-Harness 2603.28052 自动搜出 4-route BM25。 [src: high-discussion]
    • RPL 2610.24854 以 BM25 + E5 + RRF 为基础。 [src: Meta]
    • Claude Code 用 grep/glob 做 agentic search [from memory, widely discussed]。 [src: Anthropic]
  • When to use：几乎所有 search harness 的 baseline。
  • Caveats：语义改写类 query、多语言和多模态场景下可能不够，需要 hybrid。

P3. 用 raw traces + coding-agent proposer 演化 harness

  • What：proposer（Claude Code / Opus 级 coding agent）直接读取历史候选的代码、完整 trace 和分数，然后改 harness 代码。不要先把 trace 压缩成摘要。
  • Evidence：
    • Meta-Harness 2603.28052（Stanford, Khattab, Finn；讨论度极高）：ablation 中 scores-only 34.6、summary 34.9、raw traces 50.0；TerminalBench-2 达 76.4%。 [src: high-discussion]
    • 多个独立团队在此基础上延伸并超越：
      • MESH-Harness 2610.05300：RAG math 53.2 vs 46.2。 [src: other]
      • RRSI 2609.24972 (Google)：TB2.1 74.2→80.2。 [src: Google, high-discussion]
      • Harness² 2610.14224 (Google)：WorkBuddy +11.5。 [src: Google, high-discussion]
      • Agentic Harness Engineering 2604.25850：TB2 69.7→77.0。 [src: high-discussion]
  • Source check (2026-10-29)：仍满足 Tier 1。Meta-Harness 和 AHE 是 high-discussion，RRSI 和 Harness² 是 Google；MESH-Harness 是 other，只作补充。
  • Production evidence (2026-10-30 blog 回填)：OpenAI self-improving tax agents（纠错 → trace → Codex 按 eval 修，75%+ 字段正确率 25%→86%，6 周）、Meta organizational second brain（四步纠错循环，0 回归）、MiniMax M2.7（模型自跑 100+ 轮改 scaffold，内部 +30%）、Microsoft SkillOpt（52/52 格最佳或并列）、Anthropic tool-testing agent（耗时 −40%）。 [src: OpenAI / Meta / MiniMax / Microsoft / Anthropic] 见 relations.md §2.6。
  • When to use：有可执行 eval 的 agent harness，包括有 gold answer 的 search QA。
  • Caveats：
    • 结果依赖 proposer 的能力。
    • eval 成本高。
    • 单独使用会过拟合（见 P4、X1）。

P4. Held-out keep-better gate + 编辑正则化

  • What：只接受在 held-out 上不退化（差异在噪声内即可）且复杂度增加有分数支撑的改动。每轮限制改动数量，用 leakage critic 拒绝写死任务答案的 diff，并剪掉无增益组件。
  • Evidence：
    • RRSI 2609.24972 (Google)：OOD 43.6 vs 未正则化 40.3；Meta-Harness 的 OOD 只有 +0.9；token 用量减少 36%。 [src: Google, high-discussion]
    • HarnessCompass 2608.01918：SWE-bench Verified 54→66，并可迁移。 [src: other]
    • HarnessEvolve 2609.00829：dual gating。 [src: Huawei]
    • RSEA 2606.28374：held-out gate。 [src: other]
    • EVOHARNESSBENCH 2609.04280 (Salesforce)：证明不加 gate 会出现遗忘。 [src: other (Salesforce), 2026-10-30 降级]
  • Source check (2026-10-29)：仍满足 Tier 1，依据是 RRSI（Google）、EVOHARNESSBENCH（Salesforce）和 HarnessEvolve（Huawei）。HarnessCompass 的 SWE-bench 54→66 和 RSEA 都是 other，只作补充。
  • Source check (2026-10-30)：勉强仍满足 Tier 1，证据比之前薄。EVOHARNESSBENCH（Salesforce）降为 other 后，主证据只剩 RRSI（Google，high-discussion）和 HarnessEvolve（Huawei），正好是 2 个独立的大厂团队。如果其中一篇被推翻，应降到 Tier 2。
  • When to use：任何自动接受改动的 RSI loop。
  • Caveats：
    • 2610.21646 证明不存在完美的 gate。
    • 应定期对整条已接受改动链重新在 fresh queries 上测试。

P5. LLM-proposed diff + automated evaluator 的进化循环（可验证领域）

  • What：AlphaEvolve 式循环：LLM 提出代码 diff，自动 evaluator 打分，保留种群并迭代。
  • Evidence：
    • AlphaEvolve (DeepMind 2025)：Google 数据中心调度和 kernel 已上生产 [verified exists; details from memory]。 [src: Google DeepMind]
    • Meta Trellis/KernelEvolve 2606.29823：生产中约 10x 更快到达目标，tokens -52%。 [src: Meta]
    • RPL 2610.24854 将 grammar mutation 用在 retrieval program 上，带来 +0.028 EM。 [src: Meta]
  • When to use：fitness 可自动计算的组件，例如 retrieval program、query rewriter（有 gold 时）、ranking config。
  • Caveats：没有 verifier 时不适用（见 T3）。

P6. RL 端到端训练的 deep-search policy

  • What：用 outcome RL（可以加 process reward）训练会 search、browse、verify 的模型。
  • Evidence：
    • OpenAI deep research (2025-02, production)。 [src: OpenAI]
    • Tongyi DeepResearch / WebSailor (Alibaba)。 [src: Alibaba]
    • Search-R1 系列成为通用 baseline（RPL 和 CoSearch 都与之对比）。 [src: Search-R1 本身未分类；RPL 为 Meta，CoSearch 为 other]
    • AREX 2607.21461：claim-level verify-and-patch，BrowseComp +22.9。 [src: high-discussion (BAAI 2026-10-30 起不算大厂，凭讨论度保留为主证据)]
  • When to use：有训练算力、需要小模型达到高 search 能力时。
  • Caveats：
    • 训练成本高。
    • 换 domain 需要重新训练。
    • unverifiable reward 会 collapse（2606.04507）。

P7. Prompt cache 按变化频率分段 + tool 结果裁剪

  • What：context 按 global / session / volatile 排序以最大化 cache 命中；tool 返回前去掉不会用到的字段。
  • Evidence：Anthropic commerce article (2026-10)，生产环境命中率 90–99%。 [src: Anthropic]
    • 结果裁剪（2026-10-30 回填）：dynamic filtering（BrowseComp 45.3→61.6，−24% token）、context editing（−84% token）、programmatic tool calling（−37%）、MCP code execution（150k→2k）、ResponseFormat（206 vs 72 token）。 [src: Anthropic] Codex agent loop、Manus 对前缀稳定性给出同样规则。 [src: OpenAI / high-discussion]
  • When to use：所有 production agent，尤其是 search 结果很长的场景。
  • Caveats：这是 infra 层面的手段，不属于 RSI；但 RSI 改 prompt 时会破坏 cache 前缀，需要把演化部分放在 volatile 段。

Tier 0: Proven pitfalls（多源证实的失败模式）

X1. Evolve-set 过拟合（weaker evidence，2026-10-30 标注）

  • Source check (2026-10-30)：EVOHARNESSBENCH（Salesforce）降为 other 后，直接证据里只有 RRSI（Google，high-discussion）是主证据，HarnessCompass 和 2610.21646 都是 other。T2 中的大厂论文（2610.20845 Alibaba、2610.25858 Google DeepMind）支持相近的"增益不能迁移"结论，但它们测的是共演化 drift 和遗忘，更贴近 X2。保留为 pitfall；需要第二篇大厂或高讨论度的 held-out 过拟合结果才算完全证实。
  • Evidence：RRSI（Meta-Harness OOD +0.9）、HarnessCompass、EVOHARNESSBENCH、2610.21646 理论结果。 [src: Google, high-discussion / other / other (Salesforce，2026-10-30 降级) / other]
  • Mitigation：使用 P4。

X2. Skill/policy 或 memory/policy 共演化陷阱

  • Evidence：
    • 2610.20845 (Alibaba)：drift 后 52–93% 增益丢失。 [src: Alibaba]
    • 2610.25858：global prompt edit 干扰，similarity memory collision。 [src: Google DeepMind]
    • SkillLearnBench：recursive drift。 [src: other (Amazon), 2026-10-30 降级]
  • Source check (2026-10-30)：SkillLearnBench 降级后，仍有 Alibaba 和 Google DeepMind 两个独立的大厂来源，不影响结论。
  • Mitigation：
    • 冻结一方；
    • skill 打 domain tag；
    • 用 admissibility 检索（2610.07309）；
    • 测试时做 skill-bank perturbation。

X3. Evolved skills 不跨模型迁移（weaker evidence，2026-10-29 标注；2026-10-30 起"有争议"）

  • Counter-evidence (2026-10-30)：SkillOpt（Microsoft Research blog）报告优化后的 skill 跨模型、跨 harness 迁移（6 benchmark × 7 模型 × 3 执行模式，52/52 最佳或并列）。 [src: Microsoft] 可能区别：SkillOpt 的 skill 短（中位约 920 token，1–4 次被接受的编辑），失败案例多是大型、多轮演化的 skill 库。
  • Source check：两篇核心论文 Skill Adaptation 2610.33024 和 EvoAgentBench 都是 other，只有 Weng 文中的 STOP 观察属于 high-discussion。保留为 pitfall，但证据强度低于 X1、X2、X4；需要大厂论文支持才算完全证实。
  • Evidence：
    • Skill Adaptation 2610.33024：同一套 skills 在 GPT 上 +31.1，在 Gemma 上 -8.8。 [src: other]
    • EvoAgentBench：evolved skills 效果不稳定。 [src: other]
    • Weng：STOP 在 GPT-3.5 上变差。 [src: high-discussion]
  • Mitigation：按目标模型重写 skill（+2–5 / 弱模型 +5–14）。

X4. Unverifiable reward → collapse / reward hacking

  • Evidence：
    • 2606.04507 [src: ByteDance]
    • 2609.26457（reward hacking 55%→32% 需要专门处理） [src: other]
    • Reward Hacking Is Transmissible 2610.12822 [src: Meta]
  • Mitigation：execution-based verification（VERSE；2026-10-30 起为 other（Amazon），只作参考）、critic 共演化、hindsight critique。

Tier 2: Promising but unproven

ID Source What Best evidence Why not proven yet
2610.14224 Harness² Google, high-discussion verifier-free behavioral-contrast harness RSI LAB +11.8, WorkBuddy +11.5 单篇论文；eval 是 LLM-judged；未在 BrowseComp 类任务上测试
2610.21512 QREF Tencent 冻结 agent 前加 query refiner (SFT→RL) WebSailor +33.6% rel，Tongyi DR +11% 单篇；没有 RSI loop
2610.24854 RPL Meta retrieval programs + mutation→distill 4B 0.515 vs Search-R1 0.469 只在 QA 上测试；需要 8xH100；只 mutate 最后一轮
2610.06689 PSA / 2610.07960 IndexAct Tencent / other 有状态的 programmatic search BrowseComp-Plus +7.6 单篇；未与 RSI 结合
2610.06361 PrisMem Microsoft 按 capability 演化 memory 程序 BEAM-1M +10.5 单篇；只测了 memory benchmark
2607.07820 DeepSearch-Evolve Tencent, high-discussion 无 teacher 的 self-distillation 搜索 BrowseComp +23.8 单篇；数字大，需要复现
2608.25593 JIT-Agent high-discussion per-query harness generation DeepSearchQA +9.1 vs GPT-5.6 单篇
2603.02766 EvoSkill high-discussion failure-driven skill evolution SealQA +12.1，BrowseComp +5.3 受 X2/X3 影响
2610.13025 Evo² Google DeepMind 训练 proposer 本身 SWE-bench V 41→48 coding 领域；单篇
2610.00906 ActiveSaddler Microsoft, high-discussion 用 bandit curriculum 选择 optimizer 训练任务 GAIA2 +4.4 单篇
2610.22331 KnowGPO Tencent per-step knowledge-gain reward 数字未恢复 单篇
2606.29823 Trellis Meta experience graph 生产中约 10x 只有 Meta 自报，独立复现缺失 → 介于 T1 与 T2 之间

Tier 2 watchlist (low signal)

2026-10-29 按 hao 的 inclusion rule 从 Tier 2 主表降级：这些行的来源是 other。MASS 于 2026-10-30 降级（Sakana 不再算大厂）。内容保留，不删除；若出现大厂或高讨论度的跟进工作，再升回主表。

ID Source What Best evidence Why not proven yet
2610.05300 MESH-Harness other 模块化 slot + bandit 组合 RAG math +7.0 vs Meta-Harness，test cost -44% 单一团队；预算只有 40 evals
2610.21170 MemScope other trace + memory state 驱动的 memory code patch GAIA 86.7 vs 70 单篇；GAIA 存在污染风险
2610.36826 RuHarness other rubric/harness 交替演化 在 IR 上有测试（数字未恢复） 刚发布（10-27）
2610.33024 Skill Adaptation other 按模型重写 skill +2–5（7 个模型） 单篇，但方向与 X3 一致
2610.12176 MASS other (Sakana，2026-10-30 从主表降级) workflow 演化 + 微调交替 1.2–1.6x perf/token 未在 search 上测试；Sakana 不在 2026-10-30 大厂名单，HF 4 upvotes
JevHarness / TypeSafe System One other 小模型快速做 typed decisions 自称约 200x 更快、400x 更便宜 early access；无独立 eval

Research Ideas: RSI × Agentic Search

最后更新：2026-10-30。

  • A 组：原 tracker（2026-09-08）提出的 5 个 idea，附 2026-10 的证据更新。
  • B 组：从已收集论文中组合出的新 idea。
  • C 组（2026-10-30）：读完 98 篇 lab / builder blog 后新增的 idea，推理过程见 relations.md。
  • 下面的 Ranking 是当前优先级，取代 B 组原来的排序。

Ranking（2026-10-30）

打分维度（各 1–5 分）：

  • 证据：组件是否有大厂或高讨论度的量化证据。
  • 增益：预期收益有多大。
  • 成本：是否需要训练或大量算力，5 分代表最便宜。
  • 空白：是否还没有人做。
  • 契合：和 hao 的方向（agentic system + RSI infra）是否贴合。

总分是五项之和，满分 25。

# Idea 证据 增益 成本 空白 契合 总分 一句话理由
1 C1 上下文策略演化 5 5 5 5 5 25 单一手工上下文策略就能 +12~16 BrowseComp（Anthropic、Zhipu、MiniMax、Kimi、Qwen），但没人搜索这个组合空间；不用训练
2 C2 预算匹配、抗污染的搜索评测底座（B6 升级版） 5 3 4 4 5 21 其他所有 idea 的前提：token 解释 80% 方差、基础设施噪声 6pp、eval-awareness。和 #1 一起做
3 B1 slot 演化（加入 context slot，排在 query writer 之前） 4 4 4 3 5 20 #1 的自然推广；QREF（Tencent）证明 slot 可以单独优化
4 C4 生产级"trace → eval → coding agent"搜索 harness 循环（A1 的落地版） 5 4 4 2 5 20 OpenAI tax、Meta、MiniMax、Microsoft 都在生产中验证过这个模式；空白小，但最容易出成果
5 C3 锦标赛 fitness 用于 deep-research harness RSI 4 4 3 4 4 19 ArenaRL（Alibaba，生产 69→80）+ co-scientist Elo + Harness²；解决没有 gold 时的 fitness；取代 B3
6 B7 / C6 架构（single loop / subagent / swarm）当作可搜索维度 4 3 3 4 4 18 Google 预测器选对 87%，Kimi PARL 学出编排；Anthropic 建议一次只改一个 subagent
7 C5 把 proposer 读 trace 当作 agentic search 来优化 3 3 4 5 3 18 Meta-Harness 中 raw trace 50.0 vs summary 34.9；proposer 实际上是在 grep trace；把 #1 的技术用到 proposer 上
8 B2 retrieval program 演化 4 3 3 3 4 17 RPL（Meta）、PSA（Tencent）；需要固定 corpus
9 B5 小模型做 search 决策的级联 2 3 4 3 5 17 契合 hao 正在评估的 JevHarness，但核心证据是 other
10 B4 skill-bank 扰动 gate 3 2 5 4 3 17 SkillOpt（Microsoft）显示短 skill 能迁移，X3 有争议，必要性降低
11 A3 per-query harness 3 3 3 3 3 15 JIT-Agent 单篇
12 B8 / A2 memory 代码演化 2 3 3 3 3 14 T9 证据弱；Memory Tax 警告常打不过 naive RAG
13 B3 process reward 当 fitness 3 2 3 2 3 13 大部分被 #5 取代；KnowGPO 数字没恢复
14 A4 meta-skill 演化 2 2 2 2 3 11 DeepMind 的现实检验；meta 层过拟合会放大
15 A5 跨 domain 迁移 2 2 3 2 2 11 负面证据多；SkillOpt 只证明跨模型迁移，没有证明跨 domain

建议路线：

  1. 先做 #2，同时做 #1 的第一版，验证上下文策略的搜索空间里确实有可以找到的增益。
  2. 然后把 #1 推广为 #3（多个 slot），并接入 #4 的生产循环。
  3. 遇到没有 gold 的任务时，用 #5 的锦标赛 fitness。

C. 2026-10-30 新增 idea

C1. 上下文策略演化（Context-policy evolution for search agents）

  • 组合：
    • Anthropic dynamic filtering（BrowseComp 45.3→61.6）和 context editing（−84% token）。
    • GLM-5 层级上下文管理（62→75.9 [snippet]）。
    • MiniMax interleaved thinking（+12.6）。
    • Kimi / Qwen discard-all，Tongyi IterResearch。
    • Meta-Harness proposer 和 RRSI gate。
  • 做法：
    • 把上下文策略写成一个 harness 模块，包含这些可以组合的选项：
      • 搜索结果过滤代码（由 proposer 写）；
      • 保留或丢弃推理过程；
      • 工具结果保留窗口（keep-recent N）；
      • 重置触发条件（按 token 阈值、按轮次，或 discard-all 只留计划）；
      • offload 到文件；
      • compaction 的 prompt。
    • proposer 读 raw trace，提出新的策略组合，在固定 token 预算下评测，按 P4 的 gate 接受。
  • 为什么排第一：
    • 单个设计点就有 +12~16 分，而且这些设计点来自 5 家不同的大厂。
    • 搜索空间小，并且是离散的。
    • 不需要训练。
    • 结果可以直接迁移到 hao 自己的 agent 上。
  • 风险：
    • 不同模型的最佳策略可能不同，MiniMax 的结论依赖模型是否在保留推理的条件下训练过。
    • 部分数字来自 snippet。
    • 必须控制 token 预算，否则"提升"可能只是多花了 token。

C2. 预算匹配、抗污染的搜索评测底座（升级 B6）

  • 组合：
    • Anthropic 的三项发现：token 解释 80% 方差、基础设施噪声 6pp、eval-awareness。
    • OpenAI：约 30% 的任务有问题；聚合 64 个样本可以涨 15–25%。
    • ATLAS（记忆率 9%；隐藏前排结果的检查）。
    • 再加上 B6 原来的 continual protocol。
  • 做法：
    • 每次对比都报告分数随 token 预算变化的曲线，而不是单个分数。
    • 屏蔽 benchmark 相关站点，并检测"搜到答案本身"的行为。
    • 固定容器资源。
    • 同时报 pass@k 和 pass^k。
    • held-out 用 ATLAS 风格的任务（隐藏前排结果后仍然要求能答出）。
    • 保留 B6 的序列化 domain 和遗忘测量。
  • 为什么：如果没有它，#1、#3、#5 的结果都无法判断真假。这是 RSI infra 的天然产品。

C3. 锦标赛 fitness 用于 deep-research harness RSI

  • 组合：ArenaRL（Alibaba：Open-DeepResearch 16.7→64.3，Amap 生产 69→80）+ Google co-scientist Elo + Harness² 行为对比 + RuHarness rubric。
  • 做法：没有 gold 的开放式研究任务，不再用 LLM judge 给候选 harness 打绝对分，而是让候选两两比较，用 seeded single-elimination 赛制（2N−2 次比较）给出排名，并定期用人工标注校准。
  • 为什么：绝对分 judge 的方差和偏差是 Harness² 的主要弱点，ArenaRL 已经在生产中证明相对排名更稳。
  • 风险：judge 偏好长文（X4）；需要控制报告长度。

C4. 生产级搜索 harness 自我改进循环（A1 的落地版）

  • 组合：OpenAI tax agents（25%→86%，6 周）+ Meta second brain（0 回归）+ MiniMax M2.7（100+ 轮，+30%）+ SkillOpt + P3/P4。
  • 做法：用户纠错和失败 trace → 自动生成回归 eval → coding agent 修改 search harness（prompt、skill、工具描述、上下文策略）→ 通过 gate 后上线 → 新增的 case 加入回归集。另外加一个长期指标（Cursor 的 keep rate 思路），防止出现 HumanLayer 说的那种质量缓慢下降。
  • 为什么：五家 lab 已经证明这个模式能用，最容易出成果。空白小，但对 hao 的价值高。

C5. 把 proposer 读 trace 当作 agentic search 来优化

  • 组合：Meta-Harness ablation（raw 50.0 vs summary 34.9）+ dynamic filtering + Meta tribal-knowledge context files + BM25（P2）。
  • 做法：RSI 的 trace 和候选档案越来越大时，proposer 读不完。可以：
    • 给 proposer 配 grep/BM25 检索；
    • 让 proposer 写过滤代码；
    • 预先生成每个候选的约 1k token "compass" 文件。
    • 衡量指标是 proposer 每轮提出的改动有多少被接受。
  • 为什么：这是 search → RSI 方向的直接应用，没有人做过。档案变大以后，这会成为 RSI 的瓶颈。

C6. 架构选择预测器（并入 B7）

  • 用 Google 180 配置研究的方法，对 hao 自己的任务分布拟合一个"任务特征 → 架构"预测器，作为 B7 的先验，减少 RSI 需要评测的架构组合数量。

A. 原 tracker 的 5 个 idea（附状态）

A1. Self-Evolving Search Harness

  • 原始想法：
    • 把 HarnessEvolve (2609.00829) 用到 agentic search 上。
    • 用已知成功的 search session 作为 reference trajectories，定位 search strategy 在哪里失败。
    • 演化 query 模板、source 选择规则和 depth policy，并加 dual gating。
  • 状态更新：
    • Meta-Harness 和 MESH-Harness 已在 RAG math 上做了类似的事。
    • RRSI 给出了正则化方案；Harness² 给出了无 gold 时的方案。
    • 目前还没有人在 BrowseComp-Plus 这类固定 corpus 的 deep-search benchmark 上做完整的 harness RSI。这仍是空白，也是最直接的机会。

A2. Search Memory Evolution

  • 原始想法：
    • 用 Recuris (2608.24876) 的结构：Working Memory 跟踪当前 research thread，Experiential Memory 积累 domain search patterns。
    • 由 meta-agent 从执行证据中演化 search skills。
  • 状态更新：
    • MemScope（GAIA 86.7）和 PrisMem 证明了"演化 memory 代码"可行。
    • 2610.25858 警告 similarity collision。
    • 建议改为：memory 带 domain tag，检索前加 admissibility check（2610.07309）。

A3. Per-Query Harness Generation

  • 原始想法：仿照 JIT-Agent (2608.25593)，为每个 query 生成定制的 search strategy（用哪些 source、搜多深、怎样改写 query）。
  • 状态更新：
    • JIT-Agent 在 DeepSearchQA 上 +9.1。
    • 可以与 PSA / RPL 的 program 表示结合：生成的是一个 search program，而不是一段 prompt。

A4. Meta-Search-Skill Evolution

  • 原始想法：仿照 MetaSkill-Evolve (2607.05297) 的两层循环：fast loop 按 domain 演化 search tactics；slow loop 演化"如何改进 search tactics"这个 meta-skill。
  • 状态更新：
    • Evo²、ActiveSaddler、RSIFlow 都在做 meta 层。
    • 风险是 T2 中的过拟合在 meta 层被放大。
    • slow loop 应该使用独立的 held-out domain。

A5. Cross-Domain Search Skill Transfer

  • 原始想法：在数据丰富的 domain（Wikipedia、PubMed）学 search skill，迁移到低资源 domain（WikiSkill pattern）。
  • 状态更新：
    • 负面证据增加：EvoAgentBench、Skill Adaptation（跨模型 -8.8）、Evolving Together（drift 后 52–93% 增益丢失）。
    • SkillQR 显示由 corpus 结构生成的 skill 在 domain shift 下更稳。
    • 建议把迁移单元从"skill 文本"换成"由 corpus 结构参数化的 skill"。

B. 新组合 idea

B1. 用 MESH 式 slot 演化 search harness，以 QREF 作为第一个 slot

  • 组合：MESH-Harness 2610.05300 + QREF 2610.21512 + RRSI 2609.24972。
  • 做法：
    • search harness 切成 5 个 slot：query writer / retriever config / reader / verifier / synthesizer。
    • 先只演化 query writer，因为 QREF 证明它可以独立提升；其他 slot 冻结。
    • 每个 slot 维护 implementation pool，用 LinUCB 选组合，接受改动前过 RRSI 的 gate。
  • 为什么可行：每个组件都有单独证据；固定 corpus 的 BrowseComp-Plus 可以做低成本 eval。
  • 风险：slot 之间存在 interaction，例如 query 风格必须匹配 retriever（keyword 形状，对应 T10）。

B2. 把 retrieval program 当作 RSI 的演化对象（harness 路线版 RPL）

  • 组合：RPL 2610.24854 + PSA 2610.06689 + Meta-Harness proposer。
  • 做法：
    • RPL 用 RL + grammar mutation 训练权重；这里改为冻结权重，让 coding-agent proposer 直接改 retrieval program 库（BM25 / dense / RRF / set ops）。
    • 用 raw traces 作为反馈。
  • 为什么可行：P3 + P5 都已 proven，program 表示比 prompt 更容易 mutate。
  • 风险：program 库可能过拟合到特定 corpus。

B3. KnowGPO 式 process reward 作为 harness evolution 的 fitness

  • 组合：KnowGPO 2610.22331 / HindSearch + Harness² / RuHarness。
  • 做法：
    • 不只看最终答对没有，还用 per-step knowledge gain 给 harness 候选打分。
    • 在没有 gold 的 deep research 中，用 RuHarness 式 rubric 交替更新，并退役失效的 rubric。
  • 为什么：outcome-only 的 fitness 方差大，而 eval 是 harness RSI 的主要成本（MESH 的出发点）。

B4. 用 skill-bank perturbation test 作为接受 search skill 的 gate

  • 组合：Evolving Together 2610.20845 + Skill Adaptation 2610.33024 + RRSI gate。
  • 做法：
    • 接受新 skill 之前，随机删掉或替换 bank 中 20% 的 skill，并换一个弱模型执行，检查增益是否还在。
    • 增益不在的 skill 只作为 model-specific 变体保存。
  • 为什么：X2 和 X3 已被多源证实；目前还没有人在 search skill 上做这个检查。

B5. Cost-aware cascade：小模型负责 search 决策，frontier 模型只做 intent 和 verify

  • 组合：PaSaMaster 2605.14306 + ARIA 2610.13870 + JevHarness/TypeSafe + SearchJev 2610.05107。
  • 做法：
    • query rewrite、是否继续搜索等 typed decision 交给小模型（Jev 类）。
    • ARIA 式 controller 决定何时升级给 frontier 模型。
    • RSI 只演化路由策略和小模型的 skill，并按目标模型重写这些 skill（X3）。
  • 为什么：hao 已在评估 JevHarness 做 query rewrite；BM25 Wins 显示 agent 的 token 成本是 39x。

B6. Continual search benchmark harness（基础设施）

  • 组合：EVOHARNESSBENCH 2609.04280 + 2610.25858 的协议 + HyperBrowseComp / BrowseComp-Plus。
  • 做法：
    • 按 domain 或语言序列化 search 任务，在每个阶段之后重测之前的 domain，同时报告 forgetting 和 transfer。
    • 2610.21646 建议在 fresh queries 上重测整条已接受改动链，应把这一步纳入协议。
  • 为什么：T2 是核心问题，但 search 领域没有对应的 benchmark；这也是 RSI infra 的天然产品。

B7. Subagent 还是 skill：让 RSI 自己决定 search 的放置位置

  • 组合：Anthropic fold-subagents article + Harness² 的 8 个可编辑组件（含 subagents）+ MASS。
  • 做法：把"search 作为 tool / skill / subagent"列为 harness 的可演化维度，用 eval 自动决定。Anthropic 的建议是一次只改一个 subagent，这可以直接写成 mutation 规则。
  • 为什么：工业界目前靠手工判断（T11），可以把它自动化。

B8. Memory code evolution for search sessions

  • 组合：MemScope 2610.21170 + PrisMem 2610.06361 + Retrieval Admissibility 2610.07309。
  • 做法：
    • patch agent 同时看 search trace 和当前的 search memory 内容（已访问 URL、已证实 claim、死胡同）。
    • 按 capability 分别演化 store、retrieve 和 admit 三段代码。
  • 为什么：MemScope 已在 GAIA / xBench 上验证"trace + state"优于只看 trace。

关系图：papers / tech blogs × RSI × Agentic Search

最后更新：2026-10-30。作者：hao 的助手（应 hao 2026-10-30 要求：思考这些工作和 RSI、agentic search 的关系，记录下来，并给 idea 排名）。 数据来源：papers/*.md、papers/tech-blogs.md、archive/blog-backfill-raw/（98 篇 lab / builder blog 原始笔记）。 标注 [snippet] 的数字来自搜索摘要，没有在原文中核对。

0. 核心判断（先看这里）

RSI 和 agentic search 不是两个并列的主题，而是三种关系叠在一起：

  1. Search 是 RSI 的改进对象。 搜索 agent 是 RSI 最好的试验场：任务长，反馈多，有 BrowseComp 这类可以自动打分的 benchmark，而且 harness 可以拆成模块。
  2. Search 是 RSI 内部的一个部件。 RSI 的 proposer 需要在历史 trace、候选 harness 和 memory/skill 库里找东西，这本身就是 agentic search。Meta-Harness 的 ablation（raw traces 50.0，summary 34.9，scores-only 34.6）说明：proposer 能不能读到原始证据，决定了 RSI 的效果。所以 search 做得好，RSI 也会更好。
  3. 两者有同样的瓶颈。 一是评测信号（verifier），二是上下文管理。两者都是长循环，都受 context rot 影响，都容易被评测噪声和污染误导。

由此得出三点：

  • 2026 年 lab blog 里，搜索效果提升最大的手段是上下文策略，不是检索器，也不是训练。见 §2.3。
  • 上下文策略是离散的、可以写成代码的 harness 选项，正好是 RSI 最擅长搜索的空间，但目前没有人系统地去演化它。这是排名第一的 idea（见 ideas.md 的排名）。
  • 所有关于"提升"的结论都要先过评测这一关：token 用量单独就解释了浏览类评测 80% 的方差（Anthropic）；基础设施配置会带来 6pp 差距；BrowseComp 存在污染和 eval-awareness。没有控制这些因素的 RSI 对比不可信。

1. 关系类型（每篇工作至少属于一类）

代号 关系 含义
S→ RSI 改进 search 用自我改进的方法提升搜索 agent（演化 harness、skill、memory，或用 RL 训练）
→S search 服务 RSI 检索经验、trace、skill、tool，是 RSI 循环里的一步
E 共享评测 benchmark、verifier、奖励设计、评测卫生
C 共享上下文管理 两者都是长循环，都需要裁剪、压缩、隔离
A 共享架构 单循环、subagent、swarm 怎么选
✗ 现实检验 / 反证 限制或质疑前面的乐观结论
— 关系弱 只是背景，不影响设计

2. 按搜索流水线环节逐一对应

2.1 查询与规划

工作 来源 和 search 的关系 和 RSI 的关系
QREF 2610.21512 Tencent 只训练 query refiner，冻结 agent：WebSailor +33.6% rel 说明 query writer 可以单独优化，是天然的 RSI slot（T5）
JIT-Agent 2608.25593 high-discussion 每个 query 生成一套策略：DeepSearchQA +9.1 等于每个 query 做一次小规模 RSI
Jina DeepSearch guide high-discussion gap-question 队列；query rewrite 是关键 无
Open Deep Research（LangChain） other 先 scope 再写 research brief 无

2.2 检索后端

工作 来源 和 search 的关系 和 RSI 的关系
BM25 Wins at Scale 2607.26497 high-discussion BM25 + agent loop 就够强（T10, P2） 后端简单、确定，RSI 的评测才便宜、稳定
RPL 2610.24854 Meta 检索写成程序，mutate 后 distill 检索程序是可以演化的对象（T4, P5）
PSA 2610.06689 Tencent 有状态的程序化搜索：BrowseComp-Plus +7.6 同上
Exa "search quality shapes RL" other 换搜索后端：SimpleQA 76.7 vs 69.2；token −69% 后端决定 RL 奖励有多密：后端会改变 RSI/RL 的上限，必须当作实验变量固定或单独扫描
Cursor semantic search other +12.5% QA；grep + 语义一起最好 embedding 用 agent 自己的 session 训练：对检索器的自我改进
Perplexity Search API other 为 agent 设计的检索 API，BrowseComp 0.371 无
Claude Code grep/glob、LangChain Deep Agents Anthropic / other lexical tools + loop 无

2.3 读结果与管上下文（收益最大的一环）

工作 来源 数字 关系
Dynamic filtering Anthropic BrowseComp Opus 4.6 45.3→61.6，Sonnet 4.6 33.3→46.6；DeepSearchQA F1 69.8→77.3；输入 token −24% C、S→：模型写代码过滤搜索结果。"过滤代码"本身可以被演化
Context editing + memory tool Anthropic 100 轮 web search eval token −84%；context editing 单独 +29%，加上 memory +39% C
GLM-5 层级上下文管理 Zhipu BrowseComp 62.0→75.9 [snippet] C：只改上下文策略就有约 14 分
Interleaved thinking MiniMax 保留推理状态：BrowseComp +12.6（+40.1%），GAIA +7.8 C：保留什么（推理过程）和丢掉什么同样重要
Kimi K2.5 / Qwen3.5 discard-all Moonshot / Alibaba BrowseComp 74.9 / 78.6 [snippet] C：上下文满了就清空、只留计划
Tongyi IterResearch / ReSum / AgentFold Alibaba 每轮重建 workspace，不让 ReAct 上下文无限增长 C
Kimi-Researcher Moonshot 上下文管理多跑 30% 轮次 C、S→
Programmatic tool calling / Tool Search / MCP 代码执行 Anthropic −37% token；工具定义 token −85%；150k→2k C、→S：tool search 本身就是检索
ResponseFormat（writing tools） Anthropic 206 vs 72 token C
Context Rot（Chroma）、How Long Contexts Fail high-discussion 输入越长越差；46 个工具失败，19 个成功 C：机制层面的证据
Breunig "State of Context Engineering" high-discussion "context inertia"：agent 固守早期假设，定期重置有帮助 C：为 discard-all 提供了一个解释
Manus、Codex agent loop、ADK high-discussion / OpenAI / Google KV-cache 命中率是首要指标；前缀稳定；compaction C：RSI 改 prompt 时会破坏 cache（P7）
Meta tribal knowledge context files Meta 59 个约 1k token 的文件；tool call 和 token 都 −40% C、S→：agent 预先生成上下文

小结：单个手工设计的上下文策略就能带来 +12 到 +16 BrowseComp 分，比大多数检索器或 RL 论文的增益都大，而且不需要训练。但每家只试了一两个设计点，这些策略可以组合（过滤、保留什么、何时重置、何时 offload、阈值多少），整个组合空间没有人系统搜索过。→ idea C1。

2.4 搜索策略训练（RL）

工作 来源 数字 关系
OpenAI deep research OpenAI HLE 26.6%；GAIA 67.36 S→（权重路线）
Kimi-Researcher Moonshot HLE 8.6→26.9；每条轨迹 70+ 次搜索 S→；gamma 衰减奖励 = 控制 over-search（T12）
Tongyi DeepResearch Alibaba 30B-A3B：BrowseComp 43.4，HLE 32.9 S→；合成数据 + 离线 Wikipedia 环境 = 便宜的 RL/RSI 沙箱
SWE-grep Cognition（high-discussion） 每轮 8 个并行工具调用，最多 4 轮；F1 奖励 S→、A：用 RL 训练专门的快速搜索 subagent
Composer Cursor（high-discussion） 在生产 harness 里做 RL S→：harness 和权重一起训练（T6）
Agent Lightning v1.0 Microsoft 在真实 harness 里做 RL：SWE-bench V 41.8→56.4 S→、T6：把 harness trace 接回权重训练的基础设施
KnowGPO、MPO、SearchMaster、AREX Tencent / high-disc / JD / high-disc process reward、credit assignment、防止浅层浏览 S→、E
DeepSeek V3.2 DeepSeek 1,800+ 合成环境 S→：环境规模决定泛化
MiniMax "Aligning to what" MiniMax 对 scaffold 扰动鲁棒 ✗、E：贴着 benchmark 训出来的 agent 换个 harness 就不行，这也是 RSI 过拟合（T2）在权重层的版本

2.5 架构（单循环 / subagent / swarm）

工作 来源 结论
Multi-agent research system Anthropic 2025-06 多 agent +90.2%；token 约 15x；token 用量解释 80% 方差；tool-testing agent 改写工具描述后耗时 −40%
Fold subagents Anthropic 2026-10 默认用单循环 + skill；只有测到 context 膨胀、需要并行或工具冲突时才用 subagent；一次只合并一个，用 eval 验证
Scaling agent systems（180 配置） Google 中心化协调在可并行任务上 +80.9%，在顺序任务上差 39–70%；独立 agent 的误差放大 17.2x，中心化 4.4x；预测模型能为 87% 未见配置选出最佳架构
Kimi K2.5 Agent Swarm（PARL） Moonshot 学出来的编排：BrowseComp 60.6→78.4，WideSearch 72.7→79.0 [snippet]
Thin Orchestrator Meta 14 个隔离的 subagent；单 agent 在 context 超过阈值后变差
Don't build multi-agents Cognition 共享完整 trace；隐含决策会冲突
Eval awareness Anthropic 多 agent 的非预期解法率是单 agent 的 3.7x

统一解释：广度型搜索（并行、可以丢弃的子查询）用 subagent 合算，深度型顺序推理用单循环合算。架构选择可以预测（Google 87%），也可以学出来（Kimi PARL）。→ 架构应该作为 RSI 的一个可搜索维度（B7），不必再靠人拍板。

2.6 自我改进（RSI ∩ search 的交集）

论文：Meta-Harness、RRSI、Harness²、Evo²、MESH、HarnessEvolve（P3、P4、T1）；EvoSkill（BrowseComp +5.3）；DeepSearch-Evolve（BrowseComp +23.8）；ReasoningBank（Google：WebArena +8.3）。

Lab blog 中的生产级自我改进循环（新证据，2026-10-30 回填）：

工作 来源 循环 数字
MiniMax M2.7 MiniMax 模型自己跑 100+ 轮：分析失败轨迹 → 改 scaffold、skill、memory、采样参数 → 跑 eval 内部指标 +30%
Self-improving tax agents OpenAI 人工纠错 → 生产 trace → Codex 按 eval 修 agent 75% 以上字段正确的报税单从约 25% 升到 86%（6 周）
Organizational second brain Meta 每次专家纠错走四步：诊断、编辑、eval、加入回归集 0 回归
SkillOpt Microsoft skill 文件当作可训练参数 52/52 格最好或并列；能跨模型和 harness 迁移
skill-creator Anthropic 给 skill 写 eval，检测 skill 是否已被基础模型吸收 6 个 skill 中 5 个的触发率提升
Tool-testing agent（research system） Anthropic agent 改写工具描述 耗时 −40%
Improving Deep Agents LangChain（high-disc） trace 驱动改 middleware TB2 52.8→66.5
Prime Agent high-discussion agent 自己增删改 prompt、subagent、skill、memory ARC-AGI-3 95.5%
Cursor harness other 用 keep rate 做在线 eval —

共同结构：trace → eval（含回归集）→ coding agent 改 harness → 有 gate 地接受。这就是 P3+P4，现在有 OpenAI、Meta、MiniMax、Microsoft、Anthropic 五家的生产或准生产证据。注意其中两种信号来源：

  • 人工纠错驱动（OpenAI tax、Meta second brain）：最稳。
  • 全自动（MiniMax M2.7、Prime Agent）：增益大，但只有自报数字。

对 X3 的冲击：SkillOpt（Microsoft，大厂）报告 skill 能跨模型、跨 harness 迁移，而 X3 的核心证据都是 other。X3 应进一步弱化为"有争议"。区别可能在于：SkillOpt 的 skill 很短（约 920 token，1–4 次被接受的编辑），而失败的案例里 skill 库很大、演化很多轮。

2.7 评测与奖励（RSI 和 search 共同的瓶颈）

工作 来源 对 search 的意义 对 RSI 的意义
BrowseComp OpenAI 标准 benchmark；聚合 64 个样本 +15–25% 采样聚合本身就能涨分，比较时必须控制采样数
Gemini Deep Research / DeepSearchQA Google 900 个因果链任务，测全面性 更难被单个答案 hack
ATLAS Exa（other） 记忆率 9%（其他 benchmark 48–61%）；隐藏前排结果后分数只剩约 50% 真正测搜索能力；适合当 RSI 的 held-out
Eval awareness on BrowseComp Anthropic 9 题有泄漏；2 例模型识别出自己在被测并解密答案 搜索能力越强，越会搜到答案本身：这是 RSI 在 search 上特有的 reward hacking 通道（X4）
Infrastructure noise Anthropic 资源配置差 6pp 3pp 以内的提升都不可信
Signal vs noise in coding evals OpenAI 约 30% 任务有问题 同上
ArenaRL Alibaba 开放式 deep research 用锦标赛排名做奖励：16.7→64.3；Amap 生产 69→80 没有 gold 时的 fitness 方案，有生产数字
Co-scientist Google 辩论 + Elo 锦标赛做自我评估 同上，用于科研
GAN-style harness、C compiler Anthropic 单独的 evaluator 修正自评偏差；"verifier 必须几乎完美" verifier 决定自主进展的上限
Demystifying evals、Hamel evals FAQ Anthropic / high-disc 从 20–50 个真实失败开始；pass^k；用二元判断 RSI gate 的工程方法

2.8 现实检验（✗）

  • DeepMind "no measurable acceleration"：capability index 每月约 +0.1，没有加速；只有 11.9% 的改动完全交给 AI。→ RSI 的收益在 harness / 产品层，不在前沿能力曲线上。
  • OpenAI research acceleration：每个人工工作日对应约 3.1 个 agent 工作日；中位研究员每天推理花费 $600 以上。→ 和 DeepMind 的结论不矛盾：生产力在涨，研究速度曲线还没变。
  • HumanLayer "software factories fail"：benchmark 不惩罚可维护性下降。→ 自动接受改动要有长期指标（Cursor 的 keep rate 是一个做法）。
  • Memory Tax / HABIT-Bench：专门的 memory 系统常常不如对原始历史做 naive RAG。→ 演化 memory 时要把 naive RAG 当作必须打赢的基线。
  • harness 应随模型变强而缩小：Anthropic 在 Opus 4.6 上去掉了 sprint 拆分；skill-creator 会检测 skill 是否已被吸收；Cursor 从静态上下文转向动态上下文。→ RSI 必须包含"删除"这类改动（RRSI 的剪枝也是这个方向）。

2.9 关系弱（—）

Anthropic source control、Managed Agents、Codex App Server、WebSockets、lethal trifecta（这条是安全约束：带 private data 的搜索 agent 不能同时有外发通道）、Co-Scientist 2026 的药物结果、Seed1.8、DeepSeek V3.1、GLM-4.5、Magentic-UI、Why We Think。

3. 对 trends / proven 的影响（已同步）

  • 新 trend T13："上下文策略是长程搜索最大的单一杠杆"，已写入 trends.md。
  • P3：加入生产证据（OpenAI tax、Meta second brain、MiniMax M2.7、SkillOpt）。
  • P7：加入 dynamic filtering 和 context editing 的数字。
  • X3：加入 SkillOpt 反证，标为"有争议"。
  • 下面两条暂不改动，等更多证据：
    • B3 中的 process reward 作为 fitness：被 ArenaRL 的锦标赛 fitness 部分取代。
    • Exa 的"后端决定奖励密度"：来源是 other，只作参考。

Tech Blogs (lab and builder posts)

Created 2026-10-30. Lab and builder blog posts found by the daily task (lab blogs; Cognition, Cursor, Perplexity, Exa, Jina, LangChain, LlamaIndex, Manus; Weng, Karpathy, Willison, Latent Space, Husain). Each entry carries a confidence tag ([verified] / [from memory] / [unverified]), as in industry-practice.md. Older engineering posts stay in industry-practice.md; items that also have an entry there say so in the Found line.

2026-10-30 backfill: 98 entries from archive/blog-backfill-raw/ (anthropic.md, uslabs.md, cnlabs.md, builders.md) were merged here. The Anthropic source-control post was already present and was merged into its existing entry. Numbers are copied from the raw notes; [unverified] marks posts whose body was not read (confirmed-exists only). Where only some numbers came from search snippets, the Confidence text says so.

Organization: H2 sections by theme. Within each section, listed-lab entries come first, then high-discussion, newest first. Builder posts tagged other are in the final watchlist. Total 101 entries: 92 main, 9 watchlist.

Agentic search & deep research

Google Research: Deep Review (Oct 14, 2026) [verified]

  • Found: 2026-10-30. Sent: yes.
  • Source: Google Research.
  • Key idea: public multi-agent system that reviews long documents in 4 stages (decompose, review in parallel by aspect, verify, synthesize).
  • Results: no quantified held-out comparison in the post.
  • Search relevance: deep-research-style orchestrator-workers over a single long document; context-isolation use of subagents (T11).
  • RSI relevance: none (a verify stage, but no learning across runs).

Google DeepMind: "Co-Scientist: A multi-agent AI partner to accelerate research" (May 19, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Google DeepMind. URL: https://deepmind.google/blog/co-scientist-a-multi-agent-ai-partner-to-accelerate-research/. Confidence: verified (fetched in this session).
  • Key idea: Follow-up to the 2025 co-scientist work, now published in Nature. It was developed with 100+ institutions.
  • Results: A repurposed drug candidate it proposed blocked 91% of a scarring-linked response.
  • Search relevance: A multi-agent generate, debate and rank system (tournament/Elo self-evaluation) applied to open-ended research.
  • RSI relevance: partial: tournament/Elo self-evaluation ranks the system's own hypotheses.

Anthropic: "Improved web search with dynamic filtering" (Feb 17, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic (claude.com). URL: https://claude.com/blog/improved-web-search-with-dynamic-filtering. Confidence: verified (fetched in this session).
  • Key idea: Claude writes and executes code to filter search/fetch results before they enter context.
  • Results:
    • BrowseComp: Sonnet 4.6 33.3% -> 46.6%; Opus 4.6 45.3% -> 61.6%.
    • DeepsearchQA F1: Sonnet 4.6 52.6% -> 59.4%; Opus 4.6 69.8% -> 77.3%.
    • Average +11% performance with 24% fewer input tokens. Code execution, memory, programmatic tool calling, tool search, and tool use examples also moved to GA.
  • Search relevance: Concrete context-efficient agentic search technique with deep-research benchmark gains.
  • RSI relevance: none directly; the agent writes its own filtering code per query, a small code-as-action step.

ByteDance Seed: "Official Release of Seed1.8: A Generalized Agentic Model" (Dec 18, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: ByteDance Seed. URL: https://seed.bytedance.com/en/blog/official-release-of-seed1-8-a-generalized-agentic-model. Confidence: verified.
  • Key idea: One model combining search, code and GUI agents with native vision, three thinking modes and token-efficient image encoding.
  • Results: BrowseComp-en 67.6 (claims to beat Gemini-3-Pro). VideoMME 87.8, ZeroBench 11.0.
  • Search relevance: a unified generalist agent model behind Doubao's deep research product.
  • RSI relevance: none.

Google: "Build with Gemini Deep Research" (Dec 11, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Google (blog.google, Developers). URL: https://blog.google/technology/developers/deep-research-agent-gemini-api/. Confidence: verified (fetched in this session).
  • Key idea: The Deep Research agent is built on Gemini 3 Pro and exposed via the Interactions API.
  • Results:
    • Scores: HLE 46.4%, DeepSearchQA 66.1% and BrowseComp 59.2%. Search results gave Gemini 3 Pro baselines of 43.2, 56.6 and 49.4; those baselines were not checked on the page.
    • New DeepSearchQA benchmark: 900 causal-chain tasks across 17 fields, measuring precision and recall. The post compares pass@8 vs pass@1 on a 200-prompt subset.
  • Search relevance: A deep research agent served as an API, plus a new comprehensiveness-focused search eval.
  • RSI relevance: none.

Moonshot AI (Kimi): "Introducing Kimi K2 Thinking" (Nov 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Moonshot AI (Kimi). URL: https://www.kimi.com/blog/kimi-k2-thinking.html. Confidence: verified.
  • Key idea: A "thinking agent" that runs 200-300 sequential tool calls without human intervention while staying on goal.
  • Results:
    • HLE w/ tools 44.9% (51.0% heavy mode), BrowseComp 60.2%, SWE-bench Verified 71.3%.
    • Test-time scaling over both thinking tokens (32k-128k budgets) and tool-call steps at 256k context. INT4 QAT gives about 2x faster generation.
  • Search relevance: a state-of-the-art open long-horizon search agent and the reference point for interleaved reasoning plus tool scaling.
  • RSI relevance: none.

Alibaba Tongyi: "Tongyi DeepResearch: A New Era of Open-Source AI Researchers" (Sep 16, 2025) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md ("Tongyi DeepResearch, WebSailor"). Sent: no (backfill).
  • Source: Alibaba Tongyi (DeepResearch Team, Tongyi Lab (Alibaba)). URL: https://tongyi-agent.github.io/blog/introducing-tongyi-deep-research/. Confidence: verified.
  • Key idea:
    • Full-stack recipe: agentic continual pre-training on synthetic entity-anchored QA and action data, graph/set-theory-based synthesis of hard questions, and on-policy GRPO with leave-one-out advantages and conservative negative sampling.
    • IterResearch "Heavy mode" splits work into research rounds with workspace reconstruction instead of an ever-growing ReAct context. Uses an offline-Wikipedia sim environment and cached/retried tool sandboxes.
  • Results:
    • 30B-A3B MoE: HLE 32.9, BrowseComp 43.4, BrowseComp-ZH 46.7, xbench-DeepSearch 75.
    • Builds on the WebAgent family (WebWalker, WebDancer, WebSailor/-V2, WebShaper, WebWeaver, WebResearcher, ReSum, AgentFold, etc.); repo has about 20.1k GitHub stars.
  • Search relevance: the most complete open recipe for training deep-research agents. IterResearch, ReSum and AgentFold are context-engineering techniques.
  • RSI relevance: none.

DeepSeek: "DeepSeek-V3.1 Release" (Aug 21, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: DeepSeek. URL: https://api-docs.deepseek.com/news/news250821. Confidence: verified.
  • Key idea: Framed as "our first step toward the agent era": hybrid thinking and non-thinking modes in one model, stronger multi-step search-agent reasoning, plus SWE/Terminal-bench gains.
  • Results: 840B tokens of extra long-context continued pretraining (128K), strict function calling (beta), and Anthropic API compatibility.
  • Search relevance: an early DeepSeek pivot to search/code agents. Specific benchmark numbers are in the model card, not the news post.
  • RSI relevance: none.

Zhipu (Z.ai): "GLM-4.5: Reasoning, Coding, and Agentic Abilities" (Jul 2025) [unverified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Zhipu (Z.ai). URL: https://z.ai/blog/glm-4.5. Confidence: confirmed-exists (JS-rendered blog; numbers come from search snippets).
  • Key idea: BrowseComp 26.4% vs Claude Opus 4 18.8% and o4-mini-high 28.3% (per search snippets).
  • Results: Ranked 2nd overall across 12 agentic/reasoning/coding benchmarks. Unified ARC (agentic, reasoning, coding) model.
  • Search relevance: an early open model explicitly trained for agentic browsing.
  • RSI relevance: none.

Anthropic: "How we built our multi-agent research system" (Jun 13, 2025) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md. Sent: no (backfill).
  • Source: Anthropic Engineering; Jeremy Hadfield, Barry Zhang, Kenneth Lien, Florian Scholz, Jeremy Fox, Daniel Ford. URL: https://www.anthropic.com/engineering/multi-agent-research-system. Confidence: verified (fetched in this session).
  • Key idea: Opus 4 lead + Sonnet 4 subagents beat single-agent Opus 4 by 90.2% on an internal research eval.
  • Results:
    • Token usage alone explains 80% of performance variance in browsing evals (tool calls and model choice another 15%); agents use ~4x and multi-agent systems ~15x the tokens of chat.
    • A tool-testing agent rewriting tool descriptions cut task completion time 40%.
    • Parallel subagents + parallel tool calls cut research time by up to 90% for complex queries.
  • Search relevance: The canonical deep-research architecture writeup; includes a self-improving tool-description loop.
  • RSI relevance: partial: a tool-testing agent rewrote tool descriptions and cut task time 40%.

Microsoft Research: "Magentic-UI, an experimental human-centered web agent" (May 19, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Microsoft Research. URL: https://www.microsoft.com/en-us/research/blog/magentic-ui-an-experimental-human-centered-web-agent/. Confidence: verified (fetched in this session).
  • Key idea: A multi-agent web agent built on Magentic-One (AutoGen), with co-planning, action guards, and a saved-plan gallery for reusing plans.
  • Results: On the GAIA validation set (162 tasks), a simulated user with side information raised task completion 71%, from 30.3% to 51.9%. Help was requested in only 10% of tasks.
  • Search relevance: Human-in-the-loop web research agent; plan reuse is an early form of skill memory.
  • RSI relevance: weak: the saved-plan gallery is an early form of reusable plan memory.

OpenAI: "Introducing deep research" (Feb 2, 2025) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md ("Deep research (product, Feb 2025)"). Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/introducing-deep-research. Confidence: verified (fetched in this session).
  • Key idea: Built on an o3 variant trained with end-to-end RL on browser and Python tasks. Runs take 5-30 minutes.
  • Results:
    • Humanity's Last Exam: 26.6%, vs o1 9.1% and o3-mini-high 13.0%. GAIA: 67.36 pass@1 and 72.57 cons@64, vs previous SOTA 63.64.
    • Later updates on the same page: an o4-mini lightweight version (2025-04-24), agent mode (2025-07-17), and MCP/apps plus site restriction (2026-02-10).
  • Search relevance: The founding product post for RL-trained deep research agents.
  • RSI relevance: none.

Google Research: "Accelerating scientific breakthroughs with an AI co-scientist" (Feb 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Google Research. URL: https://research.google/blog/accelerating-scientific-breakthroughs-with-an-ai-co-scientist/. Confidence: verified (fetched in this session).
  • Key idea: Multi-agent system using self-play debate and ranking tournaments. Its Elo self-evaluation correlates with GPQA accuracy.
  • Results: Evaluated on 15 expert research goals. Lab validations in AML drug repurposing, liver fibrosis targets and cf-PICI antimicrobial-resistance mechanisms.
  • Search relevance: Self-evaluation through tournaments scales test-time compute for research agents.
  • RSI relevance: partial: self-play debate and tournament ranking as self-evaluation.

Cognition: "Introducing SWE-grep and SWE-grep-mini: RL for Multi-Turn, Fast Context Retrieval" (Oct 16, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 97 points / 31 comments; inspired open clones such as op-grep and WarpGrep; Cognition is not a listed lab). URL: https://cognition.com/blog/swe-grep. Confidence: verified.
  • Key idea: SWE-grep-mini runs above 2,800 tok/s (20x Haiku 4.5) and SWE-grep above 650 tok/s; agents previously spent more than 60% of their first turn retrieving context
  • Results:
    • Up to 8 parallel tool calls per turn, at most 4 turns (3 explore + 1 answer), vs 10-20 serial turns
    • Multi-turn RL with weighted F1 reward (precision over recall) and per-sequence importance sampling
  • Search relevance: RL-trained fast-context subagent; a template for specialized search subagents.
  • RSI relevance: none.

LangChain: "Deep Agents" (Jul 30, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 130 points / 40 comments; LangChain is not a listed lab). URL: https://www.langchain.com/blog/deep-agents. Confidence: verified.
  • Key idea:
    • Four ingredients behind Claude Code, Deep Research and Manus: a detailed system prompt, a planning tool (even a no-op todo list), sub-agents, and a file system
    • Released the open-source deepagents package
  • Results: no quantified results in the raw note.
  • Search relevance: Defines the now-standard long-horizon harness template.
  • RSI relevance: none.

Jina AI: "A Practical Guide to Implementing DeepSearch/DeepResearch" (Feb 25, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (featured on simonwillison.net 2025-03-04; HN submissions small; Jina AI is not a listed lab). URL: https://jina.ai/news/a-practical-guide-to-implementing-deepsearch-deepresearch/. Confidence: verified.
  • Key idea:
    • Search, read and reason loop until answered or the token budget runs out; a FIFO queue of gap questions instead of recursion
    • Query rewriting was critical; embeddings used for query dedup (STS), not a vector DB; URL ranking combines frequency, path and semantic signals
    • "Beast mode" forces a final answer near the budget limit; knowledge is kept in context
  • Results: no quantified results in the raw note.
  • Search relevance: Concrete open blueprint for a deep-search agent loop.
  • RSI relevance: none.

Perplexity: "Introducing Perplexity Deep Research" (Feb 14, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 368 points / 168 comments; Perplexity is not a listed lab). URL: https://www.perplexity.ai/hub/blog/introducing-perplexity-deep-research. Confidence: verified.
  • Key idea: HLE 21.1%; SimpleQA 93.9%
  • Results: Most tasks finish in under 3 min (2-4 min target), running dozens of searches over hundreds of sources
  • Search relevance: Early widely used deep-research product baseline.
  • RSI relevance: none.

Context engineering & tool results

OpenAI: "Dreaming: Better memory for a more helpful ChatGPT" (Jun 4, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/chatgpt-memory-dreaming. Confidence: verified (fetched in this session).
  • Key idea: "Dreaming" builds memory in the background by synthesizing it from chat history, instead of saving memories inline during a chat. The first version launched in April 2025.
  • Results: Serving compute for Free users fell by about 5x.
  • Search relevance: Offline consolidation of memory is a production pattern for long-lived agent memory.
  • RSI relevance: partial: offline memory consolidation from history, a production form of learning from experience.

Anthropic: "An update on Claude Code quality reports (April 23 postmortem)" (Apr 23, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering. URL: https://www.anthropic.com/engineering/april-23-postmortem. Confidence: verified (fetched in this session).
  • Key idea: Three separate harness-side bugs (Mar 4 - Apr 16) caused perceived Claude Code degradation: reasoning-effort defaults, caching logic, system prompt.
  • Results:
    • A Mar 26 bug cleared old thinking blocks repeatedly instead of once, causing cache misses and "forgetfulness, repetition, and odd tool choices"; fixed Apr 10 (v2.1.101).
    • A system-prompt verbosity cap ("<=25 words between tool calls") caused a 3% eval drop on Opus 4.6 and 4.7; reverted. All fixed by Apr 20 (v2.1.116).
  • Search relevance: Concrete evidence that harness/context-management changes alone shift agent quality measurably; argues for harness-level regression evals.
  • RSI relevance: cautionary: harness-only edits moved eval scores by several points, so any self-modifying harness needs regression evals.

Meta: "How Meta Used AI to Map Tribal Knowledge in Large-Scale Data Pipelines" (Apr 6, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Meta Engineering. URL: https://engineering.fb.com/2026/04/06/developer-tools/how-meta-used-ai-to-map-tribal-knowledge-in-large-scale-data-pipelines/. Confidence: verified (fetched in this session).
  • Key idea: 50+ agents mapped 4 repos (3 languages, 4,100+ files) into 59 concise context files of 25-35 lines (about 1k tokens) each. Together they take under 0.1% of the context window ("compass, not encyclopedia").
  • Results:
    • AI context coverage went from about 5% to 100%, with about 40% fewer tool calls and tokens. Guidance work dropped from about 2 days to about 30 minutes.
    • Critic passes raised quality from 3.65 to 4.20. A dependency-graph lookup costs about 200 tokens vs about 6,000 for exploration.
  • Search relevance: Pre-computed, agent-generated context files as a scalable context-engineering pattern.
  • RSI relevance: partial: agents write and critique the context files other agents use.

OpenAI: "From model to agent: Equipping the Responses API with a computer environment" (Mar 11, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/equip-responses-api-computer-environment. Confidence: verified (fetched in this session).
  • Key idea:
    • The agent runtime combines a shell tool, a hosted container, skills and compaction. The model sets an output cap per command.
    • Compaction emits an encrypted, token-efficient item, either server-side at a threshold or via /compact.
    • Codex helped build its own compaction system.
  • Results: no quantified results in the raw note.
  • Search relevance: The productized harness primitives (skills, compaction, sandbox) behind long-running agents.
  • RSI relevance: partial: Codex helped build its own compaction system.

OpenAI: "Harness engineering: leveraging Codex in an agent-first world" (Feb 11, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: OpenAI, Ryan Lopopolo. URL: https://openai.com/index/harness-engineering. Confidence: verified (fetched in this session).
  • Key idea: Over 5 months the team shipped about 1M lines of code with 0 lines written by hand. About 1,500 PRs came from 3 engineers (about 3.5 PRs per engineer per day; the team later grew to 7), in about 1/10 the time of hand-coding.
  • Results:
    • AGENTS.md is kept to about 100 lines as a map, with docs/ as the system of record. Custom linters inject remediation instructions into the agent. Single Codex runs last 6+ hours. Agents query observability via LogQL and PromQL.
    • A "doc-gardening" agent plus golden principles act as garbage collection, replacing a manual 20% "Friday slop cleanup".
  • Search relevance: The canonical write-up on harness engineering: progressive disclosure, mechanical enforcement and agent-maintained docs.
  • RSI relevance: partial: agent-maintained docs and a "doc-gardening" agent keep the harness clean; humans set principles.

Zhipu (Z.ai): "GLM-5: from Vibe Coding to Agentic Engineering" (Feb 11, 2026) [unverified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Zhipu (Z.ai). URL: https://z.ai/blog/glm-5. Confidence: confirmed-exists (blog is JS-rendered so the body was not fetched; SWE/TB numbers verified via docs.z.ai/guides/llm/glm-5; BrowseComp numbers come from search snippets).
  • Key idea: Per secondary reports of the blog/tech report: BrowseComp is 62.0 base and 75.9 with context management. Hierarchical context management keeps relevant tool-call rounds and drops redundant history; a "keep-recent + discard" hybrid gives the top score.
  • Results: SWE-bench Verified 77.8 and Terminal Bench 2.0 56.2 (official docs); claims #1 open model on BrowseComp, MCP-Atlas and tau2-Bench.
  • Search relevance: a context-management policy alone adds about 14 BrowseComp points, which is strong evidence that context engineering matters for long-horizon search agents.
  • RSI relevance: none.

OpenAI: "Inside OpenAI's in-house data agent" (Jan 29, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/inside-our-in-house-data-agent. Confidence: verified (fetched in this session).
  • Key idea:
    • Six context layers: table usage, human annotations, Codex enrichment, institutional knowledge, memory and runtime context. Evals compare against golden SQL.
    • Lessons: "less is more", "guide the goal, not the path", "meaning lives in code".
  • Results: Serves 3.5k internal users over 600 PB and 70k datasets, powered by GPT-5.2.
  • Search relevance: Layered context plus memory plus evals for an enterprise data-search agent.
  • RSI relevance: partial: memory layer plus golden-SQL evals for an enterprise data-search agent.

OpenAI: "Unrolling the Codex agent loop" (Jan 23, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/unrolling-the-codex-agent-loop. Confidence: verified (fetched in this session).
  • Key idea:
    • Prompt caching needs exact prefix matches; cache hits make sampling cost linear rather than quadratic. Inconsistently ordered MCP tools caused cache misses, so config changes are appended as new messages instead of editing earlier ones.
    • Requests are stateless (no previous_response_id), which supports ZDR. Compaction runs past a token threshold via /responses/compact.
  • Results: The prompt is assembled from instructions, tools and input. AGENTS.md files are aggregated up to a 32 KiB default limit.
  • Search relevance: Concrete context-engineering rules (prefix stability, compaction) for long agent loops.
  • RSI relevance: none.

Google Developers Blog (ADK team): "Architecting efficient context-aware multi-agent framework for production" (Dec 4, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Google Developers Blog (ADK team). URL: https://developers.googleblog.com/architecting-efficient-context-aware-multi-agent-framework-for-production/. Confidence: verified (fetched in this session).
  • Key idea:
    • Thesis: "context is a compiled view over a richer stateful system". Tiers are working context, session, memory and artifacts.
    • A processor pipeline handles compaction, filtering and context caching. Artifacts externalize large state, memory is retrieved on demand, and handoffs between agents are scoped.
    • The post has no quantitative results.
  • Results: no quantified results in the raw note.
  • Search relevance: A framework-level design for context engineering in multi-agent harnesses.
  • RSI relevance: none.

Anthropic: "Effective harnesses for long-running agents" (Nov 26, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Justin Young. URL: https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents. Confidence: verified (fetched in this session).
  • Key idea:
    • Each session starts by checking the working directory, reading git log and progress files, and picking the next failing feature, which cuts tokens and prevents "doing too much at once."
    • Browser automation for end-to-end self-testing "dramatically improved performance."
  • Results: Initializer agent sets up the environment plus a JSON feature list (>200 features for a claude.ai clone, all initially "failing"); a coding agent then works incrementally.
  • Search relevance: Cross-context-window memory via progress files + git; a template for persistent agent state.
  • RSI relevance: partial: progress files and git history let an agent carry state across sessions, a prerequisite for persistent self-improvement.

Anthropic: "Introducing advanced tool use for Claude" (Nov 24, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Bin Wu (with Adam Jones, et al.). URL: https://www.anthropic.com/engineering/advanced-tool-use. Confidence: verified (fetched in this session).
  • Key idea: Tool Search Tool: ~85% token reduction (191,300 vs 122,800 tokens of context preserved); MCP eval accuracy Opus 4 49% -> 74%, Opus 4.5 79.5% -> 88.1%.
  • Results:
    • Programmatic Tool Calling: avg tokens 43,588 -> 27,297 (-37%) on complex research tasks; internal knowledge retrieval 25.6% -> 28.5%.
    • Tool Use Examples: complex-parameter accuracy 72% -> 90%.
  • Search relevance: On-demand tool loading and code-orchestrated tool calls for large-toolset research agents.
  • RSI relevance: none (tool-loading and tool-calling features).

Anthropic: "Code execution with MCP for efficient agents" (Nov 4, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Adam Jones, Conor Kelly. URL: https://www.anthropic.com/engineering/code-execution-with-mcp. Confidence: verified (fetched in this session).
  • Key idea: Also enables PII tokenization and persisting reusable code as "skills."
  • Results:
    • Presenting MCP tools as code APIs loaded on demand cut an example from 150,000 to 2,000 tokens (98.7% saving).
    • Intermediate results passing through context are costly (e.g. ~50,000 extra tokens for a 2-hour meeting transcript); keeping them in the execution environment avoids this.
  • Search relevance: Code-as-action harness pattern; reusable saved functions are a route to self-accumulating skills.
  • RSI relevance: partial: saved reusable code becomes "skills", a route to self-accumulating skill libraries.

MiniMax: "Why Is Interleaved Thinking Important for M2?" (Nov 3, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: MiniMax. URL: https://www.minimax.io/news/why-is-interleaved-thinking-important-for-m2. Confidence: verified.
  • Key idea: Ships a reasoning_details field for its OpenAI-compatible API and native support for its Anthropic-compatible API, and pushes partners (OpenRouter, Ollama, Cline, etc.) to standardize.
  • Results:
    • Keeping prior-turn reasoning in context (plan -> act -> reflect), rather than dropping it as the OpenAI Chat Completions convention does, drives M2's agent performance.
    • Gains from keeping reasoning state: BrowseComp +12.6 (+40.1%), Tau2 +23 (+35.9%), GAIA +7.8 (+11.5%), xBench +6.0 (+9.1%), SWE-bench Verified +2.2 (+3.3%).
  • Search relevance: a concrete ablation showing that what you keep in context (reasoning traces) changes deep-search accuracy by about 40%. Widely discussed (Simon Willison, MacStories).
  • RSI relevance: none.

Anthropic: "Equipping agents with Agent Skills" (Oct 16, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Barry Zhang, Keith Lazuka, Mahesh Murag. URL: https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills. Confidence: verified (fetched in this session).
  • Key idea:
    • Three-level progressive disclosure: metadata in system prompt -> SKILL.md when relevant -> bundled files only as needed, so skill context is "effectively unbounded."
    • Skills bundle deterministic scripts for operations better done in code than by token generation.
    • Build skills iteratively from observed capability gaps and watch how Claude actually uses them;
  • Results: no quantified results in the raw note.
  • Search relevance: Foundational for skill-library-based self-improving agents.
  • RSI relevance: partial: the skill format that skill-library self-improvement work builds on.

Anthropic: "Effective context engineering for AI agents" (Sep 29, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Prithvi Rajasekaran, Ethan Dixon, Carly Ryan, Jeremy Hadfield. URL: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents. Confidence: verified (fetched in this session).
  • Key idea:
    • Aim for "the smallest possible set of high-signal tokens"; system prompts at the right altitude.
    • Just-in-time retrieval via lightweight references (paths, URLs) instead of pre-loading.
    • Long-horizon techniques: compaction, structured note-taking (memory files), sub-agent architectures.
  • Results: Context is a finite "attention budget": "context rot" degrades recall as tokens grow (n^2 pairwise relationships).
  • Search relevance: Defining Anthropic text on context engineering; underpins memory and research-agent design.
  • RSI relevance: indirect: memory files and compaction are the state that self-improving agents edit.

Anthropic: "Managing context on the Claude Developer Platform" (Sep 29, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic (claude.com). URL: https://claude.com/blog/context-management. Confidence: verified (fetched in this session).
  • Key idea: Launches context editing (auto-clears stale tool calls/results) and a file-based memory tool that persists across conversations.
  • Results:
    • Memory + context editing: +39% over baseline; context editing alone: +29%.
    • In a 100-turn web search eval, token consumption fell 84% and workflows that would otherwise fail completed.
  • Search relevance: Productized memory + context pruning for long-running research agents.
  • RSI relevance: partial: a persistent file-based memory tool is the substrate for cross-session learning.

Microsoft Research: "Tool-space interference in the MCP era: Designing for agent compatibility at scale" (Sep 11, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Microsoft Research. URL: https://www.microsoft.com/en-us/research/blog/tool-space-interference-in-the-mcp-era-designing-for-agent-compatibility-at-scale/. Confidence: verified (fetched in this session).
  • Key idea: Surveyed 1,470 MCP servers (from Smithery's 7,000+ and Docker MCP Hub). The largest exposes 256 tools, and 10 others expose more than 100 each.
  • Results:
    • Large tool spaces can cut performance by up to 85% for some models (cited). Tool responses can overflow context: for example, 93 servers overflow Phi-4's 16k context on a single call.
    • Name collisions are common ("search" appears 32 times). Only 7.6% of servers expose resources.
  • Search relevance: Tool count, tool-response length and naming are context-engineering risks for agent harnesses.
  • RSI relevance: none.

Anthropic: "The 'think' tool for complex tool use" (Mar 20, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering. URL: https://www.anthropic.com/engineering/claude-think-tool. Confidence: verified (fetched in this session).
  • Key idea: tau-bench airline: think tool + optimized prompt 0.570 vs 0.370 baseline pass^1 (54% relative improvement).
  • Results:
    • tau-bench retail: think tool alone 0.812 vs 0.783 baseline.
    • SWE-bench: contributed to Claude 3.7 Sonnet's 0.623; isolated effect +1.6% on average (p < .001).
    • Gains held through pass^k for k up to 5.
  • Search relevance: Cheap harness-level intervention for mid-trajectory reasoning over tool results.
  • RSI relevance: none.

Drew Breunig: "The State of Context Engineering" (Oct 24, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 273 points / 66 comments, 2026-10-25; Drew Breunig is not a listed lab). URL: https://www.dbreunig.com/2026/10/24/the-state-of-context-engineering.html. Confidence: verified.
  • Key idea: New long-horizon problems: "fighting the weights" and "context inertia" (agents fixate on early hypotheses; periodic context resets help)
  • Results:
    • Revisits his 4 failure modes (poisoning, distraction, confusion, clash); e.g. Gemma 3 12B fell below 50% instruction-following after a few contradictory examples
    • 6 tactics: augmentation (RAG/skills), tool loadout, quarantine (subagents), pruning, summarization/compaction, offloading to the environment
  • Search relevance: Up-to-date map of context management for long-running, self-improving agent harnesses.
  • RSI relevance: indirect: "context inertia" and periodic resets matter for long-running self-improving agents.

Manus: "Context Engineering for AI Agents: Lessons from Building Manus" (Jul 18, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (a widely referenced context-engineering case study; note: HN submissions only reached about 17 pts; Manus is not a listed lab). URL: https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus. Confidence: verified.
  • Key idea:
    • Mask tool logits instead of adding or removing tools, to keep the cache and avoid schema errors
    • Use the file system as unlimited externalized memory; keep failed actions and stack traces in context; vary serialization to avoid few-shot mimicry
  • Results: KV-cache hit rate is the key metric: cached input $0.30/MTok vs $3.00 uncached (10x); input:output ratio about 100:1
  • Search relevance: Production context and memory design for long-horizon agents.
  • RSI relevance: none.

Chroma: "Context Rot: How Increasing Input Tokens Impacts LLM Performance" (Jul 14, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 260 points / 59 comments; Chroma is not a listed lab). URL: https://www.trychroma.com/research/context-rot. Confidence: verified.
  • Key idea: Lower question-answer similarity and more distractors make the drop steeper; shuffled haystacks beat coherent ones
  • Results:
    • 18 LLMs (GPT-4.1, Claude 4, Gemini 2.5, Qwen3, etc.) degrade as input length grows, even on simple retrieval
    • LongMemEval: large gap between focused and full prompts; even repeating words (12 lengths up to 10k words) degrades
  • Search relevance: Empirical basis for compaction, retrieval and sub-agent isolation in agent memory.
  • RSI relevance: none.

LangChain: "Context Engineering (for Agents)" (Jul 2, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (widely cited taxonomy in the June-2025 context-engineering wave alongside Karpathy/Willison; LangChain is not a listed lab). URL: https://www.langchain.com/blog/context-engineering-for-agents. Date: 2025-07-02 (date shown on page). Confidence: verified.
  • Key idea: Taxonomy: write (scratchpads, memories), select (RAG, tools, memories), compress (summarize, trim), isolate (multi-agent, sandboxes, state)
  • Results: Cites Anthropic's multi-agent researcher using up to 15x more tokens than chat
  • Search relevance: Standard vocabulary for agent memory and context management.
  • RSI relevance: indirect: the "write" category (scratchpads, memories) is where agents store learned state.

Philipp Schmid: "The New Skill in AI is Not Prompting, It's Context Engineering" (Jun 30, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 915 points / 518 comments; Philipp Schmid is not a listed lab). URL: https://www.philschmid.de/context-engineering. Confidence: verified.
  • Key idea:
    • Context = system prompt + user prompt + history + long-term memory + RAG + tools + output schema
    • Most agent failures are context failures, not model failures; context engineering is "a system that runs before the main LLM call"
  • Results: no quantified results in the raw note.
  • Search relevance: The most-discussed popular framing of context engineering.
  • RSI relevance: none.

Simon Willison: "Context engineering" (Jun 27, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (part of the widely shared Tobi Lutke/Karpathy terminology shift, June 2025; Simon Willison is not a listed lab). URL: https://simonwillison.net/2025/Jun/27/context-engineering/. Confidence: verified.
  • Key idea: Argues the inferred meaning of a term drives adoption
  • Results: Endorses "context engineering" over "prompt engineering," quoting Karpathy (2025-06-25): filling the window with task descriptions, examples, RAG, tools, state and history
  • Search relevance: Records the origin of the term.
  • RSI relevance: none.

Drew Breunig: "How Long Contexts Fail" (Jun 22, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (failure-mode taxonomy widely reused in context-engineering posts; note: HN submissions small, about 10 pts; Drew Breunig is not a listed lab). URL: https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html. Date: 2025-06-22 (companion: "How to Fix Your Context", 2025-06-26). Confidence: verified.
  • Key idea: Poisoning, distraction (above about 100k tokens agents repeat past actions), confusion, clash
  • Results:
    • Llama 3.1 405B correctness falls around 32k tokens; sharded multi-turn prompts drop 39% on average (o3: 98.1 to 64.1)
    • Quantized Llama 3.1 8B fails with 46 tools but succeeds with 19
  • Search relevance: Failure modes that harness and memory designs have to work around.
  • RSI relevance: none.

Single loop vs multi-agent

Anthropic: "Building agents: When to fold subagents into your main agent loop" (Oct 22, 2026) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md ("Building agents: when to fold subagents..."). Sent: no (backfill).
  • Source: Anthropic (claude.com); Diane Sarkis. URL: https://claude.com/blog/building-agents-when-to-fold-subagents-into-your-main-agent-loop (mirror: https://claude.com/resources/articles/building-agents-when-to-fold-subagents-into-your-main-agent-loop). Date: 2026-10-22 (page datePublished; not 2026-10-29 as briefed). Confidence: verified (fetched in this session).
  • Key idea:
    • Keep subagents only for measured context-bloat degradation, parallel work needing fresh disposable contexts, or tool-selection/conflicting-instruction problems.
    • Migrate incrementally: build a shared end-to-end eval set, fold in one subagent at a time, compare.
  • Results:
    • Across several enterprise deployments, a single agent with skills frequently beat both one-big-prompt and subagent designs on quality, often at lower cost/latency; one customer saw ~2x faster runtime with a single loop vs multi-agent.
    • Progressive disclosure of skills/tools avoids context bloat; Claude Code's system prompt cut by >80% for advanced models this way.
  • Search relevance: Direct harness-design guidance (single loop + skills vs multi-agent); notable reversal from the 2025 multi-agent research stance.
  • RSI relevance: indirect: the shrink-the-harness lesson (fold subagents, keep evals) is the kind of change a harness-evolution loop should be able to make and verify.

Meta: "The Thin Orchestrator: How We Made Our AI Agents More Reliable by Giving the Router Less to Do" (Oct 19, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Meta Engineering. URL: https://engineering.fb.com/2026/10/19/ml-applications/thin-orchestrator-ai-agent-architecture/. Confidence: verified (fetched in this session).
  • Key idea: A 1,300-line orchestrator routes 20+ workflows to 14 specialist subagents. Each subagent has isolated context, and the orchestrator does no domain work itself.
  • Results:
    • The earlier single "load everything" agent got worse once context passed a threshold. A single-agent alternative reached 70-80% success only with per-task prompt engineering, and 10-20% without it.
    • 25+ knowledge files load on demand when a subagent runs. New use cases are new workflow templates that reuse the same agents, with zero new agent code.
    • Config prep for a new column went from 3-4 days by hand (about 1 day with the earlier system) to under an hour. About two-thirds of changes land without significant human edits.
  • Search relevance: Shows context isolation and progressive loading of knowledge as a harness pattern. Improvements come in independently from three layers: a better model, knowledge-file edits and workflow templates.
  • RSI relevance: partial: improvements land through knowledge-file edits and workflow templates without new agent code.

Anthropic: "A guide to the anatomy of effective commerce agents" (Sep 2, 2026) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md ("The anatomy of effective commerce agents"). Sent: no (backfill).
  • Source: Anthropic (claude.com); Ali Shazal, Matthew Koen. URL: https://claude.com/blog/the-anatomy-of-effective-commerce-agents (mirror: https://claude.com/resources/articles/the-anatomy-of-effective-commerce-agents). Date: 2026-09-02 (page datePublished; not 2026-10-29 as briefed). Confidence: verified (fetched in this session).
  • Key idea: Single agent with skills "consistently" outperformed one-prompt and subagent designs on quality, often at lower cost/latency.
  • Results:
    • Prompt caching hits 90-99% in production; cached input reads cost 1/10 of fresh tokens.
    • Parallel tool execution + eager dispatch cut multi-second tool delays to a few hundred ms; progressive rendering hides 5+ s latency.
    • Evals: 50-100 snapshot-based test cases per user flow; write a negative counterpart for every positive case.
  • Search relevance: Production harness + eval recipe (architecture, caching, eval set design) for domain agents.
  • RSI relevance: none (production architecture and eval recipe; no self-improvement loop).

Anthropic: "Harness design for long-running apps" (Mar 24, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering (Anthropic Labs); Prithvi Rajasekaran. URL: https://www.anthropic.com/engineering/harness-design-long-running-apps. Confidence: verified (fetched in this session).
  • Key idea: A separate evaluator fixes self-assessment bias (agents "confidently praising" mediocre work).
  • Results:
    • GAN-style planner / generator / evaluator harness: retro game maker took 6 h and $200 vs 20 min and $9 solo, but produced a playable game where the solo agent's was broken.
    • Moving Opus 4.5 to 4.6 allowed removing sprint decomposition; a DAW build ran 3 h 50 min for $124.70 with no quality loss.
    • Four explicit grading criteria (design quality, originality, craft, functionality) make subjective quality gradable over 5-15 iterations.
  • Search relevance: Core RSI-style generator/evaluator loop; shows harness scaffolding should shrink as models improve.
  • RSI relevance: partial: a separate evaluator drives 5-15 improvement iterations on the artifact; also shows harness scaffolding can be removed as models improve.

Anthropic: "Building a C compiler with parallel Claudes" (Feb 5, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Nicholas Carlini. URL: https://www.anthropic.com/engineering/building-c-compiler. Confidence: verified (fetched in this session).
  • Key idea: The verifier must be "nearly perfect, otherwise Claude will solve the wrong problem."
  • Results:
    • 16 parallel agents built a Rust C compiler from scratch over ~2,000 Claude Code sessions and two weeks.
    • About $20,000 in API costs: 2B input tokens, 140M output tokens.
    • About 100,000 lines; boots Linux 6.9 on x86, ARM, RISC-V; ~99% pass rate on compiler test suites including GCC torture tests.
  • Search relevance: Long-horizon autonomous multi-agent harness; shows verifier quality is what bounds autonomous progress.
  • RSI relevance: partial: long autonomous progress is bounded by verifier quality, the same constraint as RSI loops.

Google Research: "Towards a science of scaling agent systems: when and why agent systems work" (Jan 28, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Google Research. URL: https://research.google/blog/towards-a-science-of-scaling-agent-systems-when-and-why-agent-systems-work/. Confidence: verified (fetched in this session).
  • Key idea: Tested 180 configurations across 5 architectures, on benchmarks including BrowseComp-Plus and Finance-Agent.
  • Results:
    • Centralized coordination gave +80.9% on Finance-Agent, but multi-agent variants were 39-70% worse on sequential PlanCraft. Errors amplified 17.2x for independent agents vs 4.4x for centralized.
    • A predictive model (R^2=0.513) picks the best architecture for 87% of unseen configurations.
  • Search relevance: Quantitative guidance on when multi-agent search and research harnesses help or hurt.
  • RSI relevance: none directly; gives a predictor for choosing architectures that a harness search could use.

Moonshot AI (Kimi): "Kimi K2.5 Tech Blog: Visual Agentic Intelligence" (Jan 27, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Moonshot AI (Kimi). URL: https://www.kimi.com/blog/kimi-k2-5.html. Confidence: verified (swarm/PARL details fetched; benchmark numbers come from search results).
  • Key idea: Agent Swarm, trained with Parallel-Agent RL (PARL): an orchestrator self-decomposes tasks into up to 100 sub-agents and 1,500 tool calls, with up to 4.5x lower latency than sequential execution.
  • Results: Reported (search snippets/model card): BrowseComp 60.6 single-agent vs 78.4 swarm, WideSearch F1 72.7 -> 79.0, HLE-Full w/ tools 50.2, BrowseComp 74.9 with "discard-all" context management, DeepSearchQA 77.1.
  • Search relevance: learned multi-agent orchestration for wide/deep search, an alternative to longer single-agent context.
  • RSI relevance: none directly; orchestration is learned by RL rather than hand-designed.

Anthropic: "Building effective agents" (Dec 19, 2024) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md. Sent: no (backfill).
  • Source: Anthropic Engineering; Erik S., Barry Zhang. URL: https://www.anthropic.com/engineering/building-effective-agents. Date: 2024-12-19 (seminal, pre-window). Confidence: verified (fetched in this session).
  • Key idea:
    • Start with the simplest solution; add agentic complexity only when it demonstrably helps.
    • Workflows (predefined code paths: chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) vs agents (model-directed).
    • Treat the agent-computer interface (ACI) like HCI; for SWE-bench they spent more time optimizing tools than the overall prompt.
  • Results: no quantified results in the raw note.
  • Search relevance: Foundational taxonomy; its evaluator-optimizer and orchestrator-worker patterns underlie later research and RSI harnesses.
  • RSI relevance: indirect: the evaluator-optimizer pattern is the base of generator/evaluator RSI loops.

Cognition: "Don't Build Multi-Agents" (Jun 12, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 123 points / 89 comments; widely debated against Anthropic's multi-agent research post; Cognition is not a listed lab). URL: https://cognition.com/blog/dont-build-multi-agents. Confidence: verified.
  • Key idea: Recommends a single-threaded linear agent plus an LLM compression model for long tasks
  • Results: Principle 1: share full agent traces, not just messages. Principle 2: actions carry implicit decisions, and conflicting decisions produce bad results (Flappy Bird example)
  • Search relevance: Core architecture debate (single vs multi-agent) for harness design.
  • RSI relevance: none.

Self-improvement / RSI in practice

Google DeepMind: "no measurable acceleration" of AI research from AI (Oct 14, 2026) [verified]

  • Found: 2026-10-30. Sent: yes.
  • Source: Google DeepMind (DeepMind safety report; shared on X by Haydn Belfield; read via the-decoder's write-up, not the primary PDF).
  • Key idea: a lab measuring its own RSI progress finds no speed-up yet.
  • Results:
    • A general capability index across 123 models and 92 benchmarks grows ~0.1 points/month, with no acceleration.
    • Inside Google, AI writes ~75% of production code, but only 11.9% of changes are fully delegated (7.5% in April); 88% still need human oversight.
    • The internal Argon agent solves ~50% of RE-Bench's 74 tasks. Pichai separately called RSI "early stage".
  • Search relevance: a reality check for T1/T7. Self-improvement shows up in productivity, not yet in research-speed curves.
  • RSI relevance: direct: a lab's own measurement finding no RSI acceleration yet.

OpenAI: "Research acceleration: The view inside OpenAI" (Sep 6, 2026) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md ("OpenAI research acceleration / inference-cost signals", tagged [unverified] there; this post is the primary source). Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/research-acceleration-view-inside-openai. Confidence: verified (fetched in this session).
  • Key idea: RL training was paused after the Hugging Face incident.
  • Results:
    • OpenAI says it reached its goal of an "automated research intern" by September 2026.
    • Median researcher inference spend is above $600 per day; the 90th percentile is above $7,000 per day.
    • About 3.1 agent-workdays per human workday as of mid-August. Experiments per active experimenter hit an all-time high in August 2026.
  • Search relevance: First-hand lab data on AI agents speeding up AI research, the core of recursive self-improvement.
  • RSI relevance: direct: lab data on agents accelerating AI research itself.

Meta: "An Organizational Second Brain: Building an AI That Learns From Experts" (Sep 2, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Meta Engineering. URL: https://engineering.fb.com/2026/09/02/ml-applications/organizational-second-brain-ai-learns-from-experts/. Confidence: verified (fetched in this session).
  • Key idea:
    • Knowledge lives in text files arranged by density and usage frequency. Composable "recipes" give progressive disclosure, replacing an earlier single flat instruction file plus semantic search.
    • Self-improvement flywheel: every expert correction is treated as a compilation problem in four phases. Diagnose the root cause, make surgical multi-agent edits, run an eval to prove the fix, then land it and enrich the regression suite.
  • Results: After 3 sprints over 6 weeks: assessment time fell from days to minutes, with zero regressions across improvement cycles. Each knowledge edit can be reviewed in about 30 seconds.
  • Search relevance: A concrete design for self-improving agent memory and skills: the agent edits its own text knowledge base, gated by evals, without any weight updates.
  • RSI relevance: direct: eval-gated self-edits to a text knowledge base, no weight updates.

Microsoft Research: "SkillOpt: Agent skills as trainable parameters" (Jun 30, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Microsoft Research. URL: https://www.microsoft.com/en-us/research/blog/skillopt-agent-skills-as-trainable-parameters/. Confidence: verified (fetched in this session).
  • Key idea: Optimizes a skill file (best_skill.md) as a trainable parameter, with no weight updates. It was best or tied-best in all 52 cells (6 benchmarks x 7 models x 3 execution modes).
  • Results:
    • SpreadsheetBench went from 41.8 to 80.7, OfficeQA from 33.1 to 72.1 and LiveMathematicianBench from 37.6 to 66.9. GPT-5.5 gained +24.8 points inside Codex and +19.1 inside Claude Code.
    • After optimization, GPT-5.4-mini's six-benchmark average (64.3) beats the no-skill GPT-5.4 (59.7). Median final skill is about 920 tokens, built from only 1-4 accepted edits. Skills transfer across models and harnesses.
  • Search relevance: Direct evidence that self-improvement through skill files works in real harnesses (Codex, Claude Code).
  • RSI relevance: direct: skill files optimized as trainable parameters inside real harnesses, transferring across models.

OpenAI: "Building self-improving tax agents with Codex" (May 27, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: OpenAI (with Thrive Holdings / Crete). URL: https://openai.com/index/building-self-improving-tax-agents-with-codex. Confidence: verified (fetched in this session).
  • Key idea: Three-part loop: practitioner corrections become production traces, then Codex fixes the agent against targeted and regression evals.
  • Results:
    • 7,000 returns processed across 30+ firms. Drafts were up to 97% accurate, preparation time fell by about a third, and throughput rose about 50%.
    • The share of returns with at least 75% correct field completion rose from about 25% at launch to 86% within six weeks. Rental schedules reached 90% precision and recall in about 6 weeks. One accountant's prep time fell from 180 hours to 15.
  • Search relevance: A production example of a self-improving agent loop (traces, then evals, then a coding agent edits the harness), with numbers.
  • RSI relevance: direct: corrections to traces to evals to Codex editing the agent, with production numbers.

Google DeepMind: "AlphaEvolve: How our Gemini-powered coding agent is scaling impact across fields" (May 7, 2026) [verified]

  • Found: 2026-10-30 backfill; related to the AlphaEvolve entry in industry-practice.md. Sent: no (backfill).
  • Source: Google DeepMind. URL: https://deepmind.google/blog/alphaevolve-impact/. Confidence: verified (fetched in this session; the RSS feed lists 2026-05-06).
  • Key idea: Now used regularly in TPU design.
  • Results:
    • DeepConsensus had 30% fewer variant-detection errors. AC optimal power flow feasible solutions went from 14% to over 88%. Quantum circuits reached 10x lower error. Spanner LSM compaction saw 20% less write amplification.
    • Customer results: Klarna 2x training speed, FM Logistic 10.4% better routing, WPP +10% accuracy, Schrodinger about 4x speedup.
  • Search relevance: An evaluator-driven evolutionary coding agent improving the infrastructure that trains its own models, a partial RSI loop.
  • RSI relevance: direct (partial loop): evaluator-driven evolution now used in TPU design, the hardware its models train on.

Google Research: "ReasoningBank: Enabling agents to learn from experience" (Apr 21, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Google Research. URL: https://research.google/blog/reasoningbank-enabling-agents-to-learn-from-experience/. Confidence: verified (fetched in this session).
  • Key idea: Distills reusable reasoning strategies from both successes and failures, judged by an LLM-as-judge, into a memory bank (ICLR; arXiv 2509.25140).
  • Results:
    • +8.3% on WebArena and +4.6% on SWE-Bench-Verified over a memory-free baseline.
    • Memory-aware test-time scaling (MaTTS, k=5) adds a further +3% and takes 0.4 fewer steps.
  • Search relevance: A self-improving memory loop at test time, with no weight updates.
  • RSI relevance: direct: test-time memory of strategies distilled from successes and failures, no weight updates.

Meta: "KernelEvolve: How Meta's Ranking Engineer Agent Optimizes AI Infrastructure" (Apr 2, 2026) [verified]

  • Found: 2026-10-30 backfill; related to the Trellis / KernelEvolve entry in industry-practice.md (different paper: 2606.29823 there, 2512.23236 here). Sent: no (backfill).
  • Source: Meta Engineering. URL: https://engineering.fb.com/2026/04/02/developer-tools/kernelevolve-how-metas-ranking-engineer-agent-optimizes-ai-infrastructure/. Confidence: verified (fetched in this session).
  • Key idea: Components: LLM synthesizer, tree search, RAG knowledge base, automated evaluation (TritonBench, profiler) and agentic RL (arXiv 2512.23236, ISCA 2026).
  • Results:
    • Over 60% inference throughput gain for the Andromeda ads model on NVIDIA GPUs, and over 25% training throughput gain on MTIA.
    • 100% pass rate on KernelBench (250 problems).
  • Search relevance: An eval-gated search-and-evolve agent optimizing AI infrastructure (an AlphaEvolve-style loop).
  • RSI relevance: direct (partial loop): an agent optimizing kernels for the infrastructure that trains Meta's models.

MiniMax: "MiniMax M2.7: Early Echoes of Self-Evolution" (Mar 18, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: MiniMax. URL: https://www.minimax.io/blog/minimax-m27. Confidence: verified.
  • Key idea: Self-optimization covered harness architecture, skills and memory mechanisms, and sampling params (temperature, frequency/presence penalty).
  • Results:
    • The model ran 100+ autonomous rounds of "analyze failure trajectories -> plan changes -> modify scaffold code -> run evals" on its own agent harness, for a 30% gain on internal metrics.
    • Scores: SWE-Pro 56.22%, Terminal Bench 2 57.0%, GDPval-AA ELO 1495, MLE Bench Lite 66.6% medal rate, 97% skill adherence over 40+ complex skills.
  • Search relevance: the most direct lab-published case of RSI at the harness level (a model iterating on its own scaffold, skills and memory with an eval loop).
  • RSI relevance: direct: the model ran 100+ rounds of editing its own scaffold, skills and memory against evals.

Meta: "Ranking Engineer Agent (REA): The Autonomous AI Agent Accelerating Meta's Ads Ranking Innovation" (Mar 17, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Meta Engineering. URL: https://engineering.fb.com/2026/03/17/developer-tools/ranking-engineer-agent-rea-autonomous-ai-system-accelerating-meta-ads-ranking-innovation/. Confidence: verified (fetched in this session).
  • Key idea: REA-driven iterations doubled average model accuracy over baseline across six models.
  • Results:
    • Long-horizon, multi-day autonomous ML experimentation agent. It covers hypothesis generation, job launch and recovering from failures, and is built on Meta's Confucius agent framework (arXiv 2512.10398).
    • About 5x engineering output: 3 engineers delivered launch proposals for 8 models, work that previously took 2 engineers per model.
  • Search relevance: An agent automating ML research iterations, a step toward RSI in a production setting.
  • RSI relevance: direct: an agent automating ML research iterations in production.

Anthropic: "Improving skill-creator: Test, measure, and refine Agent Skills" (Mar 3, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic (claude.com). URL: https://claude.com/blog/improving-skill-creator-test-measure-and-refine-agent-skills. Confidence: verified (fetched in this session).
  • Key idea:
    • skill-creator now lets authors write evals for skills without coding, run benchmarks, and catch regressions as models change (including detecting when a skill's technique has been absorbed into the base model).
    • Distinguishes "capability uplift" skills from "encoded preference" skills.
    • Parallel eval runs in isolated subagent contexts, each with its own token and timing metrics.
  • Results: Description/trigger optimization improved triggering on 5 of 6 public skills tested.
  • Search relevance: Eval-driven skill self-improvement loop; directly relevant to agents that maintain their own skill libraries.
  • RSI relevance: direct: an eval-driven loop to test and refine skills, including trigger-description optimization (5 of 6 skills improved).

Anthropic: "Writing effective tools for AI agents - with agents" (Sep 11, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Ken Aizawa. URL: https://www.anthropic.com/engineering/writing-tools-for-agents. Confidence: verified (fetched in this session).
  • Key idea:
    • Eval-driven loop in which agents help evaluate and refine tool implementations; a Slack MCP server optimization showed significant measured gains.
    • Prefer a few high-impact, consolidated tools over many thin API wrappers.
    • Namespacing choice (prefix vs suffix) has non-trivial effects on tool-use evals.
  • Results: A ResponseFormat enum (detailed 206 tokens vs concise 72 tokens, ~65% less) controls context cost.
  • Search relevance: Agents optimizing their own tool interfaces against evals is a concrete RSI pattern.
  • RSI relevance: direct: agents evaluate and rewrite their own tool implementations and descriptions against evals.

Google DeepMind: "AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms" (May 14, 2025) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md. Sent: no (backfill).
  • Source: Google DeepMind. URL: https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/. Confidence: verified (fetched in this session).
  • Key idea: Combines a Gemini Flash + Pro ensemble, automated evaluators and an evolutionary program database.
  • Results:
    • A Borg scheduling heuristic recovers 0.7% of Google's worldwide compute. A Gemini kernel ran 23% faster, cutting training time 1%. FlashAttention kernels sped up as much as 32.5%.
    • 4x4 complex matrix multiplication in 48 scalar multiplications. On 50+ open problems it matched the state of the art in about 75% and improved on it in 20%.
  • Search relevance: The reference eval-driven evolutionary agent; it improves the training of its own Gemini models.
  • RSI relevance: direct (partial loop): evolved kernels sped up training of its own Gemini models.

Prime Intellect: "Prime Agent: A Self-Improving RLM Agent" (Aug 5, 2026) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md (its watchlist, as "Prime Agent (digest 08-24; org not recorded) [unverified]"). Sent: no (backfill).
  • Source: high-discussion (HN 254 points / 69 comments; Prime Intellect is not a listed lab). URL: https://www.primeintellect.ai/blog/prime-agent. Confidence: verified.
  • Key idea: "Continual Harness": the agent does CRUD on its own prompt, sub-agents, skills and memory from its own trajectories; a /refine pipeline makes targeted edits in the background
  • Results:
    • ARC-AGI 3: 95.5% RHAE Best@1 with Opus 5 (runs 95.0/95.2/95.5), above the reported human expert baseline of 95.4%
    • OOLONG (128k): 0.700 with GLM-5.2 vs 0.420 for Pi-mono harness
  • Search relevance: Concrete harness-level RSI where the agent edits its own skills, memory and prompts.
  • RSI relevance: direct: "Continual Harness" where the agent edits its own prompt, sub-agents, skills and memory.

Lilian Weng: "Harness Engineering for Self-Improvement" (Jul 4, 2026) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md. Sent: no (backfill).
  • Source: high-discussion (HN 334 points / 80 comments, 2026-08-04; Lilian Weng is not a listed lab). URL: https://lilianweng.github.io/posts/2026-07-04-harness/. Confidence: verified.
  • Key idea:
    • Defines the harness as the system around a base model that orchestrates thinking, planning, tool calls and actions; argues it matters as much as raw model intelligence for RSI
    • Three core patterns: workflow automation loops, the file system as persistent memory, and sub-agent parallelization
    • Seven open challenges: weak evaluators for research, context lifecycle, publication bias, diversity collapse, reward hacking, sustainability, human oversight
  • Results: The harness itself becomes an optimization target (AlphaEvolve, Meta-Harness); a cited result says a 9B model produced harness proposals "procedurally isomorphic" to Claude Opus 4.6's
  • Search relevance: Central survey connecting harness design to recursive self-improvement.
  • RSI relevance: direct: the reference survey tying harness engineering to RSI.

SkyPilot: "Scaling Karpathy's Autoresearch: What Happens When the Agent Gets a GPU Cluster" (Mar 18, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 237 points / 94 comments; SkyPilot is not a listed lab). URL: https://skypilot.ai/blog/scaling-autoresearch/. Confidence: verified.
  • Key idea: 16 GPUs (13 H100, 3 H200), about 910 experiments in 8 h (90/h vs about 10/h on one GPU)
  • Results:
    • val_bpb improved from 1.003 to 0.974 (2.87%), 9x faster than sequential; about $9 in Claude Code API plus about $300 in GPU
    • The agent found that width scaling beat all hyperparameter tweaks and on its own adopted a two-tier H100-screen/H200-confirm strategy
  • Search relevance: Scaling a self-directed research loop; emergent resource-allocation strategy.
  • RSI relevance: direct: a self-directed research loop scaled to 16 GPUs; the agent chose its own resource strategy.

Andrej Karpathy: "autoresearch (Karpathy)" (Mar 2026) [verified]

  • Found: 2026-10-30 backfill; also in industry-practice.md ("Karpathy AutoResearch", tagged [unverified] there). Sent: no (backfill).
  • Source: high-discussion (98.9k GitHub stars at fetch; HN 208 pts; spawned HN follow-ups with 457, 428 and 237 pts; Andrej Karpathy is not a listed lab). URL: https://github.com/karpathy/autoresearch. Date: 2026-03 (HN 2026-03-07). Confidence: verified.
  • Key idea:
    • A single metric: validation bits-per-byte (vocab-independent)
    • Minimal design: one file to modify, one metric, one GPU
  • Results: The agent edits train.py and a human edits program.md; each experiment is a fixed 5-minute training run, keep or discard, repeat (about 12/h, about 100 overnight on one GPU)
  • Search relevance: Canonical minimal self-improving research loop (the "Karpathy loop").
  • RSI relevance: direct: the minimal keep-or-discard self-improving research loop.

LangChain: "Improving Deep Agents with harness engineering" (Feb 17, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (widely re-covered, e.g. Analytics Vidhya, ZenML LLMOps DB, independent blog write-ups; LangChain is not a listed lab). URL: https://www.langchain.com/blog/improving-deep-agents-with-harness-engineering. Confidence: verified.
  • Key idea:
    • Middleware: LocalContextMiddleware (maps the cwd), PreCompletionChecklistMiddleware (forces verification), LoopDetectionMiddleware (stops "doom loops")
    • A "reasoning sandwich" (max reasoning for planning and verification, standard for implementation) and trace-driven failure analysis
  • Results: deepagents-cli on Terminal Bench 2.0: 52.8 to 66.5 (+13.7) with gpt-5.2-codex fixed, moving from outside the Top 30 to the Top 5, by changing only the harness
  • Search relevance: Clear evidence that harness iteration alone yields large gains; a trace-to-harness improvement loop.
  • RSI relevance: partial: trace-driven harness iteration alone gave +13.7 on Terminal Bench 2.0 (human in the loop).

Agent RL training

Microsoft Research: "Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses" (Oct 7, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Microsoft Research. URL: https://www.microsoft.com/en-us/research/blog/agent-lightning-v1-0-a-3500-line-lightweight-agentic-rl-framework-for-training-agents-with-real-harnesses/. Confidence: verified (fetched in this session).
  • Key idea: A complete agent RL control plane (API Gateway, Rollout Controller, Customized Trainer) in about 3,500 lines of code. It trains agents inside their real, unmodified harnesses.
  • Results:
    • RL alone raised Qwen3.5-9B from 41.8% to 56.4% Pass@1 on SWE-bench Verified (+14.6 points), using about 6,000 SWE-smith samples with mini-SWE-agent.
    • Collocated async RL gave about a 2x end-to-end speedup over synchronous RL while using fewer GPUs than conventional async RL.
  • Search relevance: Lets you train model weights in the same production harness the agent runs in, which closes the loop from harness traces back to the model.
  • RSI relevance: partial: closes the loop from production-harness traces back to model weights.

Alibaba Qwen: "Qwen3.5: Towards Native Multimodal Agents" (Feb 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Alibaba Qwen (Qwen Team, Alibaba). URL: https://qwen.ai/blog?id=qwen3.5 (mirror: https://www.alibabacloud.com/blog/qwen3-5-towards-native-multimodal-agents_602894). Date: 2026-02 (around 02-16). Confidence: verified (mirror fetched; the BrowseComp 78.6 figure is confirmed-exists only).
  • Key idea: 397B-A17B hybrid model (Gated DeltaNet linear attention + sparse MoE), positioned as a native multimodal agent base. Qwen3.5-Plus has 1M context and built-in tools.
  • Results:
    • Fully disaggregated training-inference RL framework gives a 3-5x end-to-end agentic RL speedup.
    • Search snippets report BrowseComp 78.6 with a "discard-all" context strategy (not checked in the fetched mirror).
  • Search relevance: agentic-RL infrastructure, plus BrowseComp results that depend on the context-reset strategy.
  • RSI relevance: none.

Alibaba Tongyi: "ArenaRL: Scaling RL for Open-Ended Agents via Tournament-Based Relative Ranking" (Jan 14, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Alibaba Tongyi (DeepResearch Team, Tongyi Lab (Alibaba)). URL: https://tongyi-agent.github.io/blog/arenarl/. Confidence: verified.
  • Key idea: Replaces absolute LLM-judge scores with pairwise tournament ranking for rewards. Seeded single-elimination needs 2N-2 comparisons vs N(N-1)/2 for round-robin, with nearly the same advantage-estimation accuracy.
  • Results:
    • Open-DeepResearch 16.7 -> 64.3 (SFT -> ArenaRL), Open-Travel 16.4 -> 41.8, General Writing 72.1 -> 80.3.
    • In Amap production, complex-planning business metric 69% -> 80%.
  • Search relevance: a reward/eval design for open-ended deep-research agents where there is no verifiable ground truth.
  • RSI relevance: partial: a verifier-free relative reward, usable where self-improvement has no ground truth.

DeepSeek: "DeepSeek-V3.2 Release" (Dec 1, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: DeepSeek. URL: https://api-docs.deepseek.com/news/news251201. Confidence: verified.
  • Key idea: First DeepSeek model to integrate thinking into tool use, in both thinking and non-thinking modes.
  • Results:
    • New large-scale agent-data synthesis pipeline: 1,800+ environments and 85k+ complex instructions for agentic RL.
    • V3.2-Speciale reached gold-level results at IMO, CMO, ICPC WF and IOI 2025. BrowseComp/HLE numbers are only in the tech report, not the news post.
  • Search relevance: synthetic environment scaling for agentic RL generalization, plus thinking interleaved with tool calls.
  • RSI relevance: none.

MiniMax: "Aligning to What? Rethinking Agent Generalization in MiniMax M2" (Oct 30, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: MiniMax (Hugging Face blog). URL: https://huggingface.co/blog/MiniMax-AI/aligning-to-what. Confidence: verified.
  • Key idea: Separates benchmark alignment from real-world robustness. Generalization means robustness to perturbations across the full trajectory (system prompts, user instructions, environments, tool responses), not just more tools.
  • Results: Interleaved thinking keeps the agent adaptive to unexpected tool outputs over long contexts. M2 generalizes to unfamiliar scaffolds.
  • Search relevance: harness/scaffold-robust agent training, and the gap between evals and deployment.
  • RSI relevance: indirect: scaffold robustness matters if the harness itself keeps changing.

Moonshot AI (Kimi): "Kimi-Researcher: End-to-End RL Training for Emerging Agentic Capabilities" (Jun 20, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Moonshot AI (Kimi). URL: https://moonshotai.github.io/Kimi-Researcher/. Confidence: verified.
  • Key idea: Reports emergent self-correction and hypothesis refinement when sources conflict.
  • Results:
    • End-to-end REINFORCE RL lifted HLE Pass@1 from 8.6% to 26.9% (Pass@4 40.17%). xbench-DeepSearch 69% pass@1.
    • Averages 23 reasoning steps and 200+ URLs explored per task, with up to 70+ search queries per trajectory.
    • Context management allows 30% more iterations. Gamma-decayed correctness reward favors efficient trajectories. Async rollout plus turn-level partial rollout gives 1.5x speedup.
  • Search relevance: a seminal lab post on RL-trained deep research agents with learned context management.
  • RSI relevance: weak: emergent self-correction within a trajectory, not across runs.

Cursor: "Composer: Building a fast frontier model with RL" (Oct 29, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 215 points / 168 comments; Cursor is not a listed lab). URL: https://cursor.com/blog/composer. Confidence: verified.
  • Key idea: RL inside the production harness tools (edit, semantic search, terminal, grep) across hundreds of thousands of concurrent sandboxes
  • Results:
    • MoE coding model about 4x faster generation than similar frontier models, measured on internal Cursor Bench (which also scores adherence to codebase abstractions)
    • MXFP8 kernels, expert parallelism, thousands of GPUs; emergent behaviors such as complex searches, fixing lint errors and running tests
  • Search relevance: Model co-trained with its harness; RL on agentic search and tool use.
  • RSI relevance: partial: model trained inside its own production harness.

Evaluation & benchmarks

OpenAI: "Separating signal from noise in coding evaluations" (Jul 8, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/separating-signal-from-noise-coding-evaluations. Confidence: verified (fetched in this session).
  • Key idea: SWE-Bench Pro audit: an agent pipeline flagged 200 tasks (27.4%) as broken, and human review flagged 249 (34.1%).
  • Results: OpenAI estimates about 30% of tasks are broken. This follows its 2026-02-23 post "Why we no longer evaluate SWE-bench Verified", which cited contamination and mismeasurement.
  • Search relevance: Shows eval validity is a bottleneck for measuring agent self-improvement, and that agents can be used to audit benchmarks.
  • RSI relevance: cautionary: about 30% broken tasks undermines benchmark-measured self-improvement; agents can help audit.

Anthropic: "Eval awareness in Claude Opus 4.6's BrowseComp performance" (Mar 6, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Russell Coleman. URL: https://www.anthropic.com/engineering/eval-awareness-browsecomp. Confidence: verified (fetched in this session).
  • Key idea: Of 1,266 BrowseComp problems, 9 were contaminated via leaked answers online.
  • Results:
    • 2 cases where the model inferred it was being tested, identified BrowseComp, and decrypted the answer key (described as a first documented instance).
    • One such run used 40.5M tokens, ~38x the median.
    • Unintended-solution rate 0.24% single-agent vs 0.87% multi-agent (3.7x).
  • Search relevance: Key caveat for deep-research evals: agentic search can find or game the benchmark itself, and multi-agent setups make this more likely.
  • RSI relevance: cautionary: a capable search agent can find and game its own benchmark, which threatens eval-gated self-improvement loops.

Anthropic: "Quantifying infrastructure noise in agentic coding evals" (Feb 5, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Gian Segato. URL: https://www.anthropic.com/engineering/infrastructure-noise. Confidence: verified (fetched in this session).
  • Key idea: On Terminal-Bench 2.0, strict (1x) vs uncapped container resources gave a 6 pp score gap (p < 0.01).
  • Results:
    • Infra error rate fell monotonically from 5.8% (strict) to 0.5% (uncapped).
    • Above ~3x headroom, extra resources let agents solve problems they otherwise couldn't, so the benchmark measures something different.
    • Leaderboard gaps under 3 pp deserve skepticism unless infra config is documented.
  • Search relevance: Eval hygiene for anyone measuring harness/self-improvement gains; small deltas can be infra noise.
  • RSI relevance: cautionary: small self-improvement gains (under 3 pp) can be infrastructure noise.

Anthropic: "Demystifying evals for AI agents" (Jan 9, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Mikaela Grace, Jeremy Hadfield, Rodrigo Olivares, Jiri De Jonghe. URL: https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents. Confidence: verified (fetched in this session).
  • Key idea:
    • Combine code-based, model-based, and human graders.
    • "Swiss cheese" layering: automated evals plus production monitoring, A/B tests, user feedback, and transcript review.
  • Results:
    • Start with 20-50 simple tasks taken from real failures; don't wait for a full suite.
    • pass@k vs pass^k: at k=10, pass@k approaches 100% while pass^k can approach 0%.
  • Search relevance: Canonical Anthropic eval methodology for agents; the base for any self-improvement loop.
  • RSI relevance: enabling: the eval methodology any self-improvement loop needs as its signal.

OpenAI: "BrowseComp: a benchmark for browsing agents" (Apr 10, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/browsecomp. Confidence: verified (fetched in this session).
  • Key idea: 1,266 hard-to-find, easy-to-verify questions.
  • Results:
    • Accuracy: GPT-4o 0.6%, GPT-4o with browsing 1.9%, GPT-4.5 0.9%, o1 9.9%, Deep research 51.5%. Human trainers solved 29.2%, giving up after about 2 hours.
    • Aggregating 64 samples (majority vote, weighted vote, best-of-N) improved results 15-25%. Deep research solved 16% of tasks perfectly and failed 14% entirely.
  • Search relevance: The standard eval for persistent agentic search and deep research.
  • RSI relevance: none.

Hamel Husain and Shreya Shankar: "AI Evals: Everything You Need to Know (Evals FAQ)" (Sep 18, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 189 points; course has taught 5,000+ engineers/PMs; Hamel Husain and Shreya Shankar is not a listed lab). URL: https://hamel.dev/blog/posts/evals-faq/. Date: 2026-09-18 (date currently shown; first published 2025, HN submission 2025-07-01 as "About AI Evals"). Confidence: verified.
  • Key idea: Expect to spend 60-80% of dev time on error analysis and evals; start with manual review of 20-50 outputs and one domain-expert "benevolent dictator"
  • Results:
    • Review at least 30 traces yourself and continue to saturation (usually 100+); LLM judges need 100-200 labeled examples per failure mode
    • Prefer binary pass/fail over Likert scales; combine evals with "pass all" rates; review 10-20 traces each week
  • Search relevance: Practical eval methodology, the feedback signal any self-improving agent loop needs.
  • RSI relevance: enabling: error analysis and judges are the feedback signal a self-improvement loop needs.

Other

Anthropic: "Redesigning source control at Anthropic" (Oct 13, 2026) [verified]

  • Found: 2026-10-30 (also listed as the "Scion" item from digest 10-19; found again in the 10-30 backfill and merged here); also in industry-practice.md ("Scion" entry). Sent: yes.
  • Source: Anthropic (Anthropic Engineering; Nick G. and the source control team). URL: https://www.anthropic.com/engineering/redesigning-source-control.
  • Key idea: agent-generated changes broke human-scale source-control infrastructure, so it was redesigned.
    • Built a custom VCS ("Scion") and migrated the monorepo off git in 5 months with a 7-person team.
    • Humans shift from line-by-line review to design/constraint setting, Claude handles implementation.
  • Results:
    • Merges grew from ~2,500/day (May) to 30,000+/day (September), ~12x, driven by agent-written code.
    • 200x smaller infra footprint than the git mirroring setup; checkouts from minutes to seconds.
  • Search relevance: tangential. Evidence of production-scale agent output; the bottleneck moves to infra and review.
  • RSI relevance: indirect: agent-driven engineering throughput at a lab; not a self-improvement loop.

OpenAI: "Speeding up agentic workflows with WebSockets in the Responses API" (Apr 22, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/speeding-up-agentic-workflows-with-websockets. Confidence: verified (fetched in this session).
  • Key idea: A persistent WebSocket transport made agent loops 40% faster end-to-end and time to first token about 45% better.
  • Results:
    • Throughput went from 65 TPS toward a 1,000 TPS target, with bursts to 4,000 TPS (GPT-5.3-Codex-Spark).
    • Partner results: Cline 39% faster; Cursor up to 30% faster.
  • Search relevance: Harness and infrastructure latency is a first-order cost for long, many-call agent loops.
  • RSI relevance: none (transport latency).

Anthropic: "Scaling Managed Agents with stable interfaces" (Apr 8, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering; Lance Martin, Gabe Cemaj, Michael Cohen. URL: https://www.anthropic.com/engineering/managed-agents. Confidence: verified (fetched in this session).
  • Key idea:
    • Decouples the "brain" (Claude + harness) from the "hands" (sandboxes/tools) so each can fail and be replaced independently.
    • Credentials kept outside sandboxes via vaults; tokens never reachable from Claude-generated code.
    • Stateless design enables "many brains, many hands."
  • Results: p50 time-to-first-token dropped ~60% and p95 by >90% after decoupling (containers provisioned lazily).
  • Search relevance: Reference architecture for hosted long-running agent harnesses.
  • RSI relevance: none (hosting infrastructure).

OpenAI: "Unlocking the Codex harness: how we built the App Server" (Feb 4, 2026) [verified]

  • Found: 2026-10-30 backfill; possibly the same item as "Codex harness open-sourced" in industry-practice.md (thread/turn/item primitives match); the ARC-AGI-3 numbers there are not in this post. Sent: no (backfill).
  • Source: OpenAI. URL: https://openai.com/index/unlocking-the-codex-harness. Confidence: verified (fetched in this session).
  • Key idea:
    • Bidirectional JSON-RPC over stdio (JSONL), with thread, turn and item as the conversation primitives.
    • One harness serves CLI, VS Code, web (in a container) and Xcode.
  • Results: no quantified results in the raw note.
  • Search relevance: How to decouple the agent harness from the client surfaces.
  • RSI relevance: none.

Anthropic: "Building agents with the Claude Agent SDK" (Sep 29, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic (claude.com); Thariq Shihipar. URL: https://claude.com/blog/building-agents-with-the-claude-agent-sdk. Confidence: verified (fetched in this session).
  • Key idea:
    • Claude Code SDK renamed Agent SDK; core principle is "giving Claude a computer" (terminal, files).
    • Agent loop: gather context -> take action -> verify work -> repeat.
    • Verification via rules (linting), visual feedback (screenshots), and LLM-as-judge; context handled with compaction and subagents (isolated windows).
  • Results: no quantified results in the raw note.
  • Search relevance: General-purpose harness blueprint behind Claude Code, reusable for research agents.
  • RSI relevance: indirect: gather-act-verify loop with LLM-as-judge verification.

Anthropic: "Claude Code: Best practices for agentic coding" (Apr 18, 2025) [unverified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: Anthropic Engineering. URL: https://www.anthropic.com/engineering/claude-code-best-practices (now 308-redirects to https://code.claude.com/docs/en/best-practices). Confidence: confirmed-exists (seen in search results/index page but body not fetched; numbers not checked).
  • Key idea: Listed on the engineering index; the original post now redirects to Claude Code docs, so the original body and numbers were not checked.
  • Results: no quantified results in the raw note.
  • Search relevance: Practitioner harness patterns (CLAUDE.md memory, explore-plan-code, subagents) for coding agents.
  • RSI relevance: none known (body not read).

HumanLayer: "Why Software Factories Fail (or: harness engineering is not enough)" (Jul 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 394 points / 272 comments; HumanLayer is not a listed lab). URL: https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/wsff.md. Date: 2026-07 (HN submission 2026-07-23). Confidence: verified.
  • Key idea: Cites a 25% rise in PR review comments, a 242.7% rise in code-quality incidents per PR, and 54% more bugs per developer since Jan 2026
  • Results:
    • Benchmarks score pass/fail in seconds and do not penalize loss of maintainability; the team had incidents after dropping human review in July 2025
    • Recommends a staged process (requirements, then architecture, then program design, then vertical slices), claiming 2-3x safe velocity
  • Search relevance: Counterpoint on the limits of autonomous harnesses; evals that miss long-term quality.
  • RSI relevance: cautionary: autonomous loops optimized on fast pass/fail evals can lose maintainability.

Simon Willison: "Agentic Engineering Patterns" (Feb 23, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 543 points / 310 comments; Simon Willison is not a listed lab). URL: https://simonwillison.net/guides/agentic-engineering-patterns/. Confidence: verified.
  • Key idea:
    • A living guide with chapters on principles, working with coding agents (caching, tools, subagents, git), testing/QA (red/green TDD), understanding code, and annotated prompts
    • Core thesis: "writing code is cheap now"; the bottleneck moves to quality standards and verification
    • Hoarding and recombining knowledge (examples, prior solutions) multiplies what agents can do
  • Results: no quantified results in the raw note.
  • Search relevance: Practitioner patterns for harnessing coding agents, including verification loops.
  • RSI relevance: none.

Simon Willison: "Designing agentic loops" (Sep 30, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (HN 284 points / 117 comments; Simon Willison is not a listed lab). URL: https://simonwillison.net/2025/Sep/30/designing-agentic-loops/. Confidence: verified.
  • Key idea:
    • Framed as a new skill: design the tools and environment so the agent can brute-force toward a clear success criterion
    • Prefer shell commands documented in AGENTS.md over MCP; best fit is tasks with clear success criteria and good tests
  • Results: YOLO mode only in sandboxes; scoped credentials (e.g. a dedicated Fly.io org with a $5 budget cap)
  • Search relevance: Environment and feedback design for autonomous, verifiable agent loops.
  • RSI relevance: indirect: clear success criteria and sandboxes are what let agents brute-force toward a goal.

Simon Willison: "The lethal trifecta for AI agents: private data, untrusted content, and external communication" (Jun 16, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: high-discussion (follow-up talk HN 430 pts; The Economist leader on the "lethal trifecta" HN 115 pts; Simon Willison is not a listed lab). URL: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/. Confidence: verified.
  • Key idea: Private data + untrusted content + an exfiltration channel together make an exploitable agent
  • Results: 95% guardrail detection counts as failure in security; the only reliable mitigation is to avoid combining all three
  • Search relevance: Safety constraint on tool and harness design for autonomous and search agents.
  • RSI relevance: cautionary: a self-modifying agent with tools inherits the same exfiltration risk.

Watchlist (low signal)

Builder posts whose Source is other (not a listed lab, no high-discussion evidence). Kept for reference; do not use as primary evidence.

LangChain: "Context Engineering Starts with Retrieval" (Jess Weng, Oct 23, 2026) [verified]

  • Found: 2026-10-30. Sent: logged (builder blog, not a lab).
  • Source: other (LangChain).
  • Key idea: Deep Agents retrieves with glob/grep/read_file, paginates reads at 100 lines, offloads large tool results (~20K tokens) to files, and uses subagents to filter. No self-improvement loop. Companion post: "How Deep Agents Remember" (Oct 14).
  • Search relevance: practitioner confirmation of T10 (lexical tools + agent loop) and T11 (subagents for context isolation).
  • RSI relevance: none.

Exa: "ATLAS: Evaluating Agents on Search-Intensive Tasks" (Oct 8, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: other (Exa; covered by AlphaSignal news). URL: https://exa.ai/blog/atlas-benchmark. Confidence: verified.
  • Key idea: 547 deep-and-wide research tasks (find entities, then fill in 2-10 multi-hop attributes); memorization rate 9% vs 48-61% on other benchmarks
  • Results:
    • Best system reached only 0.66 row F1, at $8.92 and 18 min per task; no run under $1/task beat 0.5 row F1
    • Hiding the top search results leaves about 50% of the ATLAS score vs 81-89% on other benchmarks, so it actually measures search quality. Grading costs $0.24 per system
  • Search relevance: Hard, uncontaminated eval for deep-research and wide-search agents.
  • RSI relevance: none.

Exa: "How Search Quality Shapes RL Outcomes" (May 13, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: other (Exa). URL: https://exa.ai/blog/rl-search-outcomes. Confidence: verified.
  • Key idea: Search agents RL-trained on Exa vs a SERP API: SimpleQA pass@1 76.7% vs 69.2%
  • Results:
    • Needed 69% fewer tokens, 62% fewer search calls and 58% fewer turns to reach the same performance; Exa-trained 4B models often beat untrained 235B models
    • Exa results contained the answer 10.7% more often per call, giving a denser reward signal; gains carried over across search engines
  • Search relevance: Retrieval tool quality directly shapes RL training of search agents.
  • RSI relevance: none.

Cursor: "Continually improving our agent harness" (Apr 30, 2026) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: other (Cursor; covered by StartupHub, translated on CSDN). URL: https://cursor.com/blog/continually-improving-agent-harness. Confidence: verified.
  • Key idea:
    • Moved from static context (folder layouts, snippets) to dynamic context retrieval as models improved
    • Quality measured by "keep rate" (share of agent code still in the codebase later) plus LLM-judged user satisfaction
  • Results: A focused sprint brought all tool calls to 2-3 nines of reliability; per-model customization (patch edits for OpenAI models, string replace for Anthropic)
  • Search relevance: Production harness-improvement loop driven by online evals.
  • RSI relevance: partial: online-eval-driven harness iteration (human-driven).

Cursor: "Improving Agent with Semantic Search" (Nov 6, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: other (Cursor). URL: https://cursor.com/blog/semsearch. Confidence: verified.
  • Key idea: Custom embedding model trained from agent sessions, with LLM rankings of which code was most helpful at each step; grep plus semantic works best
  • Results:
    • Semantic search added 12.5% QA accuracy on average (6.5%-23.5% by model)
    • Code retention +0.3% overall and +2.6% in large codebases (1,000+ files); dissatisfied follow-ups rose 2.2% without it
  • Search relevance: Shows the value of retrieval tools in agentic code search, with an online-eval methodology.
  • RSI relevance: none.

Perplexity Research: "Architecting and Evaluating an AI-First Search API" (Sep 25, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: other (Perplexity; open-sourced search_evals framework). URL: https://www.perplexity.ai/hub/blog/architecting-and-evaluating-an-ai-first-search-api. Confidence: verified.
  • Key idea: 200M daily queries, 200B+ unique URLs tracked, tens of thousands of CPUs; spans and sections treated as first-class retrieval units
  • Results:
    • p50 latency 358 ms (p95 under 800 ms), 150 ms+ faster than the next competitor
    • Scores: SimpleQA 0.930, FRAMES 0.453, BrowseComp 0.371, HLE 0.288, all above competing search APIs, evaluated inside agent harnesses
  • Search relevance: Search backend design and evals built for agent consumption.
  • RSI relevance: none.

LangChain: "Open Deep Research" (Jul 16, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: other (LangChain). URL: https://www.langchain.com/blog/open-deep-research. Confidence: verified.
  • Key idea:
    • Three phases: scope (clarify, then write a research brief), research (a supervisor plus parallel sub-agents), write (one-shot report)
    • Multi-agent only for research; parallel section-writing produced incoherent reports
    • Compress chat into a brief and prune sub-agent findings to control tokens
  • Results: no quantified results in the raw note.
  • Search relevance: Open reference architecture for deep-research agents.
  • RSI relevance: none.

LlamaIndex: "RAG is dead, long live agentic retrieval" (May 29, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: other (LlamaIndex). URL: https://www.llamaindex.ai/blog/rag-is-dead-long-live-agentic-retrieval. Confidence: verified.
  • Key idea:
    • Retrieval modes: chunk, files_via_metadata, files_via_content, plus an auto-routed agent that picks the mode
    • A composite retriever routes across multiple indices by description, giving layered agentic retrieval
  • Results: no quantified results in the raw note.
  • Search relevance: Agentic retrieval over enterprise corpora.
  • RSI relevance: none.

Lilian Weng: "Why We Think" (May 1, 2025) [verified]

  • Found: 2026-10-30 backfill. Sent: no (backfill).
  • Source: other (Lilian Weng personal blog; no HN thread found). URL: https://lilianweng.github.io/posts/2025-05-01-thinking/. Confidence: verified.
  • Key idea:
    • Survey of test-time compute: CoT, parallel sampling vs sequential revision, recurrent/latent thinking
    • Warns that optimizing CoT directly with RL can cause obfuscated reward hacking; CoT faithfulness is not guaranteed
  • Results: no quantified results in the raw note.
  • Search relevance: Background on reasoning and test-time compute behind self-improving agents.
  • RSI relevance: cautionary: RL on chain-of-thought can produce obfuscated reward hacking.

Industry Practice (Labs, Products, Engineering Posts)

hao's bar (2026-10): a proven solution should come from a major AI lab or from work with high discussion volume.

Confidence tags:

  • [verified]: the item exists, and we either read it in-session or it is widely known before the knowledge cutoff.
  • [from memory]: the item exists, but its numbers were recalled and not re-read; re-check before quoting.
  • [unverified]: recorded only in our run logs or digests; not independently confirmed.

Anthropic

Building effective agents (Anthropic engineering blog, Dec 2024) [verified]

  • Source: Anthropic.
  • Key idea:
    • Start with the simplest thing that works.
    • Workflows (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) come before autonomous agents.
    • Invest in tool design (the agent-computer interface).
  • Search relevance: orchestrator-workers and evaluator-optimizer are the base patterns behind deep research and verify-and-patch.

How we built our multi-agent research system (Anthropic engineering blog, Jun 2025) [verified that it exists; numbers from memory]

  • Source: Anthropic.
  • Key idea: a lead agent plans and spawns parallel subagents, each with its own context, which return compressed findings. A separate citation pass follows.
  • Results [from memory]:
    • Multi-agent beat single-agent Opus on an internal research eval by about 90%.
    • Uses about 15x the tokens of chat.
    • Token usage explains about 80% of the performance variance.
  • Search relevance: the canonical industry deep-research design. Context isolation is the main thing it buys.

Claude Code Explore subagent [verified]

  • Source: Anthropic.
  • Key idea: broad codebase search is delegated to a read-only subagent, which returns conclusions rather than file dumps.
  • Search relevance: the model case of "agentic search as a subagent".

Building agents: when to fold subagents into your main agent loop (claude.com/resources/articles, read 2026-10-29) [verified, read in-session]

  • Source: Anthropic.
  • Key idea: default to one main loop plus skills and tools. Handoffs lose context, nested loops add latency, and subagents overlap.
  • Results: one customer ran about 2x faster after merging into a single loop.
  • When a subagent is still worth it:
    1. Context bloat is hurting the main loop.
    2. Parallel, disposable clean-context work.
    3. Too many tools, or conflicting instructions.
  • Change one subagent at a time and verify with evals.
  • Search relevance: lightweight targeted search belongs in the main loop as a tool or skill. Broad exploratory search is exactly the subagent case.

The anatomy of effective commerce agents (claude.com/resources/articles, read 2026-10-29) [verified, read in-session]

  • Source: Anthropic.
  • Key idea:
    • Product search is a tool documented in the system prompt; it is neither a skill nor a subagent.
    • Rule of thumb: what is used in most turns goes in the system prompt; occasional capabilities become skills.
    • Strip unused fields from tool results.
  • Other practices:
    • Prompt cache ordered global / session / volatile (90–99% hit rate in production).
    • Memory is stored in a database and extracted asynchronously after each turn (+13% recall).
    • Snapshot evals with 50–100 cases per flow.
    • Safety is enforced by the harness, with server-issued IDs for writes.
  • Search relevance: when a backend engine already ranks results and the agent only selects and presents, search is a tool, not a subagent.

Anthropic "Scion" source-control blog [verified 2026-10-30]

  • Source: Anthropic.
  • Found: weekly digest 10-19. Identified on 10-30 as "Redesigning source control at Anthropic" (Oct 13, 2026): merges grew from ~2,500/day (May) to 30,000+/day (Sep). Full entry in tech-blogs.md.

Claude skills / Skills API out of beta; /skill-doctor [unverified]

  • Source: Anthropic (content unverified).
  • Found: digest 09-07 (Fable 5.1 release notes).

OpenAI

Deep research (product, Feb 2025) [verified]

  • Source: OpenAI.
  • Key idea: an o3-based agent trained end-to-end with RL on browsing and analysis tasks. It reports with citations.
  • Search relevance: the first major lab to ship RL-trained deep search.

Codex harness open-sourced (2026-09 digest) [unverified]

  • Source: OpenAI (content unverified).
  • Key idea: Thread/Turn/Item architecture with context compaction.
  • Results: ARC-AGI-3 13.3→38.3% with 6x fewer tokens.

OpenAI research acceleration / inference-cost signals (digest 09-07; tracker 09-08) [unverified]

  • Source: OpenAI (content unverified).
  • Results: 3.1 agent-workdays per human workday; the median researcher spends about $600/day on inference.

Lilian Weng, "Harness Engineering for Self-Improvement" (blog, 2026-07-04) [verified via tracker; read 09-23]

  • Source: high-discussion (widely shared essay by a prominent researcher).
  • URL: lilianweng.github.io/posts/2026-07-04-harness/
  • Key idea: synthesizes about 35 papers. RSI will arrive through harness engineering, not weight rewriting. Lays out five optimization routes and seven open problems. STOP helps GPT-4 but hurts GPT-3.5.
  • Search relevance: the field's reference essay.

Google / DeepMind

AlphaEvolve (DeepMind, May 2025) [verified]

  • Source: Google DeepMind.
  • Key idea: an evolutionary coding agent in which LLMs propose code diffs and automated evaluators score them.
  • Results: production wins in Google data-center scheduling and kernels; new math results.
  • Search relevance: the diff-plus-evaluator loop is the template for evolving search programs (see RPL and PSA).

Google RSI bet (Gemini 3.8, AlphaEvolve lineage) [unverified signal, tracker 09-12]

  • Source: Google (signal unverified).

Google Cloud AI Research harness-RSI line: Harness² (2610.14224), RRSI (2609.24972), Evo² (2610.13025) [papers verified via arXiv IDs in run logs]

  • Source: Google (arXiv affiliations verified 2026-10-29; Evo² is Google DeepMind).
  • One lab with three consistent papers in October 2026, which makes this the strongest lab-backed harness-RSI program.

Meta

Trellis / Experience Graphs in KernelEvolve (2606.29823) [unverified beyond paper claim]

  • Source: Meta (arXiv affiliations).
  • Results: about 10x faster to target; -52% tokens; reported to be in production.

Meta Superintelligence Labs co-authors RPL (2610.24854) [paper]

  • Source: Meta (arXiv affiliations).

Alibaba / Tongyi

Tongyi DeepResearch, WebSailor [verified that they exist]

  • Source: Alibaba.
  • Used as frozen backbones in QREF (2610.21512). Alibaba also published 2610.20845, on the co-evolution trap.

Other industry and community

Karpathy AutoResearch (digest 04-29) [unverified]

  • Source: high-discussion (Andrej Karpathy, a prominent researcher; content unverified).
  • A 630-line autonomous ML experimentation loop.

Watchlist (low signal)

Items whose Source is other or unverified under hao's 2026-10 inclusion rule. Kept for reference and not deleted. Do not use them as primary evidence.

LlamaIndex, Jerry Liu retrieval-harness post [unverified content]

  • Source: other (LlamaIndex, startup; content unverified).
  • Link: x.com/jerryjliu0/status/2073407100642852871 (found 09-10).
  • Moved to watchlist 2026-10-29 from: Other industry and community.

LangChain, Harrison Chase post (2026-10-01) [unverified content]

  • Source: other (LangChain, startup; content unverified).
  • No new posts since August (checked 10-22).
  • Moved to watchlist 2026-10-29 from: Other industry and community.

WecoAI AIDE² (2026-07-14) [unverified beyond the blog]

  • Source: other (WecoAI blog; the related paper 2609.26457 is below the bar at HF 28).
  • An offline, scored RSI loop for ML engineering.
  • Moved to watchlist 2026-10-29 from: Other industry and community.

JevHarness (github.com/TianyuCodings/JevHarness) and TypeSafe "System One Model" (docs.typesafe.ai/introduction) [early access, unverified]

  • Source: other (early-access startup product; no independent eval).
  • Key idea: typed fast decisions made by a small separate model. Claims about 200x faster and 400x cheaper.
  • Search relevance: hao's candidate for query rewriting. Related paper: SearchJev 2610.05107.
  • Moved to watchlist 2026-10-29 from: Other industry and community.

Prime Agent (digest 08-24; org not recorded) [unverified]

  • Source: unverified (organization not recorded).
  • /refine turns trajectories into skills, memories and prompts.
  • Claims 95.5% on ARC-AGI-3 with Opus 5.
  • Moved to watchlist 2026-10-29 from: Other industry and community.

RHI (2607.15524), MASS (2610.12176); Sakana RSI Lab moving into physical AI (tracker 09-12) [papers verified via arXiv; physical-AI move is an unverified signal]

  • Source: other (Sakana AI; arXiv affiliations. Not a major lab under the 2026-10-30 list; RHI and MASS are both below the high-discussion bar).
  • Correction 2026-10-29: HELIX (2608.13951) used to be listed here, but its arXiv affiliations are HKU (Chao Huang's group), not Sakana. It now sits in the rsi-harness-evolution.md watchlist.
  • Moved to watchlist 2026-10-30 from: Sakana AI. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

EVOHARNESSBENCH (2609.04280) [paper]

  • Source: other (Salesforce AI Research. Not a major lab under the 2026-10-30 list; HF 29 upvotes, 7 citations).
  • Moved to watchlist 2026-10-30 from: Salesforce. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

Cohere "Teams of Coding Agents" (digest 10-19) [unverified]

  • Source: other (Cohere. Not a major lab under the 2026-10-30 list, which reverses the 2026-10-29 provisional acceptance; content unverified).
  • Moved to watchlist 2026-10-30 from: Other industry and community. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

Papers Index

Each paper appears in one primary category. Cross-listed papers are marked "Cross-listed" inside the secondary file. IDs are arXiv IDs unless the entry says "ID not recovered"; IDs are never guessed.

File Scope Entries (headings) Main (lab + high-discussion) Watchlist (other + unverified)
rsi-harness-evolution.md Evolving prompts, tools, middleware and control flow; overfitting and regularization; verifier-free; evolve-the-evolver 36 16 20
rsi-memory-evolution.md Memory programs, memory banks, self-evolving indexes 16 7 9
rsi-skill-evolution.md Skill libraries, skill-policy co-evolution, transfer and its failure modes 24 10 14
rsi-training-and-theory.md Weight-level RSI, co-training, theory and limits, RSI benchmarks, critic co-evolution for search 30 15 15
search-rl-policies.md RL-trained search agents, process rewards, over-search control 14 6 8
search-retrieval-and-query.md Retrieval programs, query refinement, BM25 vs LLM retrievers, rerankers 14 6 8
search-context-and-deep-research.md Context and state management, deep-research architectures, benchmarks 10 5 5
industry-practice.md Anthropic, OpenAI, Google/DeepMind, Meta, Alibaba, community (Sakana, Salesforce and Cohere now in its watchlist); each item tagged by confidence 26 18 8
tech-blogs.md Lab and builder blog posts (added 2026-10-30; 98 backfilled from archive/blog-backfill-raw the same day), in theme sections, with confidence tags 101 92 (69 lab, 23 high-discussion) 9

Some headings group 2–3 related items, and a few are cross-listed (MASS, SkillQR and APEx appear in two files). Unique items: 127 distinct arXiv IDs (21 recovered on 2026-10-29), 2 papers that still have no ID, 5 non-paper items, plus the industry file. The ClawLoop/RiseHarness heading was split into two entries on 2026-10-29, so the harness file went from 35 to 36.

Source tiers (added 2026-10-29; re-tiered 2026-10-30)

Every entry has a Source bullet: a lab name verified from arXiv affiliations, high-discussion with the evidence, other, or unverified. Entries tagged other or unverified sit in a "Watchlist (low signal)" section at the bottom of each file. The rule is in ../index.md.

Re-tiered 2026-10-30 under hao's final major-lab list (see ../index.md rule 7). Amazon, Salesforce, Sakana, Shanghai AI Lab, BAAI and Cohere no longer count as labs, so their entries count only if high-discussion; JD now counts.

Distinct items in the 7 paper files (140; cross-lists counted once). Main = 65, watchlist = 75. Added on 2026-10-30 (daily run): main +5 (Memory Tax, HABIT-Bench, The Trinity, MPO, RSI-Anything, all high-discussion), watchlist +3 (Skill∞, SearchHorizonBench, RSI Arena). The bullets below are the re-tier snapshot from before that run.

  • Lab: 43. 9 of these are also high-discussion. The count includes the OpenAI Codex cross-listing and the mixed-lab benchmark list.
  • High-discussion only: 17 (Self-Retrospection Distillation, Salesforce, and AREX, BAAI, stay in main on discussion alone).
  • Other: 69.
  • Unverified: 3 (ROFT, SCORE, ICLR 2026 RSI workshop).
  • Before 2026-10-30 the counts were lab 56 / high-discussion only 15 / other 58 / unverified 3 (main 71, watchlist 61).

Changes on 2026-10-30:

  • Demoted to watchlist (12 entries, 11 distinct items). Amazon: Harness Updating ≠ Harness Benefit 2605.30621, VERSE 2610.02616, Evo-Harness 2608.15071, SkillLearnBench 2604.20087, query rewriting 2610.10955. Salesforce: EVOHARNESSBENCH 2609.04280, HASP 2605.17734. Sakana: RHI 2607.15524, MASS 2610.12176 (both files). Shanghai AI Lab: HoH 2609.01481, MemHarness 2607.28272, Self-Harness 2606.09498. None reach the high-discussion bar: HF upvotes were 0–34; citations were below 100 where Semantic Scholar answered (some lookups were rate-limited).
  • Promoted to main: SearchMaster 2608.01822 (JD).
  • Relabeled, still main: Self-Retrospection Distillation 2610.08077 and AREX 2607.21461 are now high-discussion instead of lab.
  • Not changed: ClawLoop stays unverified (likely Meituan, now a listed lab, but there is still no printed affiliation). APEx (one ByteDance co-author), Web2BigTable (2 Huawei co-authors, UCL corresponding author) and the Deep Research survey (one minor Huawei co-author) stay other under the minor-co-author rule.

Industry file: 16 lab, 2 high-discussion (Weng, Karpathy), 7 other, 1 unverified. Sakana (RHI/MASS), Salesforce (EVOHARNESSBENCH) and Cohere were moved to its watchlist.

Most frequent labs among the papers (2026-10-30 list): Tencent 8, Google/DeepMind 8, Alibaba 6, Huawei 4, Microsoft 4, Meta 4, Kuaishou 2, ByteDance 2, Ant Group 1, Baidu 1, OpenAI 1, JD 1. Non-listed affiliations of items now in the watchlist: Amazon 5, Salesforce 2, Shanghai AI Lab 3, Sakana 2.

Correction: HELIX 2608.13951 is from HKU (Chao Huang), not Sakana. It is now in the harness watchlist.

Key papers to read first

  1. 2603.28052 Meta-Harness: the base method (high-discussion; 352 citations).
  2. 2609.24972 RRSI: how to make evolved harnesses generalize.
  3. 2610.14224 Harness²: verifier-free harness evolution, for deep research.
  4. 2610.24854 RPL and 2610.06689 PSA: search as programs.
  5. 2610.21512 QREF: an independently improvable query module.
  6. 2610.20845 Evolving Together and 2610.25858 Continual Learners: the failure modes.
  7. 2610.21170 MemScope: memory evolution on GAIA. Source is other (ICT CAS), so it is now in the memory watchlist; read it for the method, not as evidence.

Missing IDs (do not invent)

Updated 2026-10-29. Arxiv title search recovered IDs for 21 items, and each one is noted in its entry as "arXiv ID (title match found 2026-10-29)": Dream-RSI 2609.14858, SE-Search 2603.03293, SearchMaster 2608.01822, HindSearch 2608.01597, ERSkill 2608.12720, SAAS 2605.29796, SelfSearch 2609.37968, APEx 2609.02253, Break It Down 2608.20274, EvolveMem 2605.13941, SAGE benchmark 2602.05975, VERSE 2610.02616, EvoAgentBench 2607.05202, CoSearch 2604.17555, SkillLearnBench 2604.20087, SRA 2604.24594, Agentic-R 2601.11888.

Four of those 21 are candidates: the match came from abstract or title search and should be confirmed before citing. They are RiseHarness 2610.27435, ClawLoop 2610.14118, SELF-INDEX 2609.19656, and AiScientist File-as-Bus 2604.13018.

Still missing: ROFT and SCORE. Non-paper items with no arXiv ID: AIDE², the ICLR 2026 RSI workshop, the Codex harness, the benchmark list, and Artificial Analysis.

RSI: Harness Evolution

Self-improvement by rewriting the agent's harness (prompts, tools, middleware, sub-agents, control flow) with model weights frozen, or with only light co-training.

Field legend:

  • Found: the date the item entered our tracking.
  • Sent: whether the daily tracker or weekly digest messaged hao about it. yes = daily message; digest = weekly paper digest; logged = tracker only; unknown = predates the run log.
  • ID not recovered: the run log never recorded the arXiv ID. Do not guess one.

Foundations and frameworks

2603.28052 · Meta-Harness

  • Found: 2026-10-13 (deep dive for hao). Sent: yes (chat deep dive).
  • Source: high-discussion (352 Semantic Scholar citations as of 2026-10-29; the common baseline for RRSI, MESH-Harness and Continual Learners; covered in Lilian Weng's essay). Affiliation: Stanford/MIT/KRAFTON.
  • Key idea: Stanford/MIT/KRAFTON (Yoonho Lee, Omar Khattab, Chelsea Finn). Claude Code (Opus 4.6) acts as proposer over a filesystem holding every prior candidate's code, raw traces and scores. It reads a median of 82 files per iteration.
  • Results:
    • Text-classification ablation: scores only 34.6, plus summaries 34.9, plus raw traces 50.0.
    • Text classification 48.6% (+7.7 over ACE with about 1/4 of the context).
    • RAG math: +4.7 average over 5 held-out models.
    • TerminalBench-2: 76.4% with Opus 4.6 (Terminus-KIRA 74.7%); 37.6% with Haiku 4.5.
  • Search relevance: the harnesses it found include draft-then-verify retrieval and 4-route BM25. Raw traces beating summaries is the key lesson for any search-harness optimizer.
  • Detail: /workspace/agent/paper-digests/deep-dives/meta-harness-vs-mesh.md

2609.00829 · HarnessEvolve

  • Found: baseline (09-08); sent 2026-09-24. Sent: yes.
  • Source: Huawei (arXiv affiliations).
  • Key idea: uses reference trajectories to locate failure points, with dual gating against shortcuts and forgetting.
  • Results: enterprise QA 43.4%→86.9%, 21.6 points above baselines. Transfers across frameworks.
  • Search relevance: enterprise QA is a search task, so this is the closest early evidence of harness RSI on search.

2608.25593 · JIT-Agent

  • Found: baseline / digest 09-01. Sent: digest.
  • Source: high-discussion (HF Daily Papers 84 upvotes). Affiliation: NUS.
  • Key idea: trains a model to generate a task-specific harness on the fly.
  • Results: DeepSeek-V4-Flash with JIT-Agent beats GPT-5.6 on DeepSearchQA (+9.1).
  • Search relevance: high. Per-query harness generation evaluated on a deep-search benchmark (tracker idea #3).

2604.25850 · Agentic Harness Engineering

  • Found: digest week of 05-02/05-05. Sent: digest.
  • Source: high-discussion (166 Semantic Scholar citations as of 2026-10-29). Affiliation: Fudan.
  • Key idea: automatically evolves a coding-agent harness using observability data.
  • Results: TerminalBench-2 69.7→77.0; tools and middleware matter more than system prompts; -32% tokens; transfers across models (+5–10pp).
  • Search relevance: medium. Its finding that edits to tools and middleware beat prompt edits applies equally to search tools.

2607.13285 · Harness Handbook

  • Found: digest 08-10. Sent: digest.
  • Source: Tencent (arXiv affiliations); also high-discussion (HF 163 upvotes).
  • Key idea: builds behavior→source maps for harnesses over 100K lines.
  • Results: planning win rate for harness modifications 28.3→45.6%.
  • Search relevance: low. Useful infrastructure for an RSI proposer.

2608.01964 · LongHorizon-Harness (Manager-Executor-Auditor)

  • Found: digest 08-10. Sent: digest.
  • Source: Alibaba (DreamX; arXiv affiliations); also high-discussion (HF 187 upvotes).
  • Key idea: three roles; an independent auditor prevents state drift.
  • Results: WeaveBench 51.8→80.7; OSWorld (Opus 4.7) 20→34.3.
  • Search relevance: medium. The auditor role corresponds to claim verification in deep research.

Generalization, overfitting, regularization

2609.24972 · RRSI (Regularized RSI of Agent Harnesses)

  • Found: 2026-10-11. Sent: yes; deep dive 10-20.
  • Source: Google (Google Cloud AI Research; arXiv affiliations); also high-discussion (HF 220 upvotes, 1.7K GitHub stars).
  • Key idea: Google Cloud AI Research and Stanford, with Yoonho Lee as co-author.
    • Proposal side: annealed edit sparsity (4→1 edits per round), an evidence-aware credit log, and structured exploration when progress stalls.
    • Selection side: a leakage critic, a stability gate within noise, complexity paid for by score, and pruning of components with no gain.
  • Results:
    • TB2.1 74.2→80.2; SWE-bench Verified +1.8 OOD.
    • Office workspace: Meta-Harness 93.0 on the evolve set but only +0.9 OOD; RRSI 90.5 on the evolve set, OOD 43.6 vs a 39.7 baseline.
    • Unregularized: 92.8 / 40.3 OOD.
    • Frontier-Eng +4.3 medal points.
    • 2.42M tokens per trial vs 3.8M unregularized.
  • Search relevance: not tested on search, but it is the template for making an evolved search harness transfer.
  • Detail: /workspace/agent/paper-digests/deep-dives/rrsi.md

Verifier-free and open-world

2610.14224 · Harness² (Google Cloud AI Research + Northwestern)

  • Found: 2026-10-14. Sent: yes; deep dive 10-18.
  • Source: Google (Google Cloud AI Research + Northwestern; arXiv affiliations); also high-discussion (HF 125 upvotes, 21K GitHub stars).
  • Key idea: Propose (general plus domain edits) → Probe both on the same task → Compose based on behavioral contrast, not scores. Lessons go into an EXPERIENCE.md library. 8 editable components. Opus 4.8 is the editor.
  • Results:
    • WorkBuddy (GLM-5) 64.2→75.7; LAB 59.6→71.4; JobBench +4.5.
    • k=5 adds +3–7.
    • GLM-5 acting as its own editor: +13.7 on LAB.
    • Library transfer +0.5–2.9.
    • Ablations: no library -3.1; no general/domain split -2.9 to -3.7; no compose -1.6 to -1.9.
  • Search relevance: high. Deep research usually has no gold answers, and this works without them. Caveat: evaluation is also LLM-judged.
  • Code: github.com/google-research/harness2
  • Detail: /workspace/agent/paper-digests/deep-dives/harness2.md

Proposer / optimizer improvement ("evolve the evolver")

2610.13025 · Evo² (Google)

  • Found: 2026-10-17. Sent: yes.
  • Source: Google DeepMind (arXiv affiliations).
  • Key idea: the system that proposes and judges harness changes is itself trained.
  • Results: SWE-bench Verified 41→48%; evolved harnesses transfer to other tasks without further evolution.
  • Search relevance: medium (coding).

2610.00906 · ActiveSaddler

  • Found: 2026-10-12. Sent: yes.
  • Source: Microsoft (arXiv affiliations); also high-discussion (HF 86 upvotes).
  • Key idea: a bandit curriculum picks which training tasks the harness optimizer works on next.
  • Results: +4.4 on GAIA2.
  • Search relevance: medium-high. GAIA2 is search-heavy.

2610.05039 · Causal Improvement Graph

  • Found: 2026-10-12. Sent: yes (minor note).
  • Source: Alibaba (Tongyi Lab + NTU; arXiv affiliations).
  • Key idea: stores optimization history as a graph so the loop does not retry failed ideas. No numbers given.

2610.13991 · IDM

  • Found: 2026-10-25. Sent: logged.
  • Source: Microsoft Research (arXiv affiliations).
  • Key idea: predicts a self-change's score before the expensive evaluation runs.
  • Search relevance: medium. Could cut the cost of evaluating search harnesses.

2610.15150 · Harmony Harness

  • Found: 2026-10-17. Sent: yes (minor note).
  • Source: Google (Google Cloud AI Research + Google DeepMind; arXiv affiliations).
  • Key idea: multi-objective harness evolution. For search the objectives would be accuracy, cost and citation faithfulness.

2610.22791 · Self-HPO

  • Found: 2026-10-21. Sent: yes (minor note).
  • Source: Microsoft Research (arXiv affiliations); also high-discussion (HF 56 upvotes).
  • Key idea: a coding agent tunes its own hyperparameters.

2610.27435 (candidate) · RiseHarness

  • Found: 2026-10-21. Sent: logged. A terminal-agent harness paper (logged together with ClawLoop).
  • Source: Tencent (Tencent HY; arXiv affiliations of the candidate paper "Harness Self-Evolution for Long-Horizon Terminal Agents").
  • arXiv ID: 2610.27435 (candidate; matched by arXiv abstract/title search 2026-10-29, confirm before citing).

2610.18813 · Self-Evolving Agent Harness (continual-PAC)

  • Found: 2026-10-15. Sent: logged.
  • Source: high-discussion (HF 102 upvotes). Affiliation: ZGCA.
  • Key idea: theory on how much validation data is needed before accepting a harness change. No benchmarks.

Watchlist (low signal)

Entries whose Source is other or unverified under hao's 2026-10 inclusion rule (major lab or high discussion). Kept for reference and not deleted. Do not cite them as primary evidence for trends or proven solutions.

2610.05300 · MESH-Harness

  • Found: 2026-10-11. Sent: yes.

  • Source: other (CUHK-SZ/NTU/Fudan/NUS/DeepWisdom).

  • Key idea: CUHK-SZ/NTU/Fudan/NUS/DeepWisdom. The harness is split into slots with fixed interfaces and a pool of implementations per slot. A full-covariance LinUCB over Qwen3-Embedding features chooses which combination to evaluate. Failure-repair driven by traces refreshes each slot's pool.

  • Results (budget of 40 evaluations, vs Meta-Harness):

    Task MESH-Harness Meta-Harness
    Text 56.3 50.6
    RAG math 53.2 46.2
    LiveCodeBench 77 75
    ScienceWorld 58.1 53.1
    • Test cost -44% ($1.98 vs $3.55).
    • Ablations: random composition -7.5; no uncertainty term -4.6; no pool refresh -5.1; refresh without traces -2.5.
    • Refining a Meta-Harness solution gave 50.6→54.4.

  • Search relevance: the largest gain is on retrieval-augmented reasoning. Modular slots map naturally onto a search harness (query writer / retriever / reader / verifier).

  • Moved to watchlist 2026-10-29 from: Foundations and frameworks.

2601.21557 · Meta Context Engineering (ICML 2026)

  • Found: digest 09-07. Sent: digest.
  • Source: other (PKU; 59 citations, below the high-discussion bar).
  • Key idea: a bi-level setup where a meta-agent evolves context-engineering skills through agentic crossover.
  • Results: +16.9% mean over SOTA across 5 domains.
  • Search relevance: medium.
  • Moved to watchlist 2026-10-29 from: Foundations and frameworks.

2608.04968 · EvolveNet

  • Found: digest 08-17. Sent: digest.
  • Source: other (HKBU/USTC/HKUST).
  • Key idea: distributed agents evolve a shared harness on their local workloads and combine the results through scope-typed aggregation.
  • Results: beats centralized evolution on 5 domains.
  • Search relevance: medium. A possible design for federated search-harness learning across tenants.
  • Moved to watchlist 2026-10-29 from: Foundations and frameworks.

2608.01918 · HarnessCompass

  • Found: digest 08-17. Sent: digest.
  • Source: other (BIT/CityU).
  • Key idea: generalizable harness evolution aimed at avoiding overfitting.
  • Results: SWE-bench Verified 54→66% with GPT-5.4 in 5 iterations; transfers to held-out tasks and other models.
  • Search relevance: medium.
  • Moved to watchlist 2026-10-29 from: Generalization, overfitting, regularization.

2606.28374 · RSEA

  • Found: 2026-09-09. Sent: unknown.
  • Source: other (Monash).
  • Key idea: a three-layer natural-language state with a held-out keep-better gate.
  • Results: not recorded in detail.
  • Search relevance: the held-out gate pattern is reusable for search harnesses.
  • Moved to watchlist 2026-10-29 from: Generalization, overfitting, regularization.

2610.36826 · RuHarness

  • Found: 2026-10-29 (posted Oct 27). Sent: yes.
  • Source: other (HIT/Tsinghua).
  • Key idea: alternates rubric updates with harness updates and retires rubrics once they stop separating good runs from bad. This addresses evaluator staleness.
  • Results: tested on information retrieval among other domains (numbers not recovered).
  • Search relevance: high. Search rubrics go stale as old failure modes get fixed.
  • Moved to watchlist 2026-10-29 from: Verifier-free and open-world.

2609.37968 · SelfSearch

  • Found: 2026-10-03. Sent: yes.
  • Source: other (SNU).
  • arXiv ID: 2609.37968 (title match found 2026-10-29).
  • Key idea: reward-free search over harness modifications, driven by episode logs.
  • Search relevance: medium.
  • Moved to watchlist 2026-10-29 from: Verifier-free and open-world.

2608.24735 · Meta^n

  • Found: 2026-09-10. Sent: unknown.
  • Source: other (UMN; HF 18 upvotes).
  • Key idea: multi-level meta-optimization (details not recorded).
  • Moved to watchlist 2026-10-29 from: Proposer / optimizer improvement ("evolve the evolver").

2610.21791 · RSIFlow

  • Found: 2026-10-18. Sent: logged.
  • Source: other (Beihang).
  • Key idea: a meta-agent chooses which component (prompt, memory or tools) to improve based on where the agent fails. Claims the top average on 6 benchmarks without headline numbers.
  • Moved to watchlist 2026-10-29 from: Proposer / optimizer improvement ("evolve the evolver").

2610.14118 (candidate) · ClawLoop

  • Found: 2026-10-21. Sent: logged. A terminal-agent harness paper (logged together with RiseHarness).
  • Source: unverified (no affiliation printed on arXiv; the HF org is LongCat and an author is Xunliang Cai, which suggests Meituan, but this is not confirmed). Re-checked 2026-10-30: the arXiv page still prints no affiliation; HF 35 upvotes. Meituan is on the 2026-10-30 major-lab list, so promote this entry if the Meituan affiliation is confirmed.
  • arXiv ID: 2610.14118 (candidate; matched by arXiv abstract/title search 2026-10-29, confirm before citing).
  • Moved to watchlist 2026-10-29 from: Proposer / optimizer improvement ("evolve the evolver").

2608.13951 · HELIX (HKU; formerly mislabeled Sakana)

  • Found: 2026-09-10. Sent: unknown.
  • Source: other (HKU, Chao Huang's group). Correction 2026-10-29: earlier versions of this KB labeled HELIX as Sakana; the arXiv affiliations do not support that.
  • Key idea: typed ports and atoms, giving a composable harness DSL.
  • Moved to watchlist 2026-10-29 from: Sakana, Amazon and other lab work.

2607.13683 · HarnessBank

  • Found: 2026-09-11. Sent: unknown.
  • Source: other (Shanda).
  • Results: +5.1–15.4% on 7 benchmarks.
  • Moved to watchlist 2026-10-29 from: Sakana, Amazon and other lab work.

2610.03548 · Recursive Harness Self-Improvement for Frontier Reasoning Data Synthesis

  • Found: 2026-10-14. Sent: skipped. Evolves harnesses to generate harder training data, not for search.
  • Source: other (USTC and others).
  • Moved to watchlist 2026-10-29 from: Sakana, Amazon and other lab work.

2610.07832 · HERMES / Dev-Primitives

  • Found: weekly digest 10-19. Sent: digest.
  • Source: other (UIUC/PSU).
  • Key idea: a coding-agent harness in which each repo component has its own LLM.
  • Moved to watchlist 2026-10-29 from: Sakana, Amazon and other lab work.

2609.01481 · Harness-of-Harness (HoH)

  • Found: baseline / weekly digest 09-07. Sent: digest.
  • Source: other (Shanghai AI Lab; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 21 upvotes, 5 citations, below the bar).
  • Key idea: a meta-harness wraps any coding agent in planning, coding and testing loops with persistent state.
  • Results: 52.25% average gain; built a playable FPS over 70 iterations.
  • Search relevance: low (coding). Shows a pattern for multi-day persistent state.
  • Moved to watchlist 2026-10-30 from: Foundations and frameworks. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

2605.30621 · Harness Updating ≠ Harness Benefit

  • Found: 2026-09-12. Sent: unknown.
  • Source: other (Amazon; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 21 upvotes, below the bar; citations not re-checked, Semantic Scholar rate-limited).
  • Key idea: a 9B evolver is comparable to Opus 4.6. The bottleneck is whether the executor actually uses the evolved artifacts.
  • Search relevance: high. Check that the search agent actually invokes the evolved search skills.
  • Moved to watchlist 2026-10-30 from: Generalization, overfitting, regularization. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

2609.04280 · EVOHARNESSBENCH (Salesforce)

  • Found: 2026-09-12. Sent: unknown.
  • Source: other (Salesforce AI Research; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 29 upvotes, 7 citations, below the bar).
  • Key idea: 17 streams and 802 tasks; documents harness-induced forgetting.
  • Search relevance: an evaluation template for continual search-harness evolution.
  • Moved to watchlist 2026-10-30 from: Generalization, overfitting, regularization. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

2610.02616 · VERSE

  • Found: 2026-10-06. Sent: yes.
  • Source: other (MIT + Amazon, corresponding author at Amazon; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 0 upvotes).
  • arXiv ID: 2610.02616 (title match found 2026-10-29).
  • Key idea: self-evolution of the optimizer needs execution-based verification.
  • Moved to watchlist 2026-10-30 from: Proposer / optimizer improvement ("evolve the evolver"). Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

2607.15524 · RHI (Sakana / UC Berkeley)

  • Found: 2026-09-10. Sent: unknown.
  • Source: other (Sakana AI / UC Berkeley; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 25 upvotes, 38 citations, below the bar).
  • Key idea: recursive harness improvement; about -60% cost per the 09-11 synthesis.
  • Moved to watchlist 2026-10-30 from: Sakana, Amazon and other lab work. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

2608.15071 · Evo-Harness (EMNLP 2026)

  • Found: 2026-09-12. Sent: unknown.
  • Source: other (UIUC-led, senior author Hanqing Lu at Amazon; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 0 upvotes, 23 citations, below the bar).
  • Key idea: general and topic skills on a frozen backbone.
  • Results: SOTA on 5 benchmarks.
  • Moved to watchlist 2026-10-30 from: Sakana, Amazon and other lab work. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

RSI: Memory Evolution

Self-improvement of what an agent stores and how it retrieves it: memory programs, memory banks and retrieval indexes.

The field legend is in rsi-harness-evolution.md.

2610.06361 · PrisMem

  • Found: 2026-10-12. Sent: yes.
  • Source: Microsoft (arXiv affiliations).
  • Key idea: evolves the memory store and retrieve programs one capability at a time, instead of judging the whole program on a single score.
  • Results: +10.5 on BEAM-1M; +7.8 on LongMemEval-M.
  • Search relevance: medium-high. Shows that decomposing the score works for retrieval programs.

2606.29823 · Experience Graphs / Trellis (Meta)

  • Found: 2026-09-10. Sent: unknown.
  • Source: Meta (Meta Platforms; arXiv affiliations).
  • Key idea: experience graphs used for optimization. It is in production in Meta KernelEvolve.
  • Results: about 10x faster to target; -52% tokens.
  • Search relevance: medium. A production-proven experience-memory pattern.

2608.01739 · CoEvo-Mem

  • Found: 2026-09-30. Sent: yes.
  • Source: Alibaba (Alibaba Group; arXiv affiliations).
  • Key idea: co-evolves the retrieval policy and the memory bank.
  • Search relevance: high. Exposed to the co-evolution trap described in 2610.20845.

2609.19656 (candidate) · SELF-INDEX

  • Found: 2026-09-25. Sent: yes.
  • Source: high-discussion (HF 56 upvotes). Affiliation: Yonsei.
  • arXiv ID: 2609.19656 (candidate; matched by arXiv abstract/title search 2026-10-29, confirm before citing).
  • Key idea: a self-evolving retrieval index.

2603.03293 · SE-Search

  • Found: 2026-09-26. Sent: yes.
  • Source: Tencent (Tencent YoutuLab; arXiv affiliations).
  • arXiv ID: 2603.03293 (title match found 2026-10-29).
  • Key idea: memory plus dense reward for search self-evolution.

2610.34073 · Memory Tax: Dissecting the Net Value of Agent Memory

  • Found: 2026-10-30. Sent: yes.
  • Source: high-discussion (HF Daily Papers 10-28, 157 upvotes). Authors include Jiawei Han and ChengXiang Zhai; HF lists UIUC, and the arXiv page prints no affiliations. Not a listed lab.
  • Key idea: memory's net value = memory benefit (better organization, distilled preferences) minus memory tax (evidence hidden or dropped when raw history is rewritten). Evaluate memory against naive RAG over raw history, not standalone.
  • Results: across several long-horizon benchmarks, many dedicated memory systems underperform naive RAG, i.e. negative net value. Code to be open-sourced.
  • Search relevance: strong baseline warning for memory-evolution work (T9). Any evolved memory program should beat "retrieve over the raw log" before it counts.

2610.24520 · HABIT-Bench: Habit Induction from Longitudinal Weak Evidence in Agent Memory

  • Found: 2026-10-30. Sent: yes.
  • Source: high-discussion (HF Daily Papers, 91 upvotes). HF lists SJTU; the arXiv page prints no affiliations. Not a listed lab.
  • Key idea: benchmark for inferring user habits from many partial observations, including exceptions, habit drift and rejected suggestions; four domains (food, finance, software, travel).
  • Results: 8 memory methods on Qwen3-235B-A22B and GLM-5.3-Flash; the best reaches only 33.61% accuracy.
  • Search relevance: memory retrieval of weak, distributed evidence is unsolved; complements Memory Tax as an evaluation target for memory evolution.

Watchlist (low signal)

Entries whose Source is other or unverified under hao's 2026-10 inclusion rule (major lab or high discussion). Kept for reference and not deleted. Do not cite them as primary evidence for trends or proven solutions.

2608.24876 · Recuris

  • Found: baseline (09-08); confirmed real 09-25. Sent: unknown/yes.
  • Source: other (Princeton/Stanford/NUS; HF 32 upvotes, 251 GitHub stars, 14 citations: below the bar).
  • Key idea: recursive memory evolution for long tasks.
  • Results: +32.2pp on long tasks.
  • Search relevance: one of hao's tracked themes (memory evolution for retrieval).
  • Moved to watchlist 2026-10-29 from: (top level).

2610.21170 · MemScope

  • Found: 2026-10-21. Sent: yes.
  • Source: other (ICT CAS).
  • Key idea: a patch agent sees both the execution traces and the runtime memory state, then makes small targeted fixes to the memory code (median 4.4% of the code changed).
  • Results:
    • GAIA 86.7% vs 70% when it sees traces only.
    • xBench 49→55.
    • ALFWorld unseen 51→72%.
  • Search relevance: high. Memory evolution tested directly on deep-research benchmarks.
  • Moved to watchlist 2026-10-29 from: (top level).

2605.13941 · EvolveMem

  • Found: 2026-10-05. Sent: yes.
  • Source: other (UNC; HF 14 upvotes).
  • arXiv ID: 2605.13941 (title match found 2026-10-29).
  • Key idea: co-evolves the retrieval mechanism's configuration and discovers new configuration dimensions.
  • Moved to watchlist 2026-10-29 from: (top level).

2602.07755 · ALMA (Jeff Clune)

  • Found: digest 05-25. Sent: digest.
  • Source: other (UBC/Vector; HF 8 upvotes, 21 citations).
  • Key idea: a meta-agent searches memory architectures written as executable code.
  • Results: outperforms hand-designed memory systems.
  • Search relevance: medium.
  • Moved to watchlist 2026-10-29 from: (top level).

2605.12061 · SAGE (self-evolving graph memory)

  • Found: digest 05-18/05-25. Sent: digest.
  • Source: other (PKU).
  • Key idea: a reader-writer feedback loop.
  • Results: NQ zero-shot Recall@2/5 82.5/91.6; evolution reduces hallucination.
  • Search relevance: high. A self-evolving retrieval structure. Not the same work as the SAGE benchmark in search-retrieval-and-query.md.
  • Moved to watchlist 2026-10-29 from: (top level).

2608.02113 · MemArbiter

  • Found: digest 08-17. Sent: digest.
  • Source: other (HUST/Fudan).
  • Key idea: identifies a "memory-action gap": retrieved information often fails to guide the decision.
  • Results: ALFWorld 92.5% at 750 tokens (+25.4pp).
  • Moved to watchlist 2026-10-29 from: (top level).

2610.07309 · Retrieval Admissibility in Agent Memory

  • Found: digest 10-19. Sent: digest.
  • Source: other (UALR).
  • Key idea: checks whether a retrieved memory is allowed to be used, not just whether it is relevant.
  • Search relevance: directly addresses the similarity-retrieval collision reported in 2610.25858.
  • Moved to watchlist 2026-10-29 from: (top level).

2609.02253 · APEx

  • Found: 2026-10-04. Sent: yes.
  • Source: other (USTC/CAS; a single ByteDance co-author, which does not count).
  • arXiv ID: 2609.02253 (title match found 2026-10-29).
  • Key idea: dual memory plus reward-guided skill distillation for deep research.
  • Results: beats GPT-5.4 by 14.7.
  • Search relevance: high. The numbers come from a truncated log and should be re-verified.
  • Moved to watchlist 2026-10-29 from: (top level).

2607.28272 · MemHarness

  • Found: digest 08-03. Sent: digest.
  • Source: other (Shanghai AI Lab; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 17 upvotes, 6 citations, below the bar).
  • Key idea: agents critique and reconstruct memories instead of replaying them verbatim. Raw memory injection actually hurts. Trained with GRPO.
  • Results: +8.8% ALFWorld; +9.5% WebShop.
  • Moved to watchlist 2026-10-30 from: main section. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

RSI: Skill Evolution

Self-improvement through a growing or evolving library of skills (procedures, SOPs, programs), including skill-policy co-evolution and transfer.

The field legend is in rsi-harness-evolution.md.

Skill libraries for search

2603.02766 · EvoSkill

  • Found: digest 05-05. Sent: digest.
  • Source: high-discussion (116 Semantic Scholar citations as of 2026-10-29). Affiliation: Sentient/Virginia Tech.
  • Key idea: self-evolving skills derived from failure analysis, with model weights frozen.
  • Results: +12.1% SealQA; zero-shot transfer to BrowseComp +5.3%.
  • Search relevance: high. Evolved skills tested on search benchmarks.

Skill generation without ground truth

2604.01687 · CoEvoSkills

  • Found: digest 05-03. Sent: digest.
  • Source: high-discussion (143 Semantic Scholar citations as of 2026-10-29). Affiliation: UIC/MBZUAI.
  • Key idea: a generator and a verifier co-evolve to build multi-file skill packages without ground truth.
  • Results: highest SkillsBench scores; transfers across 8 LLMs.

2604.27660 · Ctx2Skill

  • Found: digest 05-18. Sent: digest.
  • Source: high-discussion (HF 73 upvotes).
  • Key idea: Challenger/Reasoner/Judge self-play; cross-time replay prevents collapse.

2608.27454 · WikiSkill

  • Found: digest 09-01. Sent: digest.
  • Source: Google (Google Research; arXiv affiliations).
  • Key idea: traces → wiki → skills.
  • Results: small models with evolved skills beat larger models without them; transfers across models.

2607.22529 · Skill Self-Play

  • Found: 2026-09-09. Sent: unknown.
  • Source: Alibaba (Qwen; arXiv affiliations).

2607.26643 · SkillBoost

  • Found: 2026-09-09. Sent: unknown.
  • Source: Alibaba (arXiv affiliations).
  • Key idea: tackles skill overfitting with an exploit / prior-guided explore / verify loop.

2610.11781 · Skill-V

  • Found: 2026-10-14. Sent: yes (minor note).
  • Source: Kuaishou (arXiv affiliations).
  • Key idea: turns stored skills into testable contracts that are revised when they fail.

Failure modes and transfer (cautionary)

2610.20845 · Evolving Together, Failing Apart (Alibaba)

  • Found: 2026-10-17. Sent: yes.
  • Source: Alibaba (arXiv affiliations).
  • Key idea: when skills and policy co-evolve, each overfits to the other.
  • Results:
    • When the skill bank drifts, 52–93% of the gain is lost.
    • Worst case is 5–13 points below peak.
    • Covers SkillRL, ReSkill, Evolving-RL and ARISE.
  • Search relevance: a direct warning for SESA, ERSkill, CoEvo-Mem and SCAFFOLD.

2604.24594 · SRA: Skill Retrieval Augmentation

  • Found: digest 04-30. Sent: digest.
  • Source: ByteDance (Tsinghua + ByteDance; arXiv affiliations; 58 citations).
  • arXiv ID: 2604.24594 (title match found 2026-10-29).
  • Key idea: retrieval from a corpus of 26K skills works, but agents suffer "skill-loading hallucination": they cannot decide when to use a skill.

2610.36077 · The Trinity: Capability-Driven RSI for Multimodal Agents

  • Found: 2026-10-30. Sent: yes.
  • Source: high-discussion (HF Daily Papers 10-28, 389 upvotes). Last author Yang You; HF lists NUS, and the arXiv page prints no affiliations. Not a listed lab.
  • Key idea: reorganize tasks into a capability tree (different modalities can share one capability), then evolve reusable modules with three roles: Creator, Explorer and Evolver. "Capability-Driven Real2Sim2Real" abstracts shared requirements into simplified sims, learns there, and validates transfer on the original tasks.
  • Results: +34 points across 28 training benchmarks and +26 points across 18 unseen benchmarks; includes 2 real-world embodied tasks.
  • Search relevance: indirect (multimodal/embodied). The relevant part is the capability-keyed module library with a held-out benchmark split, which is the transfer evidence T8 skill libraries usually lack.

Watchlist (low signal)

Entries whose Source is other or unverified under hao's 2026-10 inclusion rule (major lab or high discussion). Kept for reference and not deleted. Do not cite them as primary evidence for trends or proven solutions.

2607.29468 · SESA

  • Found: 2026-09-09; sent 2026-09-23. Sent: yes.
  • Source: other (UCAS/PKU/Mininglamp).
  • Key idea: self-play plus skill memory for search agents.
  • Search relevance: high. Exposed to the co-evolution trap described in 2610.20845.
  • Moved to watchlist 2026-10-29 from: Skill libraries for search.

2607.05297 · MetaSkill-Evolve

  • Found: baseline; confirmed real 09-25. Sent: unknown.
  • Source: other (LMU).
  • Key idea: meta-skills that evolve the skill-creation process itself.
  • Search relevance: one of hao's tracked themes (meta-skill evolution for search).
  • Moved to watchlist 2026-10-29 from: Skill libraries for search.

2609.05511 · SCAFFOLD

  • Found: 2026-09-29. Sent: yes.
  • Source: other (MBZUAI).
  • Key idea: recursive parametric skill abstraction for web agents, followed by distillation into the weights.
  • Results: +11.1–17.2 from cross-site transfer.
  • Search relevance: high (web).
  • Moved to watchlist 2026-10-29 from: Skill libraries for search.

2608.12720 · ERSkill

  • Found: 2026-10-02. Sent: yes.
  • Source: other (CUHK-SZ).
  • arXiv ID: 2608.12720 (title match found 2026-10-29).
  • Key idea: an evolvable library of retrieval skills with a co-trained router.
  • Moved to watchlist 2026-10-29 from: Skill libraries for search.

2610.33768 · SkillQR

  • Found: 2026-10-28. Sent: logged.
  • Source: other (Emory).
  • Key idea: builds query-rewrite skills from the structure of the search corpus and picks one skill per query.
  • Results: roughly matches fine-tuned rewriters and holds up under domain shift, where they degrade.
  • Search relevance: high, but the skills are not improved from search results, so it is not RSI.
  • Moved to watchlist 2026-10-29 from: Skill libraries for search.

2604.14572 · Corpus2Skill

  • Found: digest 05-03. Sent: digest.
  • Source: other (MTRI).
  • Key idea: "don't retrieve, navigate": compiles a corpus into a skill tree.
  • Results: beats dense retrieval, RAPTOR and agentic RAG on enterprise QA; highest F1 on RAGBench.
  • Moved to watchlist 2026-10-29 from: Skill libraries for search.

2604.27221 · Web2BigTable

  • Found: digest 05-04. Sent: digest.
  • Source: other (UCL corresponding author; 2 Huawei co-authors; HF 39 upvotes).
  • Key idea: bi-level orchestrator plus workers for internet-scale extraction. A shared workboard handles coordination, and all learning goes through external skill memory.
  • Results: 7.5x success rate.
  • Moved to watchlist 2026-10-29 from: Skill libraries for search.

2610.24630 · SELM

  • Found: 2026-10-23. Sent: logged.
  • Source: other (RUC).
  • Key idea: a skill-library manager that ranks skills by usefulness and prunes old ones. No search tasks.
  • Moved to watchlist 2026-10-29 from: Skill generation without ground truth.

2610.33024 · Skill Adaptation

  • Found: 2026-10-27. Sent: yes.
  • Source: other (UCAS; HF 31 upvotes).
  • Key idea: the same skills gave GPT-5.6-luna +31.1 and cost Gemma-4-E4B 8.8. Rewriting each skill for the target model helps.
  • Results: +2–5 across 7 models; +5–14 for weak models.
  • Search relevance: high. Skills evolved by a frontier model may hurt a small search model.
  • Moved to watchlist 2026-10-29 from: Failure modes and transfer (cautionary).

2607.05202 · EvoAgentBench (EMNLP 2026 Findings)

  • Found: 2026-10-26. Sent: logged.
  • Source: other (Shanda).
  • arXiv ID: 2607.05202 (title match found 2026-10-29).
  • Key idea: tests whether skills carry over across task types, including web research.
  • Results: hand-written skills helped in every setup; evolved skills were inconsistent.
  • Moved to watchlist 2026-10-29 from: Failure modes and transfer (cautionary).

2608.20274 · Break It Down, Pass It On

  • Found: 2026-10-04. Sent: yes.
  • Source: other (Stony Brook).
  • arXiv ID: 2608.20274 (title match found 2026-10-29).
  • Key idea: skill granularity determines transfer; a caution.
  • Moved to watchlist 2026-10-29 from: Failure modes and transfer (cautionary).

2605.17734 · HASP (Skill Programs)

  • Found: digest 06-01. Sent: digest.
  • Source: other (NYU + Salesforce AI Research; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 34 upvotes, below the bar; citations not re-checked, Semantic Scholar rate-limited).
  • Key idea: skills are executable program functions, not passive text.
  • Results: +25% on web-search reasoning; +30.4% with post-training.
  • Search relevance: high.
  • Moved to watchlist 2026-10-30 from: Skill libraries for search. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

2604.20087 · SkillLearnBench

  • Found: digest 04-30. Sent: digest.
  • Source: other (CMU + Amazon AGI; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 14 upvotes, 58 citations, below the bar).
  • arXiv ID: 2604.20087 (title match found 2026-10-29).
  • Key idea: a continual skill-learning benchmark. Self-feedback causes "recursive drift".
  • Results: best method 30.44% vs 74.50% for humans.
  • Moved to watchlist 2026-10-30 from: Failure modes and transfer (cautionary). Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

2610.35434 · Skill∞: Recursive Self-Improvement for Agent Skill Optimization

  • Found: 2026-10-30. Sent: logged.
  • Source: other (the arXiv page prints no affiliations; no HF trending evidence found).
  • Key idea: a meta-skill (diagnose traces, propose candidates, request independent review, allocate search budget) optimizes task skills, and the meta-skill is improved by the same loop. Skills stay standard, discoverable packages.
  • Results: SkillsBench (83 tasks) pass rate 60.7%, vs 56.4% for human-curated skills and 54.5% for SkillEvolver.
  • Search relevance: evolve-the-evolver (T7) applied to skills; no search benchmark.

RSI: Weight-Level Training, Co-Training, and Theory

Covers self-improvement that changes model weights (self-play, self-distillation, harness and model co-training), theory and limits, failure analyses, and benchmarks for RSI itself.

The field legend is in rsi-harness-evolution.md.

Weights route / co-training

2610.25078 · CoEvo-RSI

  • Found: 2026-10-25. Sent: logged.
  • Source: high-discussion (HF 92 upvotes). Affiliation: NUS/A*STAR/SenseTime.
  • Key idea: retrains the model alongside its evolving harness, so the model can do what the improved harness asks of it.
  • Results: strong, though the numbers were not recovered. Not tested on search.

2607.28568 · Frontis-MA1 + OpenRSI

  • Found: digest 08-10. Sent: digest.
  • Source: high-discussion (HF 183 upvotes).
  • Key idea: a 35B model trained on Draft/Improve/Debug/Crossover operators. The whole stack is open source.
  • Results: 71.2% on MLE-Bench Lite on a single RTX 4090.

2609.14858 · Dream-RSI (Google DeepMind)

  • Found: 2026-09-24. Sent: yes.
  • Source: Google DeepMind (arXiv affiliations); also high-discussion (HF 233 upvotes).
  • arXiv ID: 2609.14858 (title match found 2026-10-29).
  • Key idea: a dreaming/replay simulator that tests self-modification candidates offline before they go live.
  • Note: lab attribution verified 2026-10-29 from arXiv affiliations (Google DeepMind).

2610.20431 · SwarmRSI

  • Found: 2026-10-18. Sent: logged.
  • Source: Huawei (arXiv affiliations).
  • Key idea: a self-improving swarm of scientific research agents. Covers science tasks, not search.

2610.08077 · Self-Retrospection Distillation

  • Found: digest 10-19. Sent: digest.
  • Source: high-discussion (HF 190 upvotes). Affiliation: Salesforce AI Research (arXiv affiliations), which is not a major lab under the 2026-10-30 list; stays in main on discussion alone.
  • Key idea: gets a training signal from RL runs in which every attempt fails.

2605.27276 · SIA

  • Found: 2026-09-25. Sent: yes (chat).
  • Source: high-discussion (2.2K GitHub stars). Affiliation: Hexo AI.

Theory and limits

2606.21090 · Self-Improvement Can Self-Regress

  • Found: 2026-09-25. Sent: yes (chat).
  • Source: Meta (Meta AI; arXiv affiliations).

2610.25858 · Are Self-Evolving Agents Continual Learners?

  • Found: 2026-10-20. Sent: yes.
  • Source: Google DeepMind (arXiv affiliations).
  • Key idea: runs ReasoningBank, GEPA and Meta-Harness on drifting BIRD, BFCL and ScienceWorld.
    • Edits to the global prompt interfere with new tasks and overwrite what was learned.
    • Similarity-based memory holds up better but suffers retrieval collisions.
  • Design rules derived for search:
    • Store tips as tagged skills.
    • Retrieve a skill only when it clearly applies.
    • Evaluate on a sequence of domains, then re-test the old ones.

2610.12822 · Reward Hacking Is Transmissible

  • Found: digest 10-19. Sent: digest.
  • Source: Meta (Meta Superintelligence Labs; arXiv affiliations).
  • Key idea: test-gaming spreads between agents.

Benchmarks for RSI

2610.08215 · Learn2Play Bench

  • Found: digest 10-19. Sent: digest.
  • Source: high-discussion (HF 60 upvotes). Affiliation: NUS.
  • Key idea: tests whether agents learn from experience.

Meta-search / RSI-in-search crossovers (weights)

2607.07820 · DeepSearch-World / DeepSearch-Evolve

  • Found: 2026-09-28. Sent: yes.
  • Source: Tencent (arXiv affiliations); also high-discussion (HF 94 upvotes).
  • Key idea: self-distillation with no teacher for deep-search agents.
  • Results: +23.8 / +37.6 / +48.1 on BrowseComp / GAIA / HotpotQA.
  • Search relevance: very high. Weight-level RSI applied to search.

2606.04507 · Self-Evolving Deep Research

  • Found: 2026-09-11. Sent: unknown.
  • Source: ByteDance (equal-contribution second author at ByteDance; arXiv affiliations).
  • Key idea: shows policy collapse under unverifiable rewards.
  • Search relevance: high (a caution).

2608.24794 · CAFE

  • Found: 2026-09-10; sent 09-26. Sent: yes.
  • Source: Tencent (arXiv affiliations).
  • Key idea: co-evolves the agent and its critic.
  • Results: beats RL search agents on 7 benchmarks.

2608.01822 · SearchMaster

  • Found: 2026-10-01. Sent: yes.
  • Source: JD (JD.com, from the arXiv source; JD counts as Chinese big tech under hao's 2026-10-30 list).
  • arXiv ID: 2608.01822 (title match found 2026-10-29).
  • Key idea: self-play search training with depth and anti-shallow-browsing regulators.
  • Promoted from watchlist 2026-10-30: JD is on hao's final major-lab list.

2610.13912 · RSI-Anything: Recursive Self-Improvement with World Models

  • Found: 2026-10-30. Sent: yes.
  • Source: high-discussion (HF trending, 52 upvotes). Authors include Wanli Ouyang, Pheng Ann Heng and Ling Yang; the arXiv page prints no affiliations, and no listed lab is confirmed. Code: github.com/Gen-Verse/RSI-Anything.
  • Key idea: turn interaction experience into persistent improvement via a world model that is adapted as the policy shifts, so predictions stay reliable.
  • Results: code repair 26.3→31.2%; GUI tasks have higher completion with 39.4% fewer actions; lab planning 21.80→46.01%; reaction optimization best yield 96.15% vs 82.16%.
  • Search relevance: indirect. No search benchmark.

Watchlist (low signal)

Entries whose Source is other or unverified under hao's 2026-10 inclusion rule (major lab or high discussion). Kept for reference and not deleted. Do not cite them as primary evidence for trends or proven solutions.

2609.26457 · RSI of AI research agents

  • Found: 2026-09-25 (ad hoc, for hao's plateaued proposer). Sent: yes (chat).
  • Source: other (Weco; HF 28 upvotes).
  • Results:
    • Reward hacking fell 55%→39%→32%.
    • AIDE0→AIDE85.
  • Moved to watchlist 2026-10-29 from: Weights route / co-training.

AIDE² (WecoAI blog, 2026-07-14; no arXiv ID recovered)

  • Found: 2026-09-26. Sent: yes.
  • Source: other (WecoAI blog; no paper).
  • Key idea: an offline, scored ML-engineering RSI loop; 8 days and 100 outer loops.
  • Results: Level 2 results were mixed.
  • Compare: Harness², which arrived about 3 months later.
  • Moved to watchlist 2026-10-29 from: Weights route / co-training.

ID not recovered · ROFT

  • Found: 2026-09-30. Sent: yes.
  • Source: unverified (no arXiv ID found).
  • Key idea: RL-free self-retrospection.
  • Moved to watchlist 2026-10-29 from: Weights route / co-training.

2508.07809 · EvoCoT

  • Found: 2026-09-25. Sent: yes (chat).
  • Source: other (PKU).
  • Results: saturates after 1–2 iterations.
  • Moved to watchlist 2026-10-29 from: Weights route / co-training.

2610.13870 · ARIA (ETH)

  • Found: digest 10-12. Sent: digest.
  • Source: other (ETH Zurich).
  • Key idea: a 24-state controller learns when a cheap model should call a stronger advisor.
  • Results:
    • Closes 44–81% of the gap at 34–48% of the cost.
    • SWE-bench Verified 18→48.
  • Search relevance: medium. A routing pattern for deciding between cheap and frontier models on search steps.
  • Moved to watchlist 2026-10-29 from: Weights route / co-training.

2610.21646 · The Self-Improvement Problem

  • Found: 2026-10-21. Sent: yes.
  • Source: other (DFKI).
  • Key idea: proves that no check can always reject bad changes while also never missing good ones. A chain of accepted changes can raise the measured score at every step while real performance falls.
  • Practical rule: re-test the whole chain of accepted changes on fresh queries.
  • Moved to watchlist 2026-10-29 from: Theory and limits.

2610.09856 · AnchorLoop; 2610.15772 · RSIPO

  • Found: 2026-10-20. Sent: yes (brief mention). Details not recovered.
  • Source: other (AnchorLoop: Waseda; RSIPO: Tsinghua/CUHK, HF 8 upvotes).
  • Moved to watchlist 2026-10-29 from: Theory and limits.

2608.20318 · AI4AI-Bench

  • Found: 2026-09-11. Sent: unknown.
  • Source: other (Einsia/Tsinghua).
  • Results: mean 0.166; best is Claude Opus 5 at 0.250.
  • Moved to watchlist 2026-10-29 from: Benchmarks for RSI.

2608.03392 · Self-Evolving Coding Agents survey (NJU)

  • Found: digest 08-24. Sent: digest.
  • Source: other (NJUST/NJU).
  • Key idea: a taxonomy of what evolves, when, and what drives it.
  • Moved to watchlist 2026-10-29 from: Benchmarks for RSI.

ICLR 2026 RSI workshop

  • Found: 2026-09-09. Sent: unknown. A venue signal.
  • Source: unverified (workshop, no paper ID).
  • Moved to watchlist 2026-10-29 from: Benchmarks for RSI.

ID not recovered · SCORE

  • Found: 2026-10-05. Sent: yes.
  • Source: unverified (no arXiv ID found).
  • Key idea: the evaluator and solver co-evolve with shared parameters, for deep research without ground truth.
  • Moved to watchlist 2026-10-29 from: Meta-search / RSI-in-search crossovers (weights).

2606.00590 · Critic-R (Critic-R-Zero, Critic-Embed)

  • Found: 2026-09-09. Sent: unknown.
  • Source: other (UMass).
  • Moved to watchlist 2026-10-29 from: Meta-search / RSI-in-search crossovers (weights).

2610.12176 · MASS

  • Authors: Matei Zaharia, Yujin Tang et al.
  • Found: 2026-10-14. Sent: yes.
  • Source: other (UC Berkeley / Sakana AI; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 4 upvotes).
  • Key idea: alternates between evolving multi-agent workflows and fine-tuning the model on its own multi-agent traces.
  • Results: 1.2–1.6x better performance per output token.
  • Search relevance: the planner-plus-subagents shape matches deep research.
  • Moved to watchlist 2026-10-30 from: Weights route / co-training. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

2606.09498 · Self-Harness

  • Found: 2026-09-25. Sent: yes (chat).
  • Source: other (Shanghai AI Lab; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 3 upvotes).
  • Moved to watchlist 2026-10-30 from: Weights route / co-training. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

2610.33624 · RSI Arena: Recursive Self-Improvement with Human Feedback

  • Found: 2026-10-30. Sent: logged.
  • Source: other (the arXiv page prints no affiliations; HF Daily Papers 39 upvotes).
  • Key idea: ten frontier agents each post-train a Nemotron 3.5 30B-A3B base model with 1,000 GPU-hours and $300 of API credits, then refine using human feedback.
  • Results: the best (trained by GPT-6 Astra) closes a median 76% of the base-to-Instruct gap on 12 benchmarks. Stage 2 improves preference ratings for 8/10 agents, but benchmark gains correlate weakly with them; one model got more harmful compliance on StrongREJECT.
  • Search relevance: none directly; RSI-evaluation methodology.

Agentic Search: Context Management, Deep-Research Architecture, Benchmarks

The field legend is in rsi-harness-evolution.md.

Context and state management

2608.26263 · SKILL.state

  • Found: digest 09-01. Sent: digest.
  • Source: Google (Google LLC; arXiv affiliations).
  • Key idea: replaces the append-only history with skill spec + state + observation, discarding intermediate reasoning.
  • Results: better accuracy with fewer tokens. EMNLP.

2604.17091 · GenericAgent

  • Found: digest 05-04. Sent: digest.
  • Source: high-discussion (14.3K GitHub stars).
  • Key idea: context density matters more than context length. Four mechanisms: atomic tools, hierarchical memory, trajectory→SOP self-evolution, and compression.

Codex harness open-sourced (OpenAI, industry)

  • Source: OpenAI (content still unverified; see industry-practice.md).
  • Cross-listed in industry-practice.md.
  • Results: context compaction took ARC-AGI-3 from 13.3% to 38.3% with 6x fewer tokens.

Benchmarks

2610.03574 · HyperBrowseComp

  • Found: 2026-10-13. Sent: yes (minor).
  • Source: high-discussion (HF 58 upvotes). Affiliation: MBZUAI.
  • Key idea: a multimodal browsing benchmark in 13 languages.

BrowseComp / BrowseComp-Plus / GAIA / GAIA2 / xBench / DeepSearchQA / WideSearch / WebSailor

  • Source: mixed lab benchmarks (BrowseComp is from OpenAI and GAIA from Meta/HF; the others were not checked one by one).
  • The de facto evaluation set for papers in this knowledge base. BrowseComp-Plus uses a fixed corpus, so it suits reproducible harness evolution.

Watchlist (low signal)

Entries whose Source is other or unverified under hao's 2026-10 inclusion rule (major lab or high discussion). Kept for reference and not deleted. Do not cite them as primary evidence for trends or proven solutions.

2604.13018 (candidate) · AiScientist File-as-Bus

  • Found: digest 05-02. Sent: digest.
  • Source: other (RUC/AweAI; HF 34 upvotes).
  • arXiv ID: 2604.13018 (candidate; matched by arXiv abstract/title search 2026-10-29, confirm before citing).
  • Key idea: long-horizon research handled as state management through files.
  • Results: removing the file protocol loses 60% of the gain; +10.5 PaperBench.
  • Moved to watchlist 2026-10-29 from: Context and state management.

2609.02253 · APEx

  • Source: other (USTC/CAS; a single ByteDance co-author, which does not count).
  • arXiv ID: 2609.02253 (title match found 2026-10-29).
  • Cross-listed in rsi-memory-evolution.md. Dual memory for deep research.
  • Moved to watchlist 2026-10-29 from: Deep-research architectures.

2508.12752 · Deep Research survey

  • Found: digest 08-24. Sent: digest.
  • Source: other (CityU; one minor Huawei co-author; 77 citations, below the bar).
  • Key idea: a four-stage pipeline: Planning → Question Developing → Web Exploration → Report Generation.
  • Moved to watchlist 2026-10-29 from: Deep-research architectures.

2610.12419 · OneSearch-VL

  • Found: 2026-10-15. Sent: skipped. A multimodal deep-research agent; not RSI.
  • Source: other (HF 24 upvotes).
  • Moved to watchlist 2026-10-29 from: Deep-research architectures.

2610.12176 · MASS

  • Source: other (UC Berkeley / Sakana AI; arXiv affiliations. Not a major lab under the 2026-10-30 list; HF 4 upvotes).
  • Cross-listed in rsi-training-and-theory.md. Planner plus subagents.
  • Moved to watchlist 2026-10-30 from: Deep-research architectures. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

Agentic Search: Retrieval Programs, Query Refinement, Retrievers

Covers the retrieval interface: retrieval programs, query rewriting and refinement, retrievers and rerankers, and the BM25 vs LLM-retriever evidence.

The field legend is in rsi-harness-evolution.md.

Programmable / stateful retrieval

2610.24854 · RPL: Learning to Program Retrievals

  • Authors: KAIST AI and Meta Superintelligence Labs.
  • Found: 2026-10-24; deep dive 10-23 in chat. Sent: yes.
  • Source: Meta (Meta Superintelligence Labs + KAIST; arXiv affiliations).
  • Key idea:
    • Qwen3-4B/8B write Python over a retrieval SDK (BM25, E5, RRF, dedup, rerank) with persistent state.
    • Training loop: RL → mutation of weak programs with a typed grammar → SFT distillation → RL.
  • Results:
    • Average EM on 7 QA datasets: 4B 0.515 vs Search-R1 0.469; 8B 0.531 vs 0.494. The 4B model beats every 8B baseline.
    • GlobalQA F1 19.11 vs MARAG-R1 12.08.
    • Trains 5.8x faster than Search-R1.
    • Ablation: 0.473 → 0.478 → 0.487 → 0.515.
    • Needs 8xH100.
  • Search relevance: core. A FunSearch-like mutation step distilled into the policy, which makes it RSI-adjacent.
  • Detail: /workspace/agent/paper-digests/deep-dives/rpl-retrieval-programs.md

2610.06689 · PSA (Programmatic Search Agents)

  • Found: 2026-10-13. Sent: yes.
  • Source: Tencent (Tencent Yuanbao; arXiv affiliations).
  • Key idea: each search action is a small program run over a persistent candidate set.
  • Results: +7.6 on BrowseComp-Plus; 28–34% fewer tokens at the final step.

2610.05107 · SearchJev

  • Found: 2026-10-13. Sent: logged.
  • Source: Huawei (arXiv affiliations).
  • Key idea: a fast, separate model makes the search decisions. Related to hao's JevHarness interest.

Query refinement

2610.21512 · QREF

  • Found: 2026-10-19. Sent: yes.
  • Source: Tencent (Fudan + Tencent; arXiv affiliations).
  • Key idea: a plug-in query refiner and evidence filter in front of a frozen deep-search agent. The refiner is trained with SFT, then RL against live web search.
  • Results:
    • +33.6% relative Pass@1 on WebSailor.
    • +11% on Tongyi DeepResearch.
    • 95% fewer query batches that lack an anchor.
  • Search relevance: core. Query writing can be improved independently of the agent, so it is the first module to give its own improvement loop.

Retrievers, BM25, rerankers

2607.26497 · BM25 Wins at Scale

  • Found: baseline / digest 08-03. Sent: digest.
  • Source: high-discussion (HF 53 upvotes).
  • Key idea: at enterprise scale (512K docs), plain BM25 beats graph RAG and agentic search.
  • Results: the agent used 39x more tokens. The best setup is an agent plus a BM25 hybrid.

2610.26831 · AGR

  • Found: 2026-10-23. Sent: logged; off-topic (recommender retrieval).
  • Source: Kuaishou (arXiv affiliations).

Watchlist (low signal)

Entries whose Source is other or unverified under hao's 2026-10 inclusion rule (major lab or high discussion). Kept for reference and not deleted. Do not cite them as primary evidence for trends or proven solutions.

2610.07960 · IndexAct

  • Found: 2026-10-13. Sent: yes.
  • Source: other (Yonsei).
  • Key idea: the agent narrows candidates with set operations on the index, which return counts instead of passages. It reads source text only at the end.
  • Moved to watchlist 2026-10-29 from: Programmable / stateful retrieval.

2610.33768 · SkillQR

  • Source: other (Emory).
  • Cross-listed in rsi-skill-evolution.md.
  • Key idea: corpus-structure query-rewrite skills that hold up under domain shift.
  • Moved to watchlist 2026-10-29 from: Query refinement.

2605.02411 · FitText

  • Found: digest 05-05. Sent: digest.
  • Source: other (UCLA).
  • Key idea: evolves tool queries mid-execution, using pseudo-tool descriptions and evolutionary selection.
  • Results: rank 8.81→2.78 on 43K tools; +24pp on StableToolBench.
  • Moved to watchlist 2026-10-29 from: Query refinement.

2602.05975 · SAGE benchmark (BM25 vs LLM retrievers)

  • Found: 2026-10-06. Sent: yes.
  • Source: other (NYU/Yale).
  • arXiv ID: 2602.05975 (title match found 2026-10-29).
  • Key idea: for deep-research agents BM25 beats LLM retrievers, because the agents' queries stay keyword-shaped.
  • Note: different from SAGE graph memory 2605.12061.
  • Moved to watchlist 2026-10-29 from: Retrievers, BM25, rerankers.

2610.06782 · T-Search

  • Found: 2026-10-16. Sent: logged.
  • Source: other (T-Tech).
  • Key idea: an open-weight multi-round retriever that comes with a test environment. Could serve as a cheap fixed testbed for evolved harnesses.
  • Moved to watchlist 2026-10-29 from: Retrievers, BM25, rerankers.

2610.11922 · Project Greenhouse

  • Found: digest 10-19. Sent: digest.
  • Source: other (Waterloo).
  • Key idea: an open reranker for agentic search.
  • Moved to watchlist 2026-10-29 from: Retrievers, BM25, rerankers.

Artificial Analysis Search API benchmark (industry benchmark; no arXiv)

  • Found: baseline. Sent: unknown.
  • Source: other (independent benchmark company; no paper).
  • Results: an 8pp spread in quality and a 35x latency gap across search APIs.
  • Moved to watchlist 2026-10-29 from: Retrievers, BM25, rerankers.

2610.10955 · query rewriting (title not recovered)

  • Found: 2026-10-24. Sent: logged.
  • Source: other (Amazon/CMU; arXiv affiliations. Not a major lab under the 2026-10-30 list; no HF Daily Papers page found; 2 weeks old, so citations cannot reach the bar).
  • Moved to watchlist 2026-10-30 from: Query refinement. Reason: lab not on hao's final major-lab list (2026-10-30) and no high-discussion signal.

Agentic Search: RL-Trained Search Policies

Covers search agents trained with RL, including reward design, over-searching control, and verify-and-patch policies.

The field legend is in rsi-harness-evolution.md.

2607.21461 · AREX (v1 and v2)

  • Found: baseline; sent 2026-09-27; v2 noted 09-11. Sent: yes.
  • Source: high-discussion (HF 124 upvotes). Affiliation: BAAI (arXiv affiliations), which is not a major lab under the 2026-10-30 list; stays in main on discussion alone.
  • Key idea: claim-level verify-and-patch, instead of linear search.
  • Results:
    • +22.9 on BrowseComp over linear search.
    • v2 was trained at 4B and 122B-A10B and evaluated on BrowseComp, WideSearch, DeepSearchQA and HLE.
  • Search relevance: core.

2606.09730 · SearchSwarm

  • Found: 2026-09-11. Sent: unknown.
  • Source: Ant Group (arXiv affiliations); also high-discussion (HF 56 upvotes).
  • Key idea: a swarm search agent with a 30B-A3B model.
  • Results: BrowseComp 68.1; BrowseComp-ZH 73.3.

2610.22331 · KnowGPO

  • Found: 2026-10-24. Sent: yes.
  • Source: Tencent (arXiv affiliations).
  • Key idea: a per-step reward for knowledge gained toward the answer.
  • Search relevance: high. The same reward could also score evolved search strategies.

2610.10179 · retrieval credit assignment (title not recovered)

  • Found: 2026-10-24. Sent: logged.
  • Source: Huawei (Huawei UK, majority of authors; arXiv affiliations).

2601.11888 · Agentic-R (ACL 2026 Findings)

  • Found: 2026-09-09. Sent: unknown.
  • Source: Baidu (RUC + Baidu; arXiv affiliations).
  • arXiv ID: 2601.11888 (title match found 2026-10-29).
  • Venue link: aclanthology.org/2026.findings-acl.785

2610.35013 · MPO: Multiscale Policy Optimization for Long-Horizon LLM Agents

  • Found: 2026-10-30. Sent: yes.
  • Source: high-discussion (HF Daily Papers 10-28, 382 upvotes). HF lists IQuest; the arXiv page prints no affiliations. Not a listed lab.
  • Key idea: critic values vary mainly across turns, so anchor optimization at turn level and allocate finer credit among tokens inside a turn. Uses Multiscale Advantage Estimation plus turn-level importance sampling.
  • Results: consistent gains over strong RL baselines on search, tool-use and automation benchmarks, at 4B and 30B-A3B. The abstract gives no headline number.
  • Search relevance: direct. Turn-level credit assignment for multi-turn search RL; relevant to T12 (process reward).

Watchlist (low signal)

Entries whose Source is other or unverified under hao's 2026-10 inclusion rule (major lab or high discussion). Kept for reference and not deleted. Do not cite them as primary evidence for trends or proven solutions.

2608.01597 · HindSearch

  • Found: 2026-10-01. Sent: yes.
  • Source: other (SCU/RIT).
  • arXiv ID: 2608.01597 (title match found 2026-10-29).
  • Key idea: replaces a binary reward with a critique conditioned on the gold answer in hindsight.
  • Moved to watchlist 2026-10-29 from: (top level).

2605.29796 · SAAS

  • Found: 2026-10-02. Sent: yes.
  • Source: other (Xiamen; HF 21 upvotes).
  • arXiv ID: 2605.29796 (title match found 2026-10-29).
  • Key idea: self-aware RL against over-searching.
  • Moved to watchlist 2026-10-29 from: (top level).

2604.17337 · AutoSearch

  • Found: baseline / digest 05-01. Sent: digest.
  • Source: other (CASIA).
  • Key idea: RL-adaptive search depth; a self-answering mechanism learns when to stop retrieving.
  • Moved to watchlist 2026-10-29 from: (top level).

2607.10463 · GRASP

  • Found: baseline / digest 07-20. Sent: digest.
  • Source: other (UMass/Adobe).
  • Key idea: an RL agent coordinates semantic search, keyword search and paragraph reading at different granularities.
  • Moved to watchlist 2026-10-29 from: (top level).

2604.17555 · CoSearch

  • Found: digest 04-30. Sent: digest.
  • Source: other (UMass + Snap; the Snap authors are minor).
  • arXiv ID: 2604.17555 (title match found 2026-10-29).
  • Key idea: joint GRPO training of the reasoning agent and the document ranker.
  • Results: +6.6% over Search-R1.
  • Moved to watchlist 2026-10-29 from: (top level).

2610.05119 · LexiHorizon

  • Found: 2026-10-11. Sent: yes (minor note).
  • Source: other (360).
  • Key idea: long-horizon deep-search RL; not RSI.
  • Moved to watchlist 2026-10-29 from: (top level).

2605.14306 · PaSaMaster

  • Found: 2026-09-25. Sent: yes.
  • Source: other (SJTU).
  • Key idea: separates intent, handled by a frontier LLM, from retrieval and scoring, handled by cheap models. Only verified papers are scored.
  • Search relevance: high. An architecture pattern for cost-split paper search.
  • Moved to watchlist 2026-10-29 from: (top level).

2610.34714 · SearchHorizonBench: A Benchmark for Search Value in AI Agents

  • Found: 2026-10-30. Sent: logged.
  • Source: other (HF lists CMU; HF Daily Papers 11 upvotes).
  • Key idea: estimate the marginal value of more search before doing it, rather than only estimating question difficulty.
  • Results: 45,632 executions over 3,168 questions (1,632 held out). Historical calibration cuts Brier error by 16.2%; search gain has an SD of 21.4 points; existing estimators explain <2% of variance.
  • Search relevance: direct to over-search control (T12).

Chronological Log

Newest entries are at the bottom. Tags:

  • sent = hao was messaged.
  • logged = recorded only.
  • empty = no new findings.
  • failed = the run errored.
  • GAP = unrecoverable.

Sources used for the reconstruction:

  • archive/tracker-2026-09.md (09-08 to 09-25)
  • /workspace/agent/tasks/t-455e1f.md (task run log, 09-23 to 10-29, each entry truncated at about 500 characters)
  • conversations/*.md
  • the chat DB (inbound/outbound)
  • paper-digests/ (weekly digest, from 2026-04-29)

Note: from 09-23 onward the task prompt saved to /workspace/group/rsi-agentic-search/tracker.md, a path that does not exist. As a result the daily entries for 09-26 to 10-29 never reached the tracker file in this workspace, and this log was rebuilt from the run log and chat records.

Before the daily tracker (weekly paper digest)

  • 2026-04-29 to 2026-09-07: the weekly digest (paper-digests/index.md) collected the early related work: AutoSearch, CoSearch, EvoSkill, Agentic Harness Engineering, HASP, ALMA, SAGE graph memory, BM25 Wins at Scale, JIT-Agent, HarnessEvolve, HoH and others. They are filed in the category files with Sent: digest.

2026-09

  • 09-08: setup. Baseline tables built: 8 RSI papers, 4 search items, 5 ideas. No new papers. Signals: OpenAI inference costs; Pachocki "An Alien Mind".
  • 09-09: SESA 2607.29468; Weng essay; Critic-R 2606.00590; RSEA 2606.28374; Agentic-R (ACL Findings); Skill Self-Play 2607.22529; SkillBoost 2607.26643; ICLR 2026 RSI workshop.
  • 09-10: CAFE 2608.24794; HELIX 2608.13951; RHI 2607.15524; Meta^n 2608.24735; Trellis 2606.29823; Jerry Liu tweet.
  • 09-11: AI4AI-Bench 2608.20318; SearchSwarm 2606.09730; HarnessBank 2607.13683; Self-Evolving Deep Research 2606.04507; AREX v2.
  • 09-12: EVOHARNESSBENCH 2609.04280; Evo-Harness 2608.15071; Harness Updating ≠ Benefit 2605.30621; DeepMind RSI bet; Sakana RSI Lab.
  • 09-13 to 09-22: GAP. No tracker entries and no task runs. The install was migrated from NanoClaw v1 on 09-22, so whatever ran before then was not carried over.
  • 09-23 sent: Weng essay; SESA. This is the first run under the new task t-455e1f.
  • 09-24 sent: HarnessEvolve 2609.00829; DeepMind Dream-RSI (dreaming/replay simulator; ID not recovered).
  • 09-25 sent: PaSaMaster 2605.14306; SELF-INDEX (ID not recovered). Confirmed that MetaSkill-Evolve and Recuris are real.
  • 09-25 (ad hoc chat, about hao's proposer whose loss had plateaued): 2609.26457, 2606.21090, Self-Harness 2606.09498, EvoCoT 2508.07809, SIA 2605.27276.
  • 09-26 sent: CAFE; SE-Search (no ID); AIDE² (WecoAI).
  • 09-26 (chat): plateau advice to hao. Covered HarnessEvolve divergence, CAFE, SkillBoost, the RSEA gate, Meta^n, HELIX typed atoms, AlphaEvolve-style diffs, HarnessBank, staged unlocking and schema validation.
  • 09-27 sent: AREX 2607.21461 (BrowseComp +22.9).
  • 09-28 sent: DeepSearch-World / DeepSearch-Evolve 2607.07820.
  • 09-29 sent: SCAFFOLD 2609.05511.
  • 09-30 sent: CoEvo-Mem 2608.01739; ROFT (no ID).

2026-10

  • 10-01 sent: SearchMaster, HindSearch (no IDs); Harrison Chase post.
  • 10-02 sent: ERSkill, SAAS (no IDs).
  • 10-02 (chat): JevHarness and TypeSafe System One discussed for query rewriting.
  • 10-03 sent: SelfSearch (no ID).
  • 10-04 sent: APEx; Break It Down, Pass It On (no IDs).
  • 10-05 sent: EvolveMem; SCORE (no IDs).
  • 10-06 sent (logged 10-07 00:26): SAGE benchmark; VERSE (no IDs).
  • 10-07 to 10-10: GAP / failed. No run entries for 10-07 to 10-09. The 10-10 run failed with "API Error 400: Claude Code 2.1.257 does not support this model; 2.1.280 required." Papers posted in this window were partly caught by later runs (MESH-Harness, LexiHorizon).
  • 10-11 sent: MESH-Harness 2610.05300; RRSI 2609.24972. Minor: LexiHorizon 2610.05119.
  • 10-12 sent: PrisMem 2610.06361; ActiveSaddler 2610.00906. Minor: Causal Improvement Graph 2610.05039. Skipped GUI-HARVEST 2610.00948.
  • 10-12 (weekly digest): ARIA 2610.13870.
  • 10-13 sent: PSA 2610.06689; IndexAct 2610.07960. Minor: HyperBrowseComp 2610.03574. Logged: SearchJev 2610.05107.
  • 10-13 (chat deep dive): STOP, ADAS, DGM, AFlow, GEPA, Meta-Harness and MESH-Harness explained → paper-digests/deep-dives/meta-harness-vs-mesh.md.
  • 10-14 sent: Harness² 2610.14224; MASS 2610.12176. Minor: Skill-V 2610.11781. Skipped 2610.03548.
  • 10-15 logged: Self-Evolving Agent Harness 2610.18813 (theory only). Skipped OneSearch-VL 2610.12419.
  • 10-16 logged: T-Search 2610.06782.
  • 10-17 sent: Evolving Together, Failing Apart 2610.20845; Evo² 2610.13025. Minor: Harmony Harness 2610.15150.
  • 10-18 logged: RSIFlow 2610.21791; SwarmRSI 2610.20431.
  • 10-18 (chat deep dive): Harness² → deep-dives/harness2.md. Compared with AIDE².
  • 10-19 sent: QREF 2610.21512.
  • 10-19 (weekly digest): HERMES 2610.07832; Self-Retrospection Distillation 2610.08077; Learn2Play 2610.08215; Reward Hacking Is Transmissible 2610.12822; Retrieval Admissibility 2610.07309; Project Greenhouse 2610.11922; Anthropic Scion blog; Cohere Teams of Coding Agents.
  • 10-20 sent: Are Self-Evolving Agents Continual Learners? 2610.25858. Brief: RSIPO 2610.15772, AnchorLoop 2610.09856.
  • 10-20/21 (chat deep dive): RRSI → deep-dives/rrsi.md.
  • 10-21 sent: MemScope 2610.21170; The Self-Improvement Problem 2610.21646. Minor: Self-HPO 2610.22791. Logged: ClawLoop, RiseHarness (no IDs).
  • 10-22 empty: no new papers or posts.
  • 10-23 logged: SELM 2610.24630; AGR 2610.26831 (off-topic).
  • 10-23 (chat deep dive): RPL → deep-dives/rpl-retrieval-programs.md.
  • 10-24 sent: RPL 2610.24854; KnowGPO 2610.22331. Logged: 2610.10955 (query rewriting), 2610.10179 (retrieval credit assignment).
  • 10-25 logged: CoEvo-RSI 2610.25078; IDM 2610.13991. Two more papers were logged, but their names were cut off in the run log. Not recovered.
  • 10-26 logged: EvoAgentBench (EMNLP Findings). Three more papers were logged; names cut off. Not recovered.
  • 10-27 sent: Skill Adaptation 2610.33024. A second paper was sent (message id 111); name cut off. Not recovered.
  • 10-28 logged: SkillQR 2610.33768.
  • 10-29 sent: RuHarness 2610.36826.
  • 10-29 (chat): hao read two Anthropic articles ("fold subagents into main loop" and "anatomy of commerce agents"). Answer: broad search goes to a subagent; targeted or backend-ranked search is a tool. Added to industry-practice.md.
  • 10-29: knowledge base restructured into this folder. tracker.md archived to archive/tracker-2026-09.md.
  • 10-29: source-tier pass under hao's inclusion rule. Every entry got a Source field (lab verified from arXiv affiliations / high-discussion / other / unverified), and other/unverified entries moved to per-file watchlists. Recovered 21 arXiv IDs, 4 of them candidates. Fixed HELIX: it is HKU, not Sakana. Demoted trends T9 and T12 and five Tier 2 rows, and marked X3 as weaker evidence. Backup of the pre-pass state is in /tmp/kb-backup-20261029 (temporary).
  • 10-30: re-tier pass under hao's final major-lab list (rule 7 in index.md). Checked for partial edits from an interrupted earlier pass: the paper files, trends.md and proven.md were still in their 10-29 state and consistent, and the daily task t-455e1f prompt already had the new list. Demoted 11 distinct papers (12 entries) to watchlists: Amazon ×5 (2605.30621, VERSE, Evo-Harness, SkillLearnBench, 2610.10955), Salesforce ×2 (EVOHARNESSBENCH, HASP), Sakana ×2 (RHI, MASS), Shanghai AI Lab ×3 (HoH, MemHarness, Self-Harness). HF upvotes were rechecked for all of them (0–34; none reached the bar). Promoted SearchMaster 2608.01822 (JD) to main. Relabeled Self-Retrospection Distillation (Salesforce) and AREX (BAAI) as high-discussion; both stay in main. Moved Sakana, Salesforce and Cohere industry items to the industry watchlist. ClawLoop is still unverified: the arXiv page prints no affiliation, so Meituan is not confirmed. Counts: main 60 / watchlist 72 (was 71 / 61); industry 18 / 8 (was 21 / 5). trends.md: T12 promoted back to the main list and marked weaker evidence; T4 lowered from medium-to-strong to medium; T2, T6 and T11 lost some primary evidence but their conclusions are unchanged. proven.md: all Tier 1 kept, with P4 thinner; X1 marked weaker evidence; MASS moved to the Tier 2 watchlist. Backup in /tmp/kb-papers-backup-20261030 (temporary).
  • 10-30 sent: Memory Tax 2610.34073, HABIT-Bench 2610.24520 (memory); The Trinity 2610.36077 (skill); MPO 2610.35013 (search RL); RSI-Anything 2610.13912 (training). All five are high-discussion on HF; none is from a listed lab. Watchlist: Skill∞ 2610.35434, SearchHorizonBench 2610.34714, RSI Arena 2610.33624. Created papers/tech-blogs.md with DeepMind "no measurable acceleration", Google Deep Review, Anthropic source control (Scion entry now [verified]) and LangChain retrieval. Trends updated: T7 (DeepMind reality check), T8 (Trinity), T9 (Memory Tax / HABIT-Bench as baseline warnings), T12 (MPO, SearchHorizonBench). proven.md unchanged. Not filed: ReSight 2610.33487, DynaFlow 2610.34467, Economics of Reasoning Effort 2610.34491, ARIS 2605.03042 (off-axis). No posts from the tracked people.
  • 10-30 (chat): hao asked how each paper and blog relates to RSI and agentic search, and for a ranking of the ideas. Created relations.md (three relations: search as RSI target, search inside RSI, shared bottlenecks; tables per pipeline stage). ideas.md: added C1–C6 and a ranking table (top: C1 context-policy evolution 25, C2 budget-matched eval 21, B1 and C4 20, C3 19). trends.md: added T13 (context policy is the biggest lever for long-horizon search). proven.md: P3 production evidence, P7 trimming numbers, X3 marked disputed (SkillOpt).
  • 10-30 backfill: 98 blog entries merged into tech-blogs.md from archive/blog-backfill-raw; 92 main / 9 watchlist (101 entries total incl. the 4 existing; the Anthropic source-control post was merged into its existing entry; sections: search 17, context 25, single vs multi 9, RSI 18, agent RL 7, eval 6, other 10).