The Purpose I Write This Blog

   Amazing agent systems have been created to change our life. I want to keep to collect information about LLM-based agents and share my thoughts here.

Resource

Agentic RL

survey

• The Hitchhiker’s Guide to Agentic AI: From Foundations to Systems
• From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models
  • 知乎解读

Blogs

• 基座的 Agentic 能力是怎么训出来的
• Reasoning LLM（四）：Agentic RL

LLM-based agents

• RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
• RAGEN-2: Reasoning Collapse in Agentic RL
• rStar2-Agent: Agentic Reasoning Technical Report
• (GiGPO) Group-in-Group Policy Optimization for LLM Agent Training
• (ARPO) Agentic Reinforced Policy Optimization
• APPO: Agentic Procedural Policy Optimization
• (AEPO) Agentic Entropy-Balanced Policy Optimization
• Turn-PPO: Turn-Level Advantage Estimation with PPO for Improved Multi-Turn RL in Agentic LLMs

VLM-based agents

• Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning

Multi-agent

• advantage computation
  • Dr.MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems
  • (M-GRPO) Multi-Agent Deep Research: Training Multi-Agent Systems with M-GRPO
• MARL credit assignment
  • (COMA) Counterfactual Multi-Agent Policy Gradients
  • (CCPO) Counterfactual Credit Policy Optimization for Multi-Agent Collaboration
  • (SHARP) Who Deserves the Reward? SHARP: Shapley Credit-based Optimization for Multi-Agent System
    • 合作博弈中夏普利值（Shapley Value）的主要思想、公理及求解公式的理解
  • (C3) Exact Is Easier: Credit Assignment for Cooperative LLM Agents

Self evolution

• harness
  • Harness Engineering for Self-Improvement
• memory
  • blogs
    • Agent Memory Architectures: Vector vs Graph vs Episodic
    • ChatGPT ‘Dreaming V3’ Memory: Self-Updating AI Recall
    • Harness Comparison: How Claude Code, Cursor, Devin, and Antigravity Each Handle Memory
    • Give Your AI Agent Memory and Guardrails: Mem0 + Docker Sandboxes
    • (wiki) Agent Memory Benchmark Guidelines
    • 面试官皱眉：“你知道 Claude Code 记忆机制吗？” 我：“何止知道？我还看过源码”，他又愣了…
  • non-parameterized memory
    • Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning
    • Mem-α: Learning Memory Construction via Reinforcement Learning
    • MemCoRL: Alternating Co-Optimization of Memory Retrieval and Utilization via Collaborative Reinforcement Learning
    • MemBuilder: Reinforcing LLMs for Long-Term Memory Construction via Attributed Dense Rewards
    • Dreaming: Better memory for a more helpful ChatGPT
    • Codex
    • Doubao
    • AUTOMEM: Automated Learning of Memory as a Cognitive Skill
    • Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
    • (M3-Agent) Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory
    • MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
    • Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
      • Introducing The Token-Efficient Memory Algorithm
    • MemGPT: Towards LLMs as Operating Systems
  • parameterized memory
    • Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories
    • (Engram) Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
• meta-RL
  • Meta-RL Induces Exploration in Language Agents
• test-time scaling
• online learning

Others

• exploration

  • Beyond Stochastic Exploration: What Makes Training Data Valuable for Agentic Search

• experience replay

  • Efficient RL Training for LLMs with Experience Replay
  • Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learning
  • AgentHER: Hindsight Experience Replay for LLM Agent Trajectory
  • ARPO1: End-to-End Policy Optimization for GUI Agents with Experience Replay

• context management

  • CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents
  • (MemAct) Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks

• data

  • Beyond Stochastic Exploration: What Makes Training Data Valuable for Agentic Search

• emergence

  • From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones
  • ARTIST: Agentic Reasoning and Tool Integration in Self-improving Transformers

• general evaluation

  • Agents’ Last Exam

On-policy distillation

Blogs

• 从技术报告看On-Policy Distillation的崛起: 大模型后训练新范式

Papers

• OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning

Coding Agent

survey

blogs

• 主流Agent Harness实现对比——Memory篇
• 主流Agent Harness实现对比——Context压缩
• 主流Agent Harness实现对比——SubAgent与MultiAgent

models

• codex
  • codex：https://zhuanlan.zhihu.com/p/2029683221646907323
  • prompt
    • (blog) system prompt：https://www.lvyouyou.dev/posts/codex_prompt/
    • general system prompt
    • on-request prompt
    • multi-agent prompt：
      • control
      • spawn
    • compact prompt
• claude *
• cursor
  • Introducing Composer 2.5

benchmarks

• DeNovoSWE: Scaling Long-Horizon Environments for Generating Entire Repositories from Scratch
• Immersion in the GitHub Universe: Scaling Coding Agents to Mastery

GUI Agent

survey

• Large Language Model-Brained GUI Agents: A Survey
  • GUI Agent综述 : 揭秘GUI智能体的前世今生-1 : 总览篇-启程

models

• UI-TARS: Pioneering Automated GUI Interaction with Native Agents
• autoglm
  • ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents
  • MobileRL: Advancing Mobile Use Agents With Adaptive Online Reinforcement Learning
  • ANDROIDGEN: Building an Android Language Agent under Data Scarcity
  • Autoglm: Autonomous foundation agents for guis
  • WebRL:Training llm web agents via self-evolving online curriculum reinforcement learning
  • AndroidLab: Training and Systematic Benchmarking of Android Autonomous Agents
• DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning
• SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
• Appagent: Multimodal agents as smartphone users
• (SeeAct) GPT-4V(ision) is a Generalist Web Agent, if Grounded
• Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V

benchmarks

• web
  • ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use
  • WebArena: A Realistic Web Environment for Building Autonomous Agents
  • Mind2web: Towards a generalist agent for the web
  • Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments
  • (MiniWob) World of Bits: An Open-Domain Platform for Web-Based Agents
• android
  • Androidworld: A dynamic benchmarking environment for autonomous agents
  • Android in the Wild: A Large-Scale Dataset for Android Device Control
  • (AndroidArena) Understanding the weakness of large language model agents within a complex android environment

DeepResearch

survey

• Deep Research Agents: A Systematic Examination And Roadmap
• Towards AI Search Paradigm

models

• Search-o1: Agentic search-enhanced large reasoning models
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• R1-searcher: Incentivizing the search capability in llms via reinforcement learning
  • repo
• (Jina) node-DeepResearch Public
• Kimi-Researcher: End-to-End RL Training for Emerging Agentic Capabilities
• Language Modeling by Language Models

AutoResearch

survey

• AI for Auto-Research: Roadmap & User Guide

papers

resources

• karpathy/autoresearch

Others

office agent

• MemSlides: A Hierarchical Memory Driven Agent Framework for Personalized Slide Generation with Multi-turn Local Revision

Engineering for Agentic RL

environment

• Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application
• Environment Scaling for Agentic RL