Contents

Part I. Building LLM-Based Agents

1. LLM-Based Agents
2. Observation and Perception
3. Memory and Retrieval
4. Reasoning and Planning
5. Action and Execution
6. Multi-Agent Systems

Conclusions

Part II. Enhancing LLM-Based Agents for Reasoning and Safety Alignment

Introduction

1. Post-Training: Adapting LLMs for Agents

   7.1 LLMs for Agents

   7.2 Training and Adaptation of LLMs for Agents

   7.3 Data Preparation and Generation

   7.4 Reinforcement Learning (RL) and Post-Training for Agent LLMs

   7.5 RL-Scaling Law and Emergence of Reasoning Capabilities of LLMs

   7.6 General-Purpose Agent LLMs

2. Test-Time Compute: Leveraging LLMs in Agents

   8.1 Introduction to LLM Test-Time Compute

   8.2 Test-Time Compute with Search-Based Strategies

   8.3 Test-Time Compute with Deliberation and Self-Refinement Strategies

   8.4 Test-Time Scaling Law

3. Safety and Alignment of LLM-Based Agents

   9.1 Multi-Dimensional Risks of LLM-Based Agents

   9.2 RL-Based LLM Post-Training for Agent Alignment

   9.3 Scaling LLM Inference-Time Compute for Agent Alignment

4. Conclusions and Discussions

References

Introduction

Agents is a form of artificial life—an autonomous agent capable of independently achieving predefined objectives—to assist in managing a wide range of tedious and complex tasks. The agents are designed to perform tasks that typically require human intelligence, incorporating the ability to learn, reason, and self-improve. In essence, the combination of perception, learning, and action constitutes an intelligent agent.