Contents

Introduction

Part I. Building LLM-Based Agents

1. LLM-Based Agents

   1.1 Agentic Attributes of LLMs

   1.2 LLMs and Agents

   1.3 Architecture of LLM-Based Agents

2. Observation and Perception

   2.1 Perception Spaces

   2.2 Observability

3. Memory and Retrieval

   3.1 Types of Memory

   3.2 Knowledge Representation in Memory

   3.3 Memory Management

   3.4 Memory Retrieval

   3.5 Memory and Retrieval-Augmented Generation (RAG)

4. Reasoning and Planning

   4.1 Overview

   4.2 Goal Identification

   4.3 Reasoning

   4.3 Planning

   4.4 Task Decomposition and Chain of Thought (CoT)

   4.5 Reflection, Feedback, Backtracking, and Revision

   4.6 Planning Tools

5. Action and Execution

   5.1 Action Spaces

   5.2 Tool Utilization

   5.3 Tool Creation

   5.4 Embodied Actions

6. Multi-Agent Systems

   6.1 Characteristics of Multi-Agent Systems

   6.2 Applications of Multi-Agent Systems

Conclusions

References

Part II. Enhancing LLM-Based Agents for Reasoning and Safety Alignment

1. Post-Training: Adapting LLMs for Agents

   7.1 LLMs for Agents

   7.2 Training and Adaptation of LLMs for Agents

   7.3 Data Preparation and Generation

   7.4 RL-Based Post-Training for Agent LLMs

   7.5 General-Purpose Agent LLMs

2. Test-Time Compute: Leveraging LLMs in Agents

   8.1 Introduction to LLM Test-Time Compute

   8.2 Test-Time Compute with Search-Based Strategies

   8.3 Test-Time Compute with Deliberation and Self-Refinement Strategies

   8.4 Test-Time Scaling Law

3. Safety and Alignment of LLM-Based Agents

   9.1 Multi-Dimensional Risks of LLM-Based Agents

   9.2 RL-Based LLM Post-Training for Agent Alignment

   9.3 Scaling LLM Inference-Time Compute for Agent Alignment

4. Conclusion and Discussion

References

Introduction