ReAct Agent Architecture

1. Introduction to ReAct Agents

ReAct (Reasoning + Acting) is a powerful framework that enables language models to solve complex tasks through an iterative process of reasoning and acting. This pattern allows an AI agent to break down problems, take appropriate actions, and use the results to inform its subsequent reasoning.

Thought

→

Action

→

Observation

→

Thought

→

Answer

The ReAct pattern follows this simple cycle:

Thought: The agent reasons about the current state and decides what to do next.
Action: The agent selects and executes an appropriate action.
Observation: The agent receives feedback from the action.
This cycle repeats until the agent is ready to provide a final answer.

Key Insight: By separating reasoning (thoughts) from acting (tool use), ReAct enables language models to solve much more complex problems than they could with either capability alone.

2. Setting Up the Environment

To build a ReAct agent, we'll first need to set up our environment and import the necessary libraries.

environment_setup.py

import openai
import re
import httpx
import os
from dotenv import load_dotenv

_ = load_dotenv()
from openai import OpenAI

client = OpenAI()

# Test the OpenAI client
chat_completion = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hello world"}]
)

print(chat_completion.choices[0].message.content)

This code does several important things:

Imports necessary libraries including the OpenAI SDK
Loads environment variables (including your API key) from a .env file
Creates an OpenAI client for making API calls
Tests the connection with a simple "Hello world" message

3. Creating the Basic Agent Class

Next, we'll create a simple Agent class that can maintain a conversation with the language model.

agent.py

class Agent:
    def __init__(self, system=""):
        self.system = system
        self.messages = []
        if self.system:
            self.messages.append({"role": "system", "content": system})

    def __call__(self, message):
        self.messages.append({"role": "user", "content": message})
        result = self.execute()
        self.messages.append({"role": "assistant", "content": result})
        return result

    def execute(self):
        completion = client.chat.completions.create(
                        model="gpt-4o",
                        temperature=0,
                        messages=self.messages)
        return completion.choices[0].message.content

The Agent class has three main components:

Constructor

Initializes the conversation history and sets an optional system message that provides instructions to the model.

Call Method

Makes the agent callable. It adds the user's message to the history, gets a response from the model, and adds that response to the history.

Execute Method

Makes the actual API call to OpenAI, passing along the entire conversation history, and returns the model's response.

Design Pattern: This implementation follows a stateful conversation pattern, where the agent maintains the full conversation history internally.

4. Defining the ReAct Prompt

The key to implementing the ReAct pattern is a well-crafted system prompt that instructs the model on how to think and act.

react_prompt.py

prompt = """
You run in a loop of Thought, Action, PAUSE, Observation.
At the end of the loop you output an Answer
Use Thought to describe your thoughts about the question you have been asked.
Use Action to run one of the actions available to you - then return PAUSE.
Observation will be the result of running those actions.

Your available actions are:

calculate:
e.g. calculate: 4 * 7 / 3
Runs a calculation and returns the number - uses Python so be sure to use floating point syntax if necessary

average_dog_weight:
e.g. average_dog_weight: Collie
returns average weight of a dog when given the breed

Example session:

Question: How much does a Bulldog weigh?
Thought: I should look the dogs weight using average_dog_weight
Action: average_dog_weight: Bulldog
PAUSE

You will be called again with this:

Observation: A Bulldog weights 51 lbs

You then output:

Answer: A bulldog weights 51 lbs
""".strip()

This prompt is carefully designed to implement the ReAct pattern:

Instruction Format: It explains the Thought-Action-Observation loop structure.
Available Actions: It defines the tools the agent can use (calculate and average_dog_weight).
Example: It provides a concrete example of the expected interaction pattern.
PAUSE Mechanism: The PAUSE keyword signals where execution should stop to await tool results.

Important: The PAUSE keyword is crucial - it tells the agent to stop generating text after specifying an action, allowing the system to execute the tool and provide observations.

5. Implementing the Tools

Next, we'll implement the tools that our agent can use to interact with the world.

tools.py

def calculate(what):
    return eval(what)

def average_dog_weight(name):
    if name in "Scottish Terrier":
        return("Scottish Terriers average 20 lbs")
    elif name in "Border Collie":
        return("a Border Collies average weight is 37 lbs")
    elif name in "Toy Poodle":
        return("a toy poodles average weight is 7 lbs")
    else:
        return("An average dog weights 50 lbs")

known_actions = {
    "calculate": calculate,
    "average_dog_weight": average_dog_weight
}

calculate

A simple calculator tool that uses Python's eval() function to perform mathematical operations.

average_dog_weight

A specialized tool that returns the average weight for different dog breeds. Has predefined values for certain breeds.

known_actions

A dictionary that maps tool names to their corresponding functions. This allows the agent to call the right function based on the action name.

These tools are simple but demonstrate the concept. In a real application, tools might interact with APIs, databases, or other external systems.

6. Creating the Agent Loop

Now we'll implement the core loop that manages the interaction between the agent and the tools.

User Question

→

Agent Reasoning

→

Action Extraction

→

Tool Execution

→

Return Observation

↩

agent_loop.py

action_re = re.compile('^Action: (\w+): (.*)$')   # python regular expression to select action

def query(question, max_turns=5):
    i = 0
    bot = Agent(prompt)
    next_prompt = question
    while i < max_turns:
        i += 1
        result = bot(next_prompt)
        print(result)
        actions = [
            action_re.match(a)
            for a in result.split('\n')
            if action_re.match(a)
        ]
        if actions:
            # There is an action to run
            action, action_input = actions[0].groups()
            if action not in known_actions:
                raise Exception("Unknown action: {}: {}".format(action, action_input))
            print(" -- running {} {}".format(action, action_input))
            observation = known_actions[action](action_input)
            print("Observation:", observation)
            next_prompt = "Observation: {}".format(observation)
        else:
            return

The query function implements the ReAct agent loop:

Initialize the agent with our ReAct prompt
Send the user's question to the agent
Parse the response using regex to extract any actions
Execute the tool if an action is found
Format the result as an observation
Send the observation back to the agent
Repeat until the agent gives a final answer (no more actions)

Safety Feature: The max_turns parameter prevents infinite loops by limiting the maximum number of iterations.

7. Complete Example

Let's see how our ReAct agent handles a more complex question that requires multiple steps:

complex_example.py

question = """I have 2 dogs, a border collie and a scottish terrier. \
What is their combined weight"""

query(question)
# The agent will:
# 1. Look up the weight of a border collie
# 2. Look up the weight of a scottish terrier
# 3. Calculate the sum of the weights
# 4. Provide the final answer

Here's the expected interaction flow:

User Question: "I have 2 dogs, a border collie and a scottish terrier. What is their combined weight?"
Agent Thought: "I need to find the weight of a border collie and a scottish terrier, then add them together."
Agent Action: "average_dog_weight: Border Collie"
Observation: "a Border Collies average weight is 37 lbs"
Agent Thought: "Now I need to find the weight of a scottish terrier."
Agent Action: "average_dog_weight: Scottish Terrier"
Observation: "Scottish Terriers average 20 lbs"
Agent Thought: "Now I can calculate the total weight by adding the two weights together: 37 + 20 = 57 lbs."
Agent Action: "calculate: 37 + 20"
Observation: "57"
Agent Answer: "The combined weight of your border collie and scottish terrier is 57 lbs."

This example demonstrates the power of the ReAct pattern - the agent breaks down the complex problem into a series of steps, gathers the necessary information using tools, and combines that information to produce a final answer.

8. Complete Code Architecture

Here's the complete code for our ReAct agent implementation:

react_agent.py

import openai
import re
import os
from dotenv import load_dotenv

_ = load_dotenv()
from openai import OpenAI

client = OpenAI()

class Agent:
    def __init__(self, system=""):
        self.system = system
        self.messages = []
        if self.system:
            self.messages.append({"role": "system", "content": system})

    def __call__(self, message):
        self.messages.append({"role": "user", "content": message})
        result = self.execute()
        self.messages.append({"role": "assistant", "content": result})
        return result

    def execute(self):
        completion = client.chat.completions.create(
                        model="gpt-4o",
                        temperature=0,
                        messages=self.messages)
        return completion.choices[0].message.content

prompt = """
You run in a loop of Thought, Action, PAUSE, Observation.
At the end of the loop you output an Answer
Use Thought to describe your thoughts about the question you have been asked.
Use Action to run one of the actions available to you - then return PAUSE.
Observation will be the result of running those actions.

Your available actions are:

calculate:
e.g. calculate: 4 * 7 / 3
Runs a calculation and returns the number - uses Python so be sure to use floating point syntax if necessary

average_dog_weight:
e.g. average_dog_weight: Collie
returns average weight of a dog when given the breed

Example session:

Question: How much does a Bulldog weigh?
Thought: I should look the dogs weight using average_dog_weight
Action: average_dog_weight: Bulldog
PAUSE

You will be called again with this:

Observation: A Bulldog weights 51 lbs

You then output:

Answer: A bulldog weights 51 lbs
""".strip()

def calculate(what):
    return eval(what)

def average_dog_weight(name):
    if name in "Scottish Terrier":
        return("Scottish Terriers average 20 lbs")
    elif name in "Border Collie":
        return("a Border Collies average weight is 37 lbs")
    elif name in "Toy Poodle":
        return("a toy poodles average weight is 7 lbs")
    else:
        return("An average dog weights 50 lbs")

known_actions = {
    "calculate": calculate,
    "average_dog_weight": average_dog_weight
}

action_re = re.compile('^Action: (\w+): (.*)$')   # python regular expression to select action

def query(question, max_turns=5):
    i = 0
    bot = Agent(prompt)
    next_prompt = question
    while i < max_turns:
        i += 1
        result = bot(next_prompt)
        print(result)
        actions = [
            action_re.match(a)
            for a in result.split('\n')
            if action_re.match(a)
        ]
        if actions:
            # There is an action to run
            action, action_input = actions[0].groups()
            if action not in known_actions:
                raise Exception("Unknown action: {}: {}".format(action, action_input))
            print(" -- running {} {}".format(action, action_input))
            observation = known_actions[action](action_input)
            print("Observation:", observation)
            next_prompt = "Observation: {}".format(observation)
        else:
            return

# Example usage
if __name__ == "__main__":
    question = """I have 2 dogs, a border collie and a scottish terrier. \
    What is their combined weight"""
    query(question)

9. Extending the Architecture

This basic ReAct agent can be extended in several ways to create more powerful applications:

Add More Tools

Integrate additional tools like web search, API calls, database queries, etc. to expand the agent's capabilities.

Improve Error Handling

Add robust error handling for tool execution failures, invalid inputs, and other edge cases.

Implement Memory

Add persistent memory to allow the agent to remember information across multiple conversations.

Enhance the Prompt

Refine the system prompt to improve the agent's reasoning capabilities and to help it use tools more effectively.

Add Tool Selection Logic

Implement more sophisticated logic for selecting which tools to use based on the context and question.

Support Multiple Actions

Allow the agent to specify multiple actions at once for more efficient processing.

Advanced Implementation: For more sophisticated agent architectures, consider using LangChain's agent frameworks which provide many of these extensions out of the box.

10. Key Takeaways

ReAct Pattern: Combining reasoning (thoughts) and acting (tools) creates a powerful problem-solving architecture.
Iterative Process: The Thought-Action-Observation loop allows the agent to break down complex problems into manageable steps.
Prompt Engineering: A well-crafted system prompt is crucial for guiding the model's behavior.
Tool Integration: External tools significantly extend what language models can accomplish.
Action Parsing: Regular expressions provide a simple way to extract actions from the model's output.
Conversation Management: The Agent class maintains the conversation state throughout the interaction.

The ReAct pattern is a powerful framework for building AI agents that can solve complex problems by breaking them down into manageable steps and using tools to gather information and perform actions.