LangGraph Chapter 4 — Tool Calling End-to-End

Senior Architect Interview Series — LangGraph & Agentic AI

Navigation

← Chapter 3 — AgentState | Chapter 5 — Routing →

4.0 What This Chapter Covers

Tool calling is the mechanism by which LLMs execute real actions in the world. It is the bridge between the LLM's language understanding and actual computation. This chapter covers:

1. What tool calling is and how OpenAI implements it
2. The @tool decorator — how LangChain exposes functions to LLMs
3. bind_tools() — how the LLM learns what tools are available
4. The complete request/response cycle for a tool call
5. ToolMessage — how results flow back to the LLM
6. How your agent.py implements the tool execution loop

4.1 The Tool Calling Mechanism

Tool calling (also called "function calling") is an OpenAI API feature where the LLM can request that your code execute a specific function with specific arguments.

The flow:

User Question
     │
     ▼
┌────────────────────────────────────────────────────┐
│  LLM (with tool schemas in the prompt)             │
│                                                    │
│  "I need to search for information about this      │
│   topic. I'll call rag_search(query='...')"        │
└────────────────────────────────────────────────────┘
     │
     │ Returns: AIMessage with tool_calls=[{...}]
     │ (NOT a text answer — a function call specification)
     ▼
┌────────────────────────────────────────────────────┐
│  Your Code (call_tools node)                       │
│                                                    │
│  1. Extract tool_calls from AIMessage              │
│  2. Execute rag_search(query='...')                │
│  3. Get result: "Agent Factory is..."              │
│  4. Wrap in ToolMessage                            │
└────────────────────────────────────────────────────┘
     │
     │ Appends: ToolMessage(content="Agent Factory is...")
     ▼
┌────────────────────────────────────────────────────┐
│  LLM (second call — now sees tool result)          │
│                                                    │
│  "Based on the search result, Agent Factory is     │
│   PepsiCo's platform for..."                       │
└────────────────────────────────────────────────────┘
     │
     │ Returns: AIMessage with content (text answer)
     ▼
  Final Answer

4.2 The @tool Decorator

The @tool decorator from LangChain converts a Python function into a StructuredTool that LangGraph/LangChain can work with.

Your Project's Tool

# From agent/agent.py
from langchain_core.tools import tool
from rag.retrieve import retrieve, build_prompt

@tool
def rag_search(query: str) -> str:
    """Search the internal knowledge base for information about PepsiCo Agent Factory.
    Use this tool when the user asks about Agent Factory, its capabilities,
    architecture, or related topics."""
    results = retrieve(query)          # query ChromaDB
    return build_prompt(query, results) # format results as context string

What @tool Does Under the Hood

The decorator:

1. Inspects the function's name → becomes the tool name the LLM uses
2. Reads the docstring → becomes the tool description sent to the LLM
3. Inspects type annotations → generates the JSON schema for arguments
4. Wraps everything in a StructuredTool object

The resulting tool schema (what's sent to the LLM in the OpenAI API call) looks like:

{
  "type": "function",
  "function": {
    "name": "rag_search",
    "description": "Search the internal knowledge base for information about PepsiCo Agent Factory...",
    "parameters": {
      "type": "object",
      "properties": {
        "query": {
          "type": "string",
          "description": "query"
        }
      },
      "required": ["query"]
    }
  }
}

Critical insight: The docstring IS the tool description. When the LLM decides whether to call your tool, it reads this description. A poor docstring means poor tool selection decisions.

4.3 Writing Effective Tool Descriptions

The docstring determines when the LLM calls your tool. Write it as if instructing the LLM.

# BAD — too vague
@tool
def rag_search(query: str) -> str:
    """Search for stuff."""
    ...

# MEDIOCRE — tells what not when
@tool
def rag_search(query: str) -> str:
    """Search the knowledge base."""
    ...

# GOOD — tells what, when, and how
@tool
def rag_search(query: str) -> str:
    """Search the internal knowledge base for information about PepsiCo Agent Factory.
    
    Use this tool when:
    - The user asks about Agent Factory features, architecture, or capabilities
    - Questions about how the system works internally
    - Questions about integration patterns or APIs
    
    Args:
        query: A natural language search query. Be specific about what you need.
    
    Returns:
        Relevant context passages from the knowledge base.
    """
    ...

4.4 bind_tools() — Connecting LLM to Tools

bind_tools() attaches tool schemas to an LLM instance so every call includes the tools in the API request.

# From agent/agent.py
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

tools = [rag_search]  # list of @tool-decorated functions

llm_with_tools = llm.bind_tools(tools)
# llm_with_tools is a new LLM wrapper that always sends tool schemas

What changes in the API call:

Without bind_tools:

{
  "model": "gpt-4o-mini",
  "messages": [{"role": "user", "content": "..."}]
}

With bind_tools:

{
  "model": "gpt-4o-mini",
  "messages": [{"role": "user", "content": "..."}],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "rag_search",
        "description": "Search the internal knowledge base...",
        "parameters": { ... }
      }
    }
  ],
  "tool_choice": "auto"
}

tool_choice: "auto" means the LLM decides whether to call a tool or respond directly. You can override this:

• "auto" — LLM decides (default)
• "required" — LLM MUST call a tool (useful for forced structured output)
• {"type": "function", "function": {"name": "rag_search"}} — force a specific tool

4.5 The AIMessage With Tool Calls

When the LLM decides to call a tool, it returns an AIMessage with a special tool_calls field:

# What call_llm receives from llm_with_tools.invoke(messages):
response = AIMessage(
    content="",     # empty string — no text, just tool call
    tool_calls=[
        {
            "id":   "call_abc123",           # unique ID for this call
            "name": "rag_search",            # which tool to call
            "args": {"query": "how does Agent Factory work?"}  # arguments
        }
    ]
)

Key points:

• content is empty when tool calls are present (the LLM isn't saying anything, it's requesting action)
• tool_calls is a list — the LLM can request multiple tool calls in parallel
• Each tool call has a unique id that must be referenced in the ToolMessage response

4.6 The call_tools Node — Your Project Implementation

# From agent/agent.py
def call_tools(state: AgentState) -> AgentState:
    last_message = state["messages"][-1]
    results = []

    for tool_call in last_message.tool_calls:
        tool_name = tool_call["name"]      # "rag_search"
        tool_args = tool_call["args"]      # {"query": "..."}
        tool_call_id = tool_call["id"]     # "call_abc123"

        # Look up and execute the tool
        tool_fn = tool_map[tool_name]      # tool_map = {"rag_search": rag_search}
        output = tool_fn.invoke(tool_args) # execute rag_search(query="...")

        # Wrap result in a ToolMessage with the matching ID
        results.append(
            ToolMessage(
                content=str(output),       # string result
                tool_call_id=tool_call_id  # MUST match tool_call["id"]
            )
        )

    return {"messages": results}  # append all ToolMessages to state

The tool_map

tool_map = {tool.name: tool for tool in tools}
# = {"rag_search": <rag_search StructuredTool>}

This is a simple lookup dict. When the LLM requests rag_search, we index into tool_map by name and invoke it.

Why tool_call_id Must Match

The tool_call_id in the ToolMessage tells the LLM which tool call this result belongs to. OpenAI validates that every tool_call in the AIMessage has a corresponding ToolMessage with the matching id. If they don't match, the API returns an error.

4.7 The Complete Message Sequence for Tool Calling

1. HumanMessage(content="How does Agent Factory work?")
   role: user

2. AIMessage(content="", tool_calls=[{id: "call_abc", name: "rag_search", args: {...}}])
   role: assistant

3. ToolMessage(content="Agent Factory is PepsiCo's...", tool_call_id: "call_abc")
   role: tool

4. AIMessage(content="Based on the search results, Agent Factory is PepsiCo's...")
   role: assistant  ← FINAL ANSWER

OpenAI processes the entire message list and uses message types to understand the conversation flow. The LLM "sees" that message 2 requested a tool call, message 3 is the result, and now it should generate a final answer.

4.8 Parallel Tool Calls

Modern LLMs support requesting multiple tool calls in a single response. LangGraph handles this automatically:

# LLM might return this during a complex query:
AIMessage(
    content="",
    tool_calls=[
        {"id": "call_1", "name": "rag_search", "args": {"query": "Agent Factory architecture"}},
        {"id": "call_2", "name": "rag_search", "args": {"query": "Agent Factory APIs"}},
        {"id": "call_3", "name": "sql_query",  "args": {"query": "SELECT * FROM metrics LIMIT 5"}}
    ]
)

# call_tools iterates over all three:
for tool_call in last_message.tool_calls:  # iterates 3 times
    output = tool_fn.invoke(tool_call["args"])
    results.append(ToolMessage(..., tool_call_id=tool_call["id"]))
# Returns 3 ToolMessages — one per tool call

Your current call_tools implementation already handles parallel tool calls correctly because it iterates over last_message.tool_calls (a list).

4.9 Tool Errors — Graceful Handling

When a tool execution fails, return the error as a ToolMessage rather than raising. This lets the LLM recover:

def call_tools(state: AgentState) -> AgentState:
    last_message = state["messages"][-1]
    results = []
    
    for tool_call in last_message.tool_calls:
        try:
            tool_fn = tool_map[tool_call["name"]]
            output = tool_fn.invoke(tool_call["args"])
            results.append(ToolMessage(
                content=str(output),
                tool_call_id=tool_call["id"]
            ))
        except Exception as e:
            # Return error as ToolMessage — LLM will see it and may retry or explain
            results.append(ToolMessage(
                content=f"Error executing tool: {str(e)}",
                tool_call_id=tool_call["id"]
            ))
    
    return {"messages": results}

When the LLM sees "Error executing tool: ..." in the ToolMessage, it typically:

1. Retries with a different query (if it was a search)
2. Apologizes and explains it couldn't fetch the data
3. Attempts to answer from its training data

4.10 The should_call_tools Routing Function

# From agent/agent.py
def should_call_tools(state: AgentState) -> str:
    last_message = state["messages"][-1]
    if hasattr(last_message, "tool_calls") and last_message.tool_calls:
        return "call_tools"   # route to call_tools node
    return END                # route to END — done

This function:

1. Gets the last message (always an AIMessage from call_llm)
2. Checks if it has tool_calls AND they're non-empty
3. Returns the appropriate routing string

Why both checks?

• hasattr(last_message, "tool_calls") — guards against non-AIMessages (ToolMessage, HumanMessage don't have this attribute)
• last_message.tool_calls — guards against empty list [] (the attribute exists but is empty when LLM chose not to call tools)

4.11 Multiple Tools

Registering multiple tools is straightforward:

@tool
def rag_search(query: str) -> str:
    """Search the knowledge base for Agent Factory information."""
    return retrieve_and_format(query)

@tool
def get_current_date() -> str:
    """Get the current date. Use when the user asks about time-sensitive information."""
    from datetime import datetime
    return datetime.now().strftime("%Y-%m-%d %H:%M UTC")

@tool
def calculate(expression: str) -> str:
    """Evaluate a mathematical expression. Use for calculations."""
    try:
        result = eval(expression, {"__builtins__": {}})  # restricted eval
        return str(result)
    except Exception as e:
        return f"Error: {e}"

tools = [rag_search, get_current_date, calculate]
tool_map = {tool.name: tool for tool in tools}
llm_with_tools = llm.bind_tools(tools)

The LLM automatically selects the best tool based on the question.

4.12 Tool Best Practices

4.13 Interview Q&A

Q: Explain the complete tool calling lifecycle in your LangGraph agent.

When call_llm runs, llm_with_tools.invoke(messages) sends the full conversation history plus tool schemas to the OpenAI API. If the LLM decides to search, it returns an AIMessage with content="" and tool_calls=[{"id": "...", "name": "rag_search", "args": {"query": "..."}}]. The should_call_tools routing function detects this and routes to call_tools. That node looks up rag_search in tool_map, calls tool_fn.invoke({"query": "..."}), wraps the result in a ToolMessage with the matching tool_call_id, and returns it to state. Then call_llm runs again — now with the ToolMessage in context — and produces the final text answer. should_call_tools returns END. Total: two LLM calls, one tool execution.

Q: What is bind_tools() and what does it change about the API request?

bind_tools(tools) creates a new LLM wrapper that automatically includes tool schemas in every API call. Without it, the OpenAI request is a plain messages array. With it, the request includes a tools array containing the JSON schema for each @tool function — name, description, and a JSON Schema for the parameters. The OpenAI API uses "tool_choice": "auto" by default, allowing the LLM to choose whether to call a tool or respond directly. bind_tools() is non-destructive — it creates a new wrapper and doesn't modify the original llm object.

Q: Why must the ToolMessage.tool_call_id match the AIMessage.tool_calls[n]["id"]?

OpenAI's API enforces a contract: every tool call requested in an AIMessage must have a corresponding ToolMessage with a matching tool_call_id. This pairing lets OpenAI correctly associate results with requests in a parallel tool call scenario — if the LLM requested 3 tools simultaneously and you return 3 ToolMessages, the IDs tell the model which result belongs to which request. Mismatched IDs result in an API error: "tool_call_id must correspond to a tool call in the previous message".

Q: How would you add a tool that queries your SQL database securely?

Define a @tool with a descriptive docstring and a query parameter. Inside the function, use parameterized queries (never f-strings or string concatenation with user input) through SQLAlchemy. Apply a whitelist of allowed operations (SELECT only, no DDL/DML), limit result sizes, and add a timeout. Return a formatted string. Register it in tools and tool_map, and add it to llm_with_tools = llm.bind_tools(tools). The SQL injection risk is in the generated query, not the parameter — so log all generated SQL in production and alert on anomalies.

Q: What happens if the LLM calls a tool that doesn't exist in tool_map?

You get a KeyError when indexing tool_map[tool_call["name"]]. The correct fix is to handle this gracefully: check if the tool name exists, and if not, append a ToolMessage with an error message like "Unknown tool: {name}. Available tools: {list(tool_map.keys())}". This returns the error to the LLM, which can then either choose a valid tool or explain that it doesn't have the capability. Raising an unhandled exception crashes the agent turn entirely.

4.14 Key One-Liners to Memorize

"The @tool docstring IS the tool description — write it for the LLM, not for humans."

"bind_tools() adds tool schemas to every LLM API call — the LLM sees what it can do."

"AIMessage with tool_calls: LLM is requesting action. With content: LLM is responding."

"ToolMessage tool_call_id MUST match the AIMessage's tool_call id — OpenAI enforces this."

"The tool loop: call_llm → detect tool_calls → call_tools → call_llm → ... → END."

"Return errors as ToolMessages, not exceptions — let the LLM recover gracefully."

Next: Chapter 5 — Conditional Routing & Graph Control Flow