Understanding Agentic AI Architecture

"Agent" is one of the most overloaded words in AI. Stripped of marketing, an agent is a language model running in a loop: it decides on an action, the action is carried out, the result is fed back, and the model decides what to do next. This article explains the parts and the design decisions around them.

Workflow or agent?

Start with a distinction that saves a lot of effort.

• A workflow is a fixed sequence of steps written in code. A model may be called at some steps, but the path is decided by the developer.
• An agent decides its own path at run time.

Workflows are predictable, cheap and easy to test. Agents are flexible and harder to control. If you can draw the process as a flowchart, build the flowchart. Reach for an agent when the steps genuinely cannot be known in advance.

The agent loop

messages = [system_prompt, user_goal]

for step in range(MAX_STEPS):
    response = model(messages, tools=TOOLS)
    if not response.tool_calls:
        return response.text          # the agent is finished
    for call in response.tool_calls:
        result = run_tool(call)       # validated and executed by your code
        messages.append(tool_result(call, result))

raise StepLimitReached()

That is the whole pattern. Everything else is engineering around it.

The four building blocks

Tools. Functions the model can ask your code to run: search, database queries, API calls. The model never executes anything itself — it produces a request and your code decides whether to honour it. Tool names, descriptions and parameter schemas are effectively part of the prompt, so write them carefully.

Context and memory. The model only knows what is in its context window. Short-term memory is the conversation so far; long-term memory is anything you store and retrieve later. Long tasks need a strategy for summarizing or trimming context, or the agent slows down and loses track.

Planning. Some agents plan up front and then execute; others decide one step at a time. Planning first helps on long tasks, and gives a human something to review before work begins.

Control. Step limits, token budgets, permission checks and approval gates. This is the block that most demos leave out and most production systems depend on.

Designing for control

A few rules hold up across projects:

• Separate read tools from write tools. Reading is low risk. Anything that changes data, sends a message or spends money should require explicit approval until trust is earned.
• Validate every tool call. Check arguments against a schema and return a clear error. A good error message lets the model correct itself on the next step.
• Set limits. A maximum number of steps and a cost ceiling per run prevent loops.
• Trace everything. Record each step, tool call and result. Without traces, an agent cannot be debugged or evaluated.

Multi-agent systems

Splitting work across several agents — an orchestrator and specialists — helps when subtasks are independent and each needs its own focused context. It also multiplies cost and introduces coordination problems. Use one agent until a single context is clearly the bottleneck.

Evaluating agents

Judging only the final answer is not enough. An agent that reaches the right answer through an unsafe or wasteful sequence of actions has still failed. Evaluate the trajectory: which tools were called, in what order, and whether any should have required approval.

A short checklist

1. Could this be a fixed workflow instead?
2. What is the smallest set of tools that solves the task?
3. Which actions need a human to approve them?
4. What stops a run that is going wrong?
5. How will you know if a change made it better or worse?

If you can answer those five questions, you have an architecture — not just a demo.