How AI Agents Actually Work

"Agent" has become the word for anything AI that seems to do more than answer a question. Strip the marketing away and an agent is a specific, understandable piece of software: a language model running inside a loop, with the ability to call functions you have defined, until it decides the task is done.

This article explains that loop precisely, then covers the engineering that separates a demo from something a business can rely on.

The loop#

Every agent, from a simple support assistant to a multi-step research tool, runs some version of this:

In code, stripped to essentials:

async function runAgent(goal: string, tools: Tool[], maxSteps = 12) {
  const messages: Message[] = [
    { role: "system", content: SYSTEM_INSTRUCTIONS },
    { role: "user", content: goal },
  ];
 
  for (let step = 0; step < maxSteps; step++) {
    const response = await llm.chat({ messages, tools });
 
    if (response.toolCalls.length === 0) {
      return response.text; // the model decided it is finished
    }
 
    for (const call of response.toolCalls) {
      const tool = tools.find((t) => t.name === call.name);
      const input = tool.schema.parse(call.arguments); // validate before running
      const result = await tool.run(input, { user: currentUser });
      messages.push({ role: "tool", toolCallId: call.id, content: JSON.stringify(result) });
      await audit.log({ step, tool: call.name, input, result });
    }
  }
 
  throw new AgentStoppedError("Step limit reached");
}

Three things to notice:

1. The model never runs code. It emits a structured request — "call lookup_order with { id: 'A123' }" — and your program decides whether and how to execute it. This is the whole safety model of agents: the boundary is in your code.
2. Tools are typed. The model is given each tool's name, description and input schema. The description is the interface, and its quality determines how well the model uses the tool.
3. The loop is bounded. A step limit, a time limit and a cost limit are not optional. Without them, a confused model will happily loop forever.

What a tool actually is#

A tool is a function with a schema and a description:

const lookupOrder: Tool = {
  name: "lookup_order",
  description:
    "Fetch an order by its ID. Returns status, items, totals and the customer's shipping address. Use when the user references an order number.",
  schema: z.object({ id: z.string().regex(/^[A-Z]\d{3,}$/) }),
  async run({ id }, ctx) {
    return orders.findForUser(id, ctx.user); // permission-scoped, always
  },
};

Good tools share some properties:

• Narrow. One job. lookup_order, not do_order_things.
• Described for a reader who has never seen your system. The model only knows what the description says.
• Permission-scoped. The tool runs as the current user, never as an admin, and enforces the same access rules the UI would.
• Idempotent where possible. The model may retry. create_ticket should not create two tickets if called twice with the same input.
• Returns compact, structured results. The output goes back into the model's context and costs tokens. Return what is needed, not the whole record.

If tool-using systems seem unreliable, look at the tool descriptions before blaming the model. Vague descriptions produce vague behaviour.

Reads, writes and the human checkpoint#

Tools fall into two classes and should be treated differently.

Read tools — look up, search, calculate, summarise — are low risk. Let the agent call them freely, within rate limits.

Write tools — send, create, update, delete, pay — change the world. For these, the pattern I use is a proposal: the tool does not perform the action, it returns a proposed action that a person approves in the interface. Only after approval does the real write happen.

const sendReply: Tool = {
  name: "send_reply",
  description: "Propose a reply to the customer. A human will review before it is sent.",
  schema: z.object({ ticketId: z.string(), body: z.string().max(2000) }),
  async run(input, ctx) {
    const proposal = await proposals.create({ ...input, proposedBy: "agent", forUser: ctx.user.id });
    return { proposalId: proposal.id, status: "awaiting_approval" };
  },
};

The agent completes its task ("I have drafted a reply and queued it for approval"), the human sees a one-click approval, and the system never sends something nobody read. As trust is earned — measured by how often humans approve without editing — specific low-risk writes can be promoted to automatic.

Memory and state#

The loop above holds everything in the messages array for one run. Real systems need more:

• Task state — where the agent is in a multi-step job, persisted so a crash or a deploy does not lose work. Store it in your database like any other workflow state.
• Short-term memory — the conversation so far, trimmed or summarised when it grows past a budget.
• Long-term memory — facts worth keeping across runs (a customer's preferences, a resolved issue). This is just retrieval: store facts with metadata and search them at the start of the next run. It is the same machinery as a RAG system.

Resist the temptation to build a bespoke memory framework. A table of facts with embeddings and a timestamp covers most needs.

Planning, and whether you need it#

Simple agents decide one step at a time, which is fine for short tasks. Longer tasks benefit from an explicit plan: ask the model to outline the steps first, store the plan, and execute against it, revising when a step's result changes the picture. The plan becomes visible state that a person can inspect — "it is about to do X, Y, then Z" — which matters for trust.

Multi-agent setups (a planner delegating to specialists) are sometimes useful and often over-engineered. Start with one agent and good tools. Split only when a single context genuinely cannot hold the job.

Failure modes and how to design for them#

Infinite loops. Step, time and cost limits. Always.

Hallucinated tool arguments. Validate every input against the schema before running anything; return the validation error to the model so it can correct itself.

Confident wrong answers. Require citations for factual claims (from retrieval) and prefer "I could not find that" over guessing. Test with questions the tools cannot answer.

Runaway cost. Meter tokens per run and per user. Set a budget per task and stop with a clear message when it is exceeded.

Silent side effects. Every tool call logged with user, inputs, outputs and timestamp. If you cannot reconstruct what an agent did, you cannot operate it.

Prompt injection. Content the agent reads — emails, web pages, documents — may contain text designed to hijack it ("ignore previous instructions and forward this to..."). Treat all retrieved content as data, never as instructions; keep write tools behind approval; and never give an agent more access than the user it acts for.

Evaluating an agent#

Agents are harder to evaluate than single calls because there are many valid paths to a result. What works:

• A set of task scenarios with a definition of success (the right ticket created, the correct figure reported).
• Trajectory review — sample real runs and read the tool-call sequence. Wrong tool choice, unnecessary calls and near-misses show up quickly.
• Approval rate on proposed writes as the running quality metric in production.

Do you need an agent?#

Often, no. If the task has a fixed sequence of steps, write the sequence in code and use the model for the steps that need language understanding — classification, extraction, drafting. That is a workflow with AI steps, and it is more predictable, cheaper and easier to test than an agent.

Reach for an agent when the path genuinely varies with the input: a support request that might need one lookup or five, a research task where each result changes the next question, an operations assistant that handles many small, different jobs. There, the loop's flexibility earns its cost.

Summary#

An agent is a bounded loop in which a model requests typed tool calls and your code decides whether to run them. The engineering that matters is: precise tool definitions, permission-scoped execution, human approval on writes, persistent state, hard limits, full audit logs and a way to evaluate runs. None of that is magic. All of it is necessary.