The Application Layer
In part 1, we mapped the AI stack. This is the layer users touch—and the layer that turns a model into a product.
A model produces an output. An application decides what that output means, what it is allowed to do, and how the system recovers when it is wrong.
A Chatbot Is Not an Agent
The simplest AI application looks like this:
user input → model → response
An agent has a loop:
goal → context → model → tool call → observation → model → result
The model still generates text and tool requests. The application owns the loop around it.
The Pieces of an Agent
Context is what the model can see: instructions, conversation history, retrieved documents, tool results, and current task state. Context is not “everything.” Good applications select the smallest useful set.
Tools are the functions the agent can call. A tool should have a clear input schema, a narrow responsibility, useful errors, and a permission boundary. A vague or overpowered tool is an invitation to failure.
Memory stores information beyond the current call. Some memory is task state. Some is durable user or project knowledge. Treating every past message as memory creates noise, not intelligence.
Orchestration decides what happens next. A single loop may be enough. More complex systems use workflows, routers, or multiple specialized agents. Add coordination only when the problem needs it; every extra handoff creates another failure surface.
Execution runs the work: code, queries, files, API calls, and transactions. Execution needs timeouts, retries with limits, idempotency, and clear failure states.
Runtime provides isolation. If an agent can run generated code or touch user data, it needs a sandbox, scoped credentials, network policy, and a way to stop it.
The Production Boundary
The prototype usually ends at “the model gave a good answer.” Production starts with harder questions:
- What can the agent read and write?
- Which actions require confirmation?
- What happens when a tool times out halfway through?
- Can the task resume after a process restart?
- How do you know the answer satisfied the requirement?
- Can you reconstruct what happened later?
These are application concerns, even when the model is the most visible part of the feature.
A Useful Control Loop
Build the application around a small, explicit loop:
- Define the goal and constraints.
- Select context and tools.
- Let the model propose the next action.
- Validate the action before executing it.
- Execute with timeouts and least privilege.
- Feed the observation back into the loop.
- Verify the result against the requirement.
- Record the episode.
The agent is not autonomous because it talks to itself. It is autonomous when it can make progress, observe consequences, recover from bounded failures, and stop with evidence.
Where Applications Break
Most failures are ordinary engineering failures wearing an AI costume:
- stale or irrelevant context,
- tools with unclear contracts,
- memory that retrieves the wrong thing,
- retries that duplicate side effects,
- workflows that lose state,
- permissions that are too broad,
- no verification beyond “the model said so.”
The fix is rarely another paragraph in the system prompt. Improve the interface, state model, permission boundary, or check that allowed the failure.
The Engineer’s Job
You do not need to write every line an agent produces. You do need to own the system around those lines.
That means defining the objective, designing the boundaries, choosing what to delegate, and building the feedback that tells you when the system is wrong. The more powerful the agent, the more important those controls become.
The application layer is where model capability becomes leverage—or becomes a liability.
Further Reading
- Model Context Protocol specification
- Temporal documentation on durable execution
- OpenTelemetry documentation
Next in the series: The Model Layer — what models are, how they are trained, and how they run.