Most people talking about “AI agents” are still thinking about chatbots with extra steps. The actual shift happening right now is infrastructural — a whole stack of tools, frameworks, and protocols is being assembled underneath these agents, and whoever understands that stack earliest has a serious advantage. Whether you’re building agents or deploying them inside a business, the decisions you make at the infrastructure layer will determine whether your agents are reliable workhorses or expensive toys that hallucinate their way through tasks and fall apart when something unexpected happens.
This is the agent stack — broken down clearly, with real tools, real trade-offs, and an honest picture of what’s production-ready versus what’s still duct tape and hope.
What the Agent Stack Actually Is
An AI agent isn’t just a language model. It’s a system. The model is the reasoning core, but around it you need memory so it doesn’t forget what happened three steps ago, tools so it can actually do things in the world, an orchestration layer to manage multi-step workflows, and infrastructure to handle things like rate limits, retries, logging, and cost tracking. Strip any one of those out and you don’t have an agent — you have a very expensive autocomplete with ambitions.
Think of it in layers:
- Model layer: The LLM doing the reasoning (GPT-4o, Claude 3.5 Sonnet, Gemini 2.0 Flash, Llama 3.3, etc.)
- Tool layer: APIs, code execution environments, web search, file systems — everything the agent can interact with
- Memory layer: Short-term context, long-term vector storage, and episodic recall
- Orchestration layer: The framework or runtime managing agent logic, loops, and multi-agent coordination
- Observation and control layer: Logging, evals, guardrails, human-in-the-loop checkpoints
Andrej Karpathy has described LLMs as the “new kernel” — the core compute layer on top of which everything else is built. That framing is useful here: just like operating systems evolved a whole ecosystem of tools on top of a CPU, the agent stack is the ecosystem building on top of the model. The kernel matters, but so does everything running on it.
The Orchestration Frameworks: Where the Real Decisions Get Made
The orchestration layer is where most developers spend most of their time, and it’s also where the ecosystem is most fragmented. Here are the frameworks actually being used in production as of early 2026:
LangChain and LangGraph
LangChain became the default starting point for agent development in 2023, and it’s still widely used — but it’s also widely cursed at. The abstraction layer is heavy, debugging is painful, and it can obscure what’s actually happening at the model level. LangGraph, built on top of LangChain, adds a graph-based execution model that’s much better suited for complex, stateful, multi-step agents. If you’re doing anything beyond a simple linear chain, LangGraph’s node-and-edge structure gives you finer control over branching logic and state management. It’s still verbose, but it’s more honest about what it’s doing.
CrewAI
CrewAI focuses specifically on multi-agent coordination — the idea that you build a “crew” of specialized agents that collaborate on a task. A researcher agent, a writer agent, and a fact-checker agent, all passing outputs to each other. It’s higher-level than LangGraph and faster to get running. The trade-off is that you give up fine-grained control. For prototyping multi-agent workflows, it’s genuinely good. For production systems where you need precise observability, you’ll likely hit its limits.
AutoGen (Microsoft)
Microsoft’s AutoGen is the serious research-and-enterprise option for multi-agent systems. It’s been rebuilt significantly with AutoGen 0.4, moving to an asynchronous, event-driven architecture that handles complex agent conversations and human-agent collaboration more robustly. It’s more complex to set up than CrewAI but gives you more control over how agents communicate, interrupt, and defer to humans. Microsoft is betting heavily on this as the foundation for enterprise agentic workflows inside Azure.
OpenAI Agents SDK
OpenAI released its own Agents SDK in early 2025 (the successor to the Swarm experiment), giving developers a native way to build agents that use OpenAI models with built-in tool calling, handoffs between agents, and guardrails. It’s tightly coupled to OpenAI’s ecosystem, which is either a feature or a limitation depending on your architecture. If you’re all-in on GPT-4o or o3, it’s the lowest-friction path.
Semantic Kernel (Microsoft)
Semantic Kernel is aimed at enterprise developers, particularly those working in C# or Java as well as Python. It’s more opinionated about how agents are structured and integrates directly with Azure AI services. Less popular in the open-source community but showing up a lot in enterprise .NET shops building internal automation tools.
The Tool Layer: What Agents Actually Do With
An agent without tools is just a chatbot. The tool layer is what gives agents agency — the ability to take actions, not just produce text. The key tools in any real agent stack include:
- Web search: Tavily has become the default search API for agent frameworks because it’s designed specifically for LLM consumption — returning clean, relevant excerpts rather than raw HTML. Brave Search API is a solid alternative. Basic Google or Bing API results require more post-processing.
- Code execution: E2B (formerly known as “e2b.dev”) provides sandboxed code execution environments that agents can use to run Python, analyze data, or test outputs. OpenAI’s Code Interpreter (inside ChatGPT) does this natively for end users. For production agent pipelines, E2B is the go-to.
- Browser control: Playwright and Puppeteer have been used for browser automation for years, but the new generation of tools — Browserbase, Steel, and Stagehand — are built specifically for AI agents navigating web interfaces. Stagehand, built by Browserbase, lets agents interact with web pages using natural language instructions mapped to DOM actions.
- File and data access: Agents reading and writing files, querying databases, or processing documents typically go through custom tool definitions or standard integrations. The Model Context Protocol (MCP), released by Anthropic in late 2024, is becoming a standard interface for connecting agents to data sources and tools — more on this below.
- External APIs: Any real business workflow involves CRMs, ERPs, communication tools, and internal databases. Composio has emerged as a useful abstraction layer here — it provides pre-built, auth-handled integrations with over 100 tools (Salesforce, Slack, GitHub, Linear, etc.) that agents can call without you having to manage OAuth flows for each one.
Memory: The Layer Most Builders Get Wrong
Memory is where agent systems fall apart most often, and it’s also the most underappreciated part of the stack. There are three distinct types, and conflating