From AI Tools to an AI Operating System: What Karpathy Got Right (and What We Already Built)

The kernel: one model dispatching everything

In a normal OS, the kernel receives syscalls and routes them to resources — disk, network, memory, CPU. In an AI OS, the LLM plays that role: it receives an instruction in natural language and dispatches it to the right tool, API, or memory.

The practical consequence is that your architecture is your prompt discipline. Context isn't something you dump data into — it's RAM, and RAM is expensive and finite. Most "the AI gave a bad answer" problems are really "we filled the RAM with garbage" problems. Treating the context window as a managed resource — not a bucket — is the single highest-leverage design decision in an agentic system.

Threads: multi-agent is multithreading

Karpathy's OS analogy has a natural home for multi-agent systems: agents are threads. A CEO agent, department heads, group leads, workers — each is a process with a defined scope, coordinating through message passing rather than a shared global state.

This is not aspirational for us. It's how this company runs right now. A hierarchy of AI agents — strategy, operations, marketing, legal, engineering — each with its own role file, each reporting up, each coordinating through explicit messages. No shared mutable state where everyone steps on everyone; just clearly addressed handoffs. That's exactly the coordination model Karpathy argues for, in production, today.

Memory hierarchy: L1, L2, L3 — for real

The part that surprised us least, because we'd already built it: the memory cache hierarchy.

• L1 — in-context: fast, expensive, volatile. What the agent is holding right now.
• L2 — vector memory: a semantic store the agent queries when the answer isn't in-context.
• L3 — durable database: the persistent record of truth.

That's not a slide. It's our stack. In-context injection at the top, a vector database in the middle, a relational database at the bottom. Karpathy's memory hierarchy isn't a diagram we admire — it's the thing that lets an agent wake up fresh each session and still know what it decided yesterday.

Why this matters if you run a business (not a lab)

The AI-OS framing changes the buying question. It stops being "which AI feature do I add?" and becomes "what does the operating layer of my company look like, and who's already run it in anger?"

Most vendors are still selling features. Very few have actually operated a business where AI agents are the org chart — where the coordination, the memory, the resource management are load-bearing, not demo-ware. The gap between "we have an AI assistant" and "we run on an AI OS" is the gap between a script and a system.

We're not claiming the metaphor. Karpathy did that, and better than we could. We're claiming the boring part underneath it: we've been running the thing he's describing, and the operational lessons — context is RAM, agents are threads, memory is a hierarchy, coordinate by messages not shared state — are exactly where the value and the failure modes live.

If you're trying to figure out what an AI operating layer would mean for your team — not a tool, an operating layer — that's a conversation worth having.