In the past three years, GPU can be said to be the only star in the AI industry. From training large models to inference services, every investor briefing and every financial report conference call focuses on GPU computing power and shipments. But in 2026, due to the rise of AI Agent, AI is no longer just a chatbot that answers questions, but an autonomous system that can read files, execute code, call tools, and coordinate multiple subtasks. When AI shifts from “generating responses” to “completing tasks”, the interaction model between people and AI has changed from “tokens in, tokens out” to “tokens in, actions out”. Generating tokens from the model is only the first step, and a large number of subsequent operations such as file reading and writing, program code compilation, tool calling, and sub-agent coordination all fall on the shoulders of the CPU. If the CPU is not fast enough, the expensive GPU can only idle and wait.
Why Agents Make CPU Important
The operation mode of traditional chatbots is very simple: the user throws a prompt, the model infers the token, and the response is displayed on the screen. The entire process is centered on the GPU, and the CPU is only responsible for scheduling and I/O. One CPU with four to eight GPUs in a server is enough. But AI Agent works in a completely different way. After an Agent receives a task, it will break the goal into multiple steps, decide what to do next, call multiple models, query the database, connect to the API, execute the program code, check permissions, verify the output results, and then repeat the entire process in a loop. A more advanced system will start multiple sub-agents to work in parallel at the same time. Each sub-agent performs reasoning, reads files, calls tools, and processes results.
In this architecture, the GPU is still responsible for model inference (the “thinking” part), but between each inference call, the CPU takes on a lot of work: parsing model output, deciding which tool to call, processing API requests, managing memory, and coordinating dependencies between sub-agents. A study published by Georgia Tech and Intel found that CPU-side tool processing accounts for 50% to 90% of overall latency in Agentic workloads.
A vice president of Intel put it more bluntly in an interview: “Agentic AI is a combination of a group of independent Agents. If there are 10, 20, 30, or even 100 Agents in a workflow, and they all need to communicate with each other, they require different computing locations. When I say location, I mean the CPU.”
What exactly is the CPU doing?
To understand why the CPU becomes a bottleneck, we must first look at the specific tasks it is responsible for in the Agent workflow:
- Orchestration: Decompose complex tasks into sub-steps, determine the execution sequence and dependencies, and manage the parallelism and synchronization of multiple sub-Agents.
- Tool Calls: Trigger API, execute Bash commands, run Python scripts, compile program code, and search databases. All the above operations are completed on the CPU.
- Data processing: Parse the JSON output by the model, process CSV/file reading and writing, perform SQLite queries, compression and hash operations.
- Security and permissions checks: Perform permission verification and security policy check on each autonomous action of the Agent.
- memory management: Maintain the context memory of the Agent, retrieve historical information, and manage the shared status between multiple Agents.
The Georgia Tech paper further quantifies the problem. In the RAG (Retrieval Augmented Generation) workload, the dynamic energy consumption of the CPU accounts for 61% of the total system dynamic energy consumption; in the Web search agent, it accounts for 43% to 57%; in the chemical reasoning agent ChemCrow, it accounts for 35% to 55%. The above figures show that even if the GPU is responsible for model inference, the CPU still accounts for a high proportion of operations in the overall system.
From 1:8 to 1:1, the CPU-GPU ratio is flipping
In the era of chatbots, the typical configuration of an AI server is 1 CPU with 4 to 8 GPUs. AMD pointed out that after entering the Agentic era, this ratio is moving towards 1:1 or even higher on the CPU side. This is not a problem that can be solved by simply plugging a few more CPUs into a GPU server. AMD believes that the solution is to add a dedicated CPU rack next to the GPU rack to form a distributed architecture: the GPU rack is responsible for intensive model calculations, and the CPU rack is responsible for orchestration, data processing, and tool execution.
This structural shift is being reflected in market data. AMD’s data center division achieved revenue of US$5.4 billion (approximately NT$175.5 billion) in the fourth quarter of 2025, an annual increase of 39%. CEO Su Zifeng specifically pointed out in the earnings call that demand for EPYC processors is soaring because “Agentic and emerging AI workloads require high-performance CPUs to drive head nodes and execute tasks in parallel with GPUs.” AMD estimates that the potential market size of server CPUs will expand at a compound annual growth rate of more than 35%, exceeding US$120 billion (approximately NT$3.9 trillion) by 2030.
Actual measurement: 6 parallel agents, the new CPU throughput is 6 times that of the old machine
AMD published a set of actual test data in its blog in July: using the Codex developer workflow to execute 6 ChatGPT 5.5 High Agents simultaneously, each Agent performs various local tool tasks such as AST analysis, compilation testing, unit testing, JSON/CSV serialization, SQLite query, compression hash, etc. As a result, the ASUS ProArt system equipped with Ryzen AI Max+ processor has 6 times the CPU throughput of a four-year-old laptop.
The significance of this set of data is that when multiple Agents execute a large number of tool calls locally at the same time, CPU performance directly determines the task completion time. No matter how fast the model inference is, if every action is waiting for the CPU to execute, the overall workflow will be slowed down. To use a simple metaphor: GPU is a machine in a factory, and CPU is a worker who carries raw materials and finished products. No matter how fast the machine is, if workers can’t move it, the entire production line still can’t move faster.
What it means to developers in general
This trend not only affects large cloud vendors. For developers who run AI Agents locally, CPU performance is also the key to determining the quality of the experience. When you use Claude Code, Codex or other AI programming assistants to open multiple Agents at the same time to write programs, run tests, and execute code, the number of CPU cores, single-core performance, and memory bandwidth directly affect the response speed.
AMD’s suggestion is pragmatic: the standard for measuring AI Agents has shifted from “how fast to generate tokens” to “how fast to complete tasks.” Network conditions and cloud inference speed will indeed affect the experience, but when the Agent starts reading files, running code, controlling applications, and coordinating tools on your machine, the local CPU becomes a key part of the overall task time.
For three years, the entire industry has focused its attention and budget on GPUs. The rise of Agentic AI reminds us that AI systems are more than just a chip. When wisdom begins to be transformed into action, the CPU is the hand that turns the reasoning results into practical work. In this new era, only upgrading the GPU and ignoring the CPU is like buying a supercar but equipped with bicycle tires. No matter how powerful the engine is, the tires can’t grip the ground, and the car just won’t run.
Source: KOCPC Chinese