In recent years, the development of large language models (LLMs) has grown rapidly, from OpenAI’s GPT series to DeepSeek R1, with their reasoning capabilities reaching unprecedented heights. In this process, “Chain-of-Thought (CoT)” has become an important approach to AI reasoning, where artificial intelligence improves accuracy through step-by-step deduction. However, the drawbacks of this approach are also evident: high computational costs and latency. Recently, Zoom’s research team proposed a new reasoning paradigm: “Chain-of-Draft (CoD),” which aims to mimic humans’ concise way of thinking, significantly reducing token usage and computational resources without sacrificing accuracy. According to statistics, when using CoD for the same tasks, token consumption can be reduced by 80%, and inference time is effectively lowered as well, making CoD a potential major trend in the AI field.
Zoom research team has proposed a new AI reasoning paradigm, “CoD” (Chain of Draft), which can reduce token usage by 80%.
Whether it’s OpenAI’s GPT-4o or DeepSeek R1, today’s LLMs already perform remarkably well at handling tasks through “chain-of-thought” reasoning. This approach lets AI break problems down step by step, much like how people write out the full derivation process when solving a math problem. However, the downside is that it consumes a large amount of computational resources, resulting in verbose responses and higher latency. In a paper on arXiv, the Zoom research team points out that humans don’t record every detail when solving problems; instead, they tend to jot down core information concisely, such as shorthand or key steps. So they designed “chain of draft,” which lets AI generate only the most streamlined intermediate reasoning steps, capturing the key information needed to solve a problem while avoiding unnecessary verbose output. The core of this new approach lies in mimicking human cognitive processes, enabling AI to operate more efficiently and cutting redundant computation in reasoning steps.
CoT vs. CoD: Using Arithmetic Problems as an Example
Let’s see how different reasoning methods perform on the same simple arithmetic problem.
Question:
杰森有 20 個棒棒糖。他給了丹尼一些棒棒糖。現在杰森有 12 個棒棒糖。杰森給了丹尼多少個棒棒糖？
Direct answer (standard method):
8
This approach directly outputs answers, but it lacks transparency and is prone to errors on more complex problems.
Chain of Thought (CoT):
-
Initially, Jason had 20 lollipops.
-
After he gave Danny some lollipops, he had 12 left.
-
Therefore, 20 – 12 = 8 should be calculated.
-
The answer is 8.
Although this method is relatively clear, the steps are lengthy and token consumption is high.
Draft Chain (CoD):
20 – x = 12；x = 20 – 12 = 8。#### 8
This method uses concise mathematical expressions to significantly reduce Token usage while retaining a certain degree of transparency.
The experimental results and advantages of the draft chain
The Zoom research team conducted multiple benchmark tests on CoD, including arithmetic reasoning, common sense reasoning, and symbolic reasoning, and the results showed:
-
On the GSM8k dataset, GPT-4o and Claude 3.5 Sonnet achieve 91.1% and 91.4% accuracy with CoD, respectively, while with CoT, the accuracy reaches 95.4% and 95.8%.
-
Although accuracy dropped slightly, CoD reduced token usage by 80% and lowered latency by 76.2% (GPT-4o) and 48.4% (Claude 3.5 Sonnet).
-
In commonsense and symbolic reasoning tasks, CoD still performs exceptionally well. For example, in the coin-flip test, GPT-4o token usage dropped by 68%, and Claude 3.5 Sonnet by 86%, while accuracy remained at 100%.
These results indicate that the draft chain can still maintain a considerable degree of accuracy while reducing computational resources.
The Transformation of Enterprise AI Deployment Costs
When enterprises deploy AI models, considerations of cost and computational resources are crucial. Similar to CoD, research last year proposed “Concise Thoughts (CCoT)” and “token budget-aware reasoning,” but these methods face challenges in real-world applications—for example, a fixed token budget may not suit all tasks. In contrast, CoD adopts a more flexible per-step token budget approach, making it applicable to a wide range of reasoning needs.
According to AI researcher Ajith Prabhakar’s analysis, if a company processes 1 million inference queries per month:
-
The cost of using CoT is approximately $3,800.
-
The cost of using CoD can be reduced to $760.
This means saving $3,000 per month, and the savings become even more significant when applied at scale. In addition, CoD can also improve the response speed of AI applications, especially in live customer service, education, and conversational AI, where its low-latency characteristic will enhance the user experience.
Will CoD become the new standard?
Although CoD offers the advantages of reducing costs and improving efficiency, OpenAI noted in a March 2024 post that “Chain-of-Thought Monitoring (CoT Monitoring)” is one of the key tools for supervising superhuman models in the future, because CoT makes AI’s reasoning process more transparent, enabling humans to monitor whether AI exhibits deceptive behavior.
Detecting misbehavior in frontier reasoning models
Chain-of-thought (CoT) reasoning models “think” in natural language understandable by humans. Monitoring their “thinking” has allowed us to detect misbehavior such as subverting tests in coding tasks, deceiving users, or giving… pic.twitter.com/uX9f5n3zB9
— OpenAI (@OpenAI) March 10, 2025
Therefore, in the context of AI safety and regulation, CoT still holds irreplaceable value. However, when it comes to reducing computational costs and increasing the accessibility of AI, CoD is undoubtedly a key technology. As the AI industry continues to evolve, future reasoning approaches may blend the transparency of CoT with the efficiency of CoD, giving rise to a hybrid reasoning model that combines accuracy, explainability, and low cost.
Summary
From “Chain of Thought” to “Chain of Draft,” AI reasoning technology continues to evolve. Whether aimed at reducing costs, improving efficiency, or accelerating AI adoption in enterprises and everyday applications, CoD brings entirely new possibilities to LLMs. In today’s fiercely competitive AI landscape, striking the optimal balance between performance, cost, and security will be a key challenge for technological development—and the emergence of CoD is undoubtedly an important step toward that goal.
Source: KOCPC Chinese