As AI Large Language Models (LLMs) have gradually evolved from pure text generation tools into “Agents” with analytical and decision-making capabilities, and people are all contemplating one question: can these AIs truly understand the complex real world and make accurate predictions? Among these efforts, an experimental project called “AI Trade Arena” recently atHacker News technology community It sparked heated discussions and topped the site’s trending list on December 4, 2025. The experiment, launched by developers Kam and Josh, attempted to test AI capabilities in the most direct and ruthless way: letting AIs enter the stock market to manage investments and reviewing their returns.
AI Trading Arena: The $100,000 Showdown of Five Top-Tier Models
The core concept of this experiment is quite simple yet highly ambitious: give five of today’s most advanced large language models $100,000 in virtual funds each, let them trade in a real stock market environment for eight months, and see who can earn the most.
The contestant list is a veritable “all-star lineup” in the AI field, including:
-
OpenAI’s GPT-5
-
Anthropic’s Claude Sonnet 4.5
-
Google’s Gemini 2.5 Pro(Not the strongest Gemini 3 Pro yet)
-
xAI’s Grok 4
-
as well as those from China DeepSeek
The experiment established a closed environment called “Trade Arena.” Here, AI agents are not merely receiving instructions—they actively research stocks, digest news information, and execute trading decisions. To ensure the experiment’s fairness and realism, the development team built a sophisticated “time machine.” The backtest simulation’s time span was set from February 3, 2025, to October 20, 2025. This period is highly representative because it covers market stagnation (such as February 2025) and the subsequent growth surge (such as summer 2025), thoroughly testing AI’s adaptability across different market cycles.
Technical Architecture: How to Prevent AI from ‘Cheating’?
In backtesting experiments, the biggest challenge is preventing “data leakage.” If a model somehow knows tomorrow’s stock price in advance, the predictions become meaningless. To overcome this, Kam and Josh developed a rigorous time-segmented API system. Acting like a filter, this system ensures that each model, on every simulated “day,” can only access market data, news APIs, and company financial reports that were publicly available on or before that day. This means that when the AI is simulating trading for February, it absolutely cannot “see” earnings reports or breaking news that wouldn’t be released until March.
Furthermore, to avoid the model simply relying on rote memorization of historical stock prices in the training data to make profits, the team deliberately chose to test on a time period after each model’s training cutoff. This design is crucial, as it ensures that the AI’s performance comes from its real-time analytical and reasoning abilities, rather than from memory of historical data.
Battle results revealed: Grok takes the crown, Gemini finishes last
After eight months of intense struggle, the experimental results were intriguing and upended many people’s expectations. Ultimately, by Grok 4 Claiming the championship with a positive return of +56.10%, this model developed by Musk’s xAI delivered the best investment return rate. Closely following was China’s DeepSeek It ranked second with a positive return of 49.01%, and the gap between the two is not that large. Meanwhile, GPT-5 and Claude are roughly tied with positive returns around 27%, with only Google Gemini losing money.
Analyzing the investment portfolios of these five major models reveals a clear common trend among the profitable ones:Heavy position in tech stocksMost top-performing models coincidentally concentrated their funds in the technology sector, which also matches the market’s primary growth driver during that period in 2025.
However, Google’s Gemini 2.5 Pro However, it unexpectedly finished last in this competition. According to trading data analysis, Gemini is the only model holding a significant amount of non-tech stocks. This portfolio diversification strategy, traditionally viewed in financial theory as a sound way to spread risk, caused its performance to lag far behind rivals who boldly bet on tech stocks during this particular bull market cycle.
This raises an interesting discussion: Is Gemini’s decision-making too conservative, or is it misreading market sentiment? This is exactly what the team hopes the public will dig into through the Interactive Demo. Users can review the reasoning behind each trading decision and judge for themselves whether the AI was “lucky” or “genuinely skilled.”
The Art and Limitations of Backtesting
Although this experiment attracted widespread attention, the development team also maintained a high degree of rationality and transparency, laying out in detail the pros and cons of this “backtesting” approach. In terms of advantages, backtesting allows researchers to run models and evaluations at scale. Through simulation, multiple scenarios can be tested in a short period, yielding preliminary results with statistical significance. This efficiency is unmatched by live trading.
However, the limitations are equally obvious. After all, backtesting is an “approximation” of reality. It cannot fully simulate the competitive and adversarial nature of real financial markets. For example:
-
SlippageIn large transactions, the act of buying itself will push up the price, resulting in an actual execution price higher than expected.
-
Liquidity constraints (Volume/Liquidity constraints)In reality, some stocks may not be able to be sold immediately when you want to sell them.
-
Overfitting (Overfitting)The model may just be fitting to the characteristics of historical data, rather than capturing the true patterns of the market.
Although the team admits that the current data is not yet statistically sufficient to draw definitive conclusions, this remains an important first step in understanding model behavior, analytical capabilities, and predictive abilities. Kam and Josh emphasized that the long-term goal of this project is not merely to build a “money-making machine,” but rather to gain a deeper understanding of how AI agents operate in complex environments.
The financial market is an ideal place to evaluate AI because it possesses both “quantitative” and “qualitative” dimensions.
-
Quantitative analysisThrough professional financial tools such as Barra factor analysis, researchers can strip out variables to try to distinguish whether a model’s performance stems from skill or luck.
-
Qualitative analysisSince the decision-making process of LLMs is based on text-based reasoning, researchers can read the model’s “thinking process.”
This allows us to distinguish between “rote memorization” and “genuine reasoning.” For example, did the model buy NVIDIA stock because it “remembers” the stock will rise, or because it read the 10-K financial report and analyzed strong market fundamentals from it? By examining the decision logic, we can clearly tell the two apart.
Next Step: Toward the Real Battlefield
The story of the “AI Trading Arena” has just begun. The team has already mapped out the next three-phase development roadmap:
-
Continuous backtestingTest the model in more historical scenarios.
-
Real-Time Paper Trading (Live Paper Trading)Conduct simulated trading at the current point in time to completely eliminate the risk of “future data leakage.”
-
Live tradingEventually entering the battlefield where real money is at stake.
This experiment not only highlights the differences among mainstream models like GPT-5, Claude, Gemini, Grok, and DeepSeek in financial investing, but more importantly, it also offers a new window into observing how AI understands the world, processes information, and makes decisions. Whether or not AI can ultimately beat the market, the market itself will become the most honest touchstone for testing AI intelligence. So the question is: would you really trust AI with your money to invest? Feel free to join the discussion.
Source: KOCPC Chinese