Recently, Jim Fan, Head of NVIDIA GEAR Lab and Distinguished Scientist, gave a 20-minute talk titled “Robotics’ End Game” at the AI Ascent 2026 conference hosted by Sequoia Capital in April. Unlike last year’s cautiously optimistic remarks about the “Physical Turing Test,” this time he brought a bolder roadmap: directly declaring that the VLA (Vision-Language-Action) architecture that has dominated the robotics field over the past three years is obsolete, and proposing an entirely new paradigm called World Action Models (WAM). This is not merely a shift in technical approach, but a strategic declaration aimed at achieving “Physical AGI” by 2040.
From DGX-1 Signatures to Underlying Isomorphism: A Decade-Spanning Parallel Replication
Jim Fan opened with a highly symbolic personal experience. In the summer of 2016, at OpenAI’s office at the time, NVIDIA founder Jensen Huang, wearing his signature leather jacket, walked in carrying a large metal tray with an inscription that read: “To Elon and the OpenAI team, to the future of computing and humanity — I present you with the world’s first DGX-1.” At the time, Jim Fan was OpenAI’s first intern and rushed to line up to sign it; Andrej Karpathy was there signing alongside him. This DGX-1 is now on display at the Computer History Museum. Jim Fan joked that he feels like a dinosaur getting old.
This story isn’t just an emotional opening, but the central metaphor of the entire talk. Jim Fan quotes his mentor Ilya Sutskever’s famous line: “If you believe in deep learning, deep learning will believe in you.” He then points out that large language models (LLMs) reached where they are today in just three leaps over six years: from GPT-3’s pretraining, to InstructGPT’s supervised fine-tuning, to o1-style reinforcement learning, and then to automated research. He then makes a key decision: copy the LLM playbook and rename it “the Great Parallel.” Rather than simulating the next string state, simulate the next physical world state; converge to the small slice robots need through action fine-tuning, and let reinforcement learning complete the last mile. As he put it: “If you can’t beat them, join them.”
The End of the VLA Era: Language Isn’t Everything
Over the past three years, the dominant architecture in robotics has been VLA (Vision-Language-Action). NVIDIA’s own GR00T humanoid robot foundation model and Physical Intelligence’s π0 both fall into this category. Theoretically, the common sense and reasoning capabilities provided by language models should empower robots, but Jim Fan has pointed out a structural problem.
He put it bluntly: these models should actually be called LVA (Language-Vision-Action), because most parameters are allocated to language. Language is the first-class citizen, followed by vision, with action relegated to the bottom. “VLAs are very good at encoding knowledge and nouns, but not so good at physics and verbs. The emphasis is in the wrong place.”
He cited a classic example from the original RT-2 paper: having a robot push a soda can next to a photo of Taylor Swift. The model did accomplish this, but the problem is that what it could generalize was the noun “Taylor Swift,” not the verb “how to push, at what angle, with how much force.” This is precisely the inherent limitation of the VLA system: the parameter configuration is top-heavy, causing physical understanding capabilities to never break through. Jim Fan explicitly stated in his talk that while this approach has completed its historical mission at this stage, it’s no longer suitable to continue down this path.
DreamZero Debuts with the World Action Model (WAM)
Since VLA isn’t the final answer, what’s the next stage of pre-training paradigm? Jim Fan’s team’s answer is video models. Research found that video diffusion models have already learned internally to simulate the next state of the physical world. The key question is: how do we turn these world models into usable robot policies?
NVIDIA’s answer is a 14 billion parameter model called DreamZero. This is a new type of policy model that “dreams” a few seconds into the future before taking actions, acting based on those dreams while simultaneously decoding the next frame and next action. Here, vision and action have truly become “first-class citizens” for the first time. Jim Fan named this new architecture the World Action Model (WAM), and candidly admitted that DreamZero is currently roughly at the GPT-2 stage: the direction is right, but performance is not yet stable and reliable enough.
Notably, DreamZero has an extremely high actual deployment threshold. According toPaperDisclosed: this 14B model requires 38x system-level optimization, paired with GB200 hardware, to achieve 7Hz closed-loop control. This means that at this stage, the WAM approach is more of a technical declaration than an immediately deployable product. However, the academic community’s response to this approach has been quite positive: by March 2026, Fast-WAM and others have already emerged.Follow-up researchExplores whether the world action model requires future-oriented imagination in its testing phase.
Data Revolution: From Teleoperation to Robot-Free Data Collection
The past three years have been the golden age of teleoperation, but Jim Fan has pointed out the hard ceiling of this approach: at most 24 hours per robot per day. “Saying it’s 24 hours is just fooling yourself. Getting three hours of actual work done in a day would be pretty good, and that still depends on whether the ‘robot god’ is in a good mood that day—after all, these machines throw tantrums and break down constantly.”
To solve this bottleneck, NVIDIA proposed a completely different strategy: bypass the robot entirely and collect data directly from humans. The specific solution is DexUMI: an exoskeleton device worn directly on human hands to record finger movements. Robot policies trained with exoskeleton data can run fully autonomously, with no teleoperation data whatsoever in the training set. For the first time, robots don’t need to participate in the data collection process at all.
The significance of this technology is profound. The typical bottleneck in data collection capacity shifts from the physical limitations of robotic hardware (requiring maintenance, recharging, and prone to failure) to human participation limits, and humans can theoretically be replicated at scale. This lays the foundation for scaling at the next stage.
EgoScale and the Scaling Law of Dexterous Manipulation
If DexUMI addresses the data collection problem, EgoScale tackles the data quantity problem. In NVIDIA’s EgoScale approach, 99.9% of the training data comes from human egocentric video: pretraining used 21,000 hours of in-the-wild human data with no robot data involved at all. The motion fine-tuning stage only required 50 hours of high-precision motion capture glove data plus 4 hours of teleoperation data—together accounting for less than 0.1% of total training volume.
The most important finding of this study is that a neural scaling law for dexterous manipulation does indeed exist. There is an extremely clear log-linear relationship between the compute hours invested in pre-training and the optimal validation loss, with a correlation coefficient R² as high as 0.998. This means that as long as human video data continues to be increased, the robot’s dexterous manipulation ability can be predictably and continuously improved.
Jim Fan compares the scalability of all data strategies: teleoperation sits at the least scalable end, while first-person video, if it can spin a data flywheel similar to FSD, could accumulate 10 million hours within a year.
Dream Dojo: A Data-Driven Simulator Without a Physics Engine
The robotics field also needs to spend big money on millions of training environments for reinforcement learning, just like LLMs do. The traditional approach is real-to-sim-to-real, but the sim-to-real gap has never been fully eliminated. NVIDIA’s take it further is Dream Dojo: skip the physics engine entirely, and turn a video world model into a complete neural simulator. Input is continuous action signals, and it outputs the next RGB frame and sensor states in real time. No physics equations, no rendering engine—purely data-driven, as Jim Fan put it: “Not a single pixel in what you see is real.”
This statement not only sounds staggering, but also hides a layer of business logic: compute = environment = data. In Jim Fan’s own words: “Or as some wise person put it: the more you buy, the more you save. This message has been approved by my boss.” This joke directly points out NVIDIA’s core business strategy in this game: whether WAM or neural simulators, the demand for compute far exceeds that of traditional VLA architecture, and this is precisely NVIDIA’s sweet spot for chip business.
Endgame Roadmap: Three Major Milestones Before 2040
At the end of his speech, Jim Fan likened the robot’s remaining path to three tech tree achievements that must be unlocked.
First, the Physical Turing TestWithin 2-3 years, you won’t be able to tell whether a human or a machine is performing the task. This was the theme of his speech last year at the same event, and has now become the first step in a more comprehensive roadmap.
Second, Physical APIOrchestrating robot configurations with software and large models to build “Lights Out Factories” and automated scientific laboratories. The core of this step lies in transforming robots from hardware problems into software problems.
Third, Physical Auto ResearchRobots begin designing, improving, and manufacturing the next generation of robots themselves. This is the ultimate goal, aligning with the “Automated Research” stage in the LLM roadmap.
As for the timeline, Jim Fan gave a 2040 prediction with 95% confidence. His reasoning goes like this: AI took 14 years to go from AlexNet (2012) to intelligent agents (2026); give robotics another 14 years, and that lands exactly at 2040. He concluded with a rather thought-provoking quote: “We, the current generation, were born too late to catch the Age of Exploration to discover Earth; yet too early to reach the stars and oceans to explore the cosmos. But we were born at just the right moment—to tackle the grand challenge of robotics.”
The lecture videos above have also been translated — feel free to check them out if you’re interested:
Source: KOCPC Chinese