When it comes to running AI language models, most people assume you need a high-end computer. In reality, that’s not necessarily the case. As technology becomes more mature, even extremely old computers can now run AI language models. Recently, a foreign team managed to do just that—running the llama98.c language model on a 26-year-old computer, with surprisingly decent response speed.
A super old computer from 26 years ago can run AI language models.
Recently, EXO Labs shared a detailed blog post about how to run the Llama language model on a 26-year-old Windows 98 computer.
According to the description, they are made up of researchers and engineers from the University of Oxford, with a mission to “democratize AI resources,” aiming to build open infrastructure for training frontier models so that anyone can run these models anywhere. This experiment also demonstrates that even ordinary people can train and run AI models on any device.
However, they didn’t have such an old computer, so they bought a Windows 98 Pentium II from eBay. The price was actually not cheap—£118.88, about NT$5,000:
Also, because this computer is really too old, it’s incompatible with many modern external devices, such as USB keyboards and mice. It has to use the old PS/2 connection interface, and the mouse must be plugged into port 1 and the keyboard into port 2—the other way around won’t work properly:
After it can operate normally, there is another challenge: how to put the model into the computer.
They tried, but the system couldn’t properly recognize the RW, and a 4TB hard drive is also too large for FAT32, so in the end they had to use the “old-fashioned FTP” method to transfer files via the Ethernet port.
EXO mentioned that compiling modern code for Windows 98 is a very tricky problem, and eventually found Andrej Karpathy’s llama2.c, which is 700 lines of pure C code that can perform inference on models based on the Llama 2 architecture. With this resource, along with the old Borland C++ 5.02 IDE and compiler (with some small tweaks), it was possible to create an executable compatible with Windows 98 and run it (source code can be found here:GitHub）：
They also recorded a short video. In it, you can see this old Pentium II computer boot into the Windows 98 operating system, launch a custom pure-C inference engine based on Andrej Karpathy’s Llama2.c, and ask the LLM to generate a story about Sleepy Joe. The impressive part is that the response speed is quite good:
LLM running on Windows 98 PC
26 year old hardware with Intel Pentium II CPU and 128MB RAM.
Uses llama98.c, our custom pure C inference engine based on @karpathy llama2.c
Code and DIY guide 👇 pic.twitter.com/pktC8hhvva
— EXO Labs (@exolabs) December 28, 2024
Of course, this language model is relatively small, only 260K, which is also one of the reasons why such an old computer, without a GPU graphics card, can still run it.
EXO shared test data: 260K has a speed of 39.31 tok/sec, but once the LLM is scaled up to 15M, generation speed drops to 1.03 tok/sec. Llama 3.2 1B is very slow, at only 0.0093 tok/sec.
EXO also admits that this speed isn’t entirely at ChatGPT’s level, but being able to run any modern AI model on CPU hardware from 25 years ago is still a major step forward for their mission.
If you want to learn more, you can read. EXO’s blog post。
Source: KOCPC Chinese