In the past, people’s understanding of large model training was that it required flagship GPUs from several computer rooms and hundreds of millions of dollars in training costs. Unless it was a large company, it was almost impossible for ordinary people to complete it with a home computer or simple equipment. However, recently there is a new open source project MiniMind, which can complete the training of a 64M parameter GPT model from scratch with only a consumer-grade NVIDIA RTX 3090 graphics card, about two hours of training time, and a cost of about NT$13. The project is fully open source under the Apache 2.0 license and has currently received approximately 47,000 stars and nearly 6,000 forks on GitHub.
MiniMind: An open source project that ordinary people can “train large models from scratch”
Not calling API, but “really making your own model”
MiniMind is not another set of AI framework abstraction layers similar to Hugging Face Transformers, nor is it an additional layer of fine-tuning tools; it is a complete set of reproducible, readable, and modifiable “create large models from scratch” teaching and implementation warehouses. Implemented with pure PyTorch native, the main core structures (attention, FFN, position coding, etc.) are all handwritten, and there are almost no black boxes. The key point is not only that it can operate, but also that it can be fully understood.
只要 2小时 + 几块钱 + 一张普通显卡，你就能从0训练一个 GPT。
MiniMind 这个开源项目已经冲上 GitHub Trending，斩获 47k ⭐。
这是一个把大模型带回普通人手里的项目：
不是调 API，而是真正从0造模型。核心亮点
1. 门槛极低
– 26M 参数，小到个人设备可跑
– 单卡即可训练
– 成本极低… pic.twitter.com/maTXmHbJ2t— 开发者Hailey (@IndieDevHailey) April 18, 2026
Model size: Multi-size lineup from 26M to 198M MoE
MiniMind currently provides several models of different sizes to correspond to different hardware conditions.Official README The files with GitHub Pages are organized as follows:
- MiniMind2-Small: 26M parameters (lightweight version, can be run on a personal laptop)
- MiniMind2:104M parameters
- MiniMind2-MoE: 145M parameters (Mixture of Experts)
- MiniMind-3: 64M parameters (the latest main dense architecture)
- MiniMind-3-MoE: 198M / A64M parameters (latest MoE version)
Among them, MiniMind-3 adopts a streamlined architecture of 768-dimensional hidden size and 8-layer Transformer. The model file size is about 0.5GB (dense) to 1.0GB (MoE). It can be loaded smoothly on local inference frameworks such as ollama and llama.cpp.
Complete training link: not just pretrain, but also RLHF, Agent RL, and distillation
Another highlight of MiniMind is that it replicates the complete process of mainstream large models from pre-training to alignment. The training phases listed in official documents include:
- Data preprocessing and tokenizer training
- Pretrain
- Supervised fine-tuning (SFT)
- LoRA low-rank fine-tuning
- RLHF stage of DPO (Direct Preference Optimization)
- RLAIF series: PPO, GRPO, CISPO and other reinforcement learning algorithms
- Agentic RL, let the model learn tool calls
- knowledge distillation
This allows MiniMind to not only “train a small model that can speak”, but to completely demonstrate the feasible paths for every link in the large model training pipeline. It is a rare systematic teaching material for students, researchers and engineers.
Cost: RTX 3090 for two hours, less than NT$13
According to the training records disclosed by the project author on the official website, taking MiniMind-3 (64M) as an example, it only takes about 2.31 hours to complete the two stages of Pretrain + SFT on a single NVIDIA RTX 3090. The total electricity and cloud costs are about 3 yuan, which is less than NT$13 after conversion; its parameter scale is about GPT-3 1/2700, but it already has a complete prototype of the ability to have conversations, perform simple reasoning, and call tools.
The author’s own development environment is quite luxurious: Intel Core i9-10980XE, 128GB memory, 8 RTX 3090s, Ubuntu 20.04, CUDA 12.2, Python 3.10.16; however, the official also emphasized that ordinary users only need one consumer-grade graphics card to completely reproduce all training processes.
Ecological compatibility: can connect to ollama, vLLM, and provide OpenAI API interface
The weights trained by MiniMind can be directly loaded into mainstream inference backends, including Hugging Face Transformers, llama.cpp, vLLM, SGLang and ollama; more importantly, it exposes an interface compatible with the OpenAI API, which means that local users can directly point FastGPT, Open-WebUI, Dify, ChatGPT-compatible ChatUI and other applications to MiniMind and use it as a private ChatGPT without sending data to the external cloud.
The project also comes with the visual multi-modal extended version MiniMind-V, which supports YaRN positional encoding extrapolation, allowing small models to be extended to longer context lengths. In terms of reasoning ability, MiniMind has implemented <think> The labeled Adaptive Thinking and reasoning content shows that the structure can be compared with the thinking token mechanism of cutting-edge models such as Claude and GPT-5.
MiniMind does not pursue the rankings or the number of parameters, but turns “training a large usable model by yourself” into a feasible option for weekend projects. For educators, it is an almost irreplaceable teaching material: students can write all the way from tokenizer to RLHF; for independent developers, it is a privacy-friendly starting point for local AI prototypes; for corporate research teams, it is a small experimental platform that can quickly verify new algorithms. This “complete, transparent, and modifiable” open source feature is the main reason why it rushed to GitHub Trending in a short period of time.
Conclusion:
The significance of MiniMind does not lie in how big the model itself can challenge the flagship, but in that it thoroughly demonstrates the disassembly of large model technology with Apache 2.0 authorization, extremely low threshold, and complete pipeline. For engineers and researchers who want to seriously understand key links such as Transformer, RLHF, Agent RL, and distillation in large language model (LLM) technology, it may be one of the most worth cloning and reading line by line open source projects in 2026.
Source: KOCPC Chinese