Many people want to run free open-source AI models on their own computers, but the biggest barrier right now is insufficient memory—especially for large models with tens of billions of parameters. Even after quantization reduces their size, they are still quite demanding for typical computers with only 8GB or 12GB of RAM or VRAM, leaving users to switch to smaller models or upgrade their memory and graphics card.
Recently, some new approaches have started to emerge. A tool called RAMDeck has appeared online that can distribute the required memory capacity across multiple devices, and one user shared test results, successfully running the Qwen3.8-27B model on an old laptop with only 12GB. Of course, the speed is still terrible, at only 1.92 Token per second, so for now it’s more like a technical demonstration.
Image source:Wccftech
A 12GB old laptop can also run Qwen3.8-27B: RAMDeck links 4 devices to distribute the model, but the speed is only 1.92 tokens per second.
Reddit user Medicine_Blogscanner recently on LocalLLM Community SharingHe said this was already the third video in the series. Previously, he had used the same concept to challenge 7B and 13B models, and this time he raised the target to Qwen3.8-27B, which has about 27B parameters. The old Windows laptop he had only had 12GB of memory, so it was clear that this computer alone could not fully load it.
He used the RAMDeck tool to split the model and distribute it across four devices. An old Windows laptop serves as the main node, along with an RTX 3060 mini PC, a Mac mini, and an Android phone. Below is the memory allocated to each device:
- The old Windows laptop is allocated 3.4GB.
- The mini PC provides approximately 20GB of video memory and system memory.
- Mac mini handles 3.7GB
- Android phones contribute 1GB
Adding the above numbers together, the allocated capacity listed for the four devices totals approximately 28.1GB, and the device providing the most resources is the mini PC equipped with an RTX 3060.
RAMDeck’s publicly available Core Engine uses llama.cpp’s RPC protocol to offload model tensors across devices. Each participating device runs a node program, providing GPU or CPU resources, and a compatible coordinator dispatches work.
One thing to know first is that the Qwen3.8-27B this user ran this time was not the fully uncompressed original model; it was a 4-bit quantized version, requiring about 15GB to 17GB of memory. If the unquantized version were used, the model weights alone could require about 55GB to 60GB. Quantization can store model weights using fewer bits, greatly reducing capacity requirements, but the more aggressive the compression, the more likely it is to affect output quality.
Although it runs successfully, the text generation speed in subsequent tests is quite terrible, at only 1.92 Token per second with latency around 25ms:
It’s actually not hard to understand why it gets this slow. With a single graphics card, data can be accessed quickly and directly over a high-bandwidth bus. But after RAMDeck distributes the model across different devices, the nodes also have to exchange data over the local network, which adds overhead from RPC communication, synchronization, and differences between CPU and GPU architectures. So performance is bound to be worse.
RAMDeck It looks pretty good, but there are still many issues at the moment. GitHub mentions that it “works but is not perfect,” and this repository does not have a built-in graphical interface or full dashboard. The installation process also requires setting up a Python environment, starting a node agent, and using compatible llama.cpp tools or preparing your own coordinator, which makes the barrier to entry very high for ordinary users.
Source: KOCPC Chinese