A while ago, I introduced something that can be used without an internet connection. H2O – Free AI Personal GPT Small Language Model AppAlthough H2O can really be used offline, to be honest, the results aren’t that good. Today I’m introducing an app that also works offline and is free, but with much better results: MLC Chat.
Free and powerful: MLC Chat, an offline AI language model for Android and iOS users
MLC Chat is part of the open-source project MLC LLM. MLC Chat lets you run different open model architectures on your phone, and you can run open language models downloaded from the internet. This allows you to have conversations with large language models offline using only your phone’s processor. The best part is that both Android and Apple users can use it!
Once you’ve downloaded MLC Chat, open the app. After opening the app, you can pick the large language model you want to use from the model list. Right now, gemma-2-2b is the smallest and best-performing model available, so you can start by selecting Gemma 2 2B to try it out. (Gemma 2 2B is a compact model recently released by Google that has been distilled and pruned, with a size of only about 1 GB. On the LMSYS Chatbot Arena leaderboard, it outranks all GPT-3.5 models and even surpasses Mixtral-8x7b!) Alternatively, you can click “Edit model” to add other models you’d like to use.
The editor used gemma 2 to test MLC Chat. Recommending attractions, planning itineraries, and translating all work. You can check out the results here:
The editor thinks what’s good about MLC Chat is that when you ask it to help plan your itinerary, it will ask you to tell it your travel needs, budget, and so on, giving you a feeling of being sincerely treated. It will also give you suggestions on transportation, beauty, shopping, and climate for your trip.
But sometimes errors occur, for example, when MLC Chat was asked to recommend attractions in Kaohsiung, it recommended Anping Elementary School in Tainan. It also answered incorrectly when asked about the time difference between Washington, D.C., and Taiwan.
It seems MLC Chat can still experience AI hallucinations, so everyone should be careful about the answers MLC Chat provides. Of course, it could also be because I chose the gemma 2 model; you can try selecting other models! (To use MLC Chat, your phone needs more than 4GB of RAM, and basically most current mid-to-high-end phones meet this requirement)
If you’re interested in the other large language models we’ve introduced before that can be used on mobile devices, you can click below to read more. I personally recommend using the built-in iMessage on iPhone to use ChatGPT for free—it can do all of the above, plus generate images. It’s really versatile, and the setup is very simple.
Source: KOCPC Chinese