Local Models
A local model runs on the device itself. Once it has been downloaded, the AI assistant works with no account, no API key, and no internet connection at all — the request never leaves your phone or tablet.
Setting one up
Add a service the same way as any other — see AI Services — and choose one of the two models below instead of a provider. There is no address to type and no API key to enter: a local model needs neither, and the service form does not ask for them.
Choosing a model shows its licence before anything downloads. Grafida does not include or distribute any model; it downloads the exact files you chose, once, directly from Hugging Face. Each model is published by its own authors under its own licence, which is not Grafida's to change or to promise on their behalf.
The two models
| Qwen3.5 2B | Gemma 4 E4B | |
|---|---|---|
| Download size | about 1.3 GB | about 6.3 GB |
| Sees pictures | No | Yes |
| Licence | Apache-2.0 | Apache-2.0 |
Qwen3.5 2B is the smaller of the two: a faster download, a smaller footprint on your device, and text only — it cannot see an article's pictures, only read its words.
Gemma 4 E4B is larger and can see images, so it is the one to choose if you want the assistant to look at a picture in the article as well as its text. The larger download and the extra memory it needs are the cost of that.
Neither is a substitute for a large hosted model. Expect shorter, plainer answers, and expect it to take noticeably longer to reply than a hosted provider does — the whole model is running on your device's own processor, with nothing behind it doing the work faster.
Be prepared for that to mean minutes rather than seconds. Qwen3.5 2B works through a problem before it answers, and on a recent phone a short reply took a little over four minutes, most of it spent on reasoning you never see. That is normal for a model this size running on a phone, and it is the trade being made: a reply that costs nothing, needs no account and works with the aeroplane mode switch on, in exchange for waiting. If you want an answer quickly, a hosted service is the right tool and Grafida supports both side by side.
Downloading
The download is large enough that it is built to be interrupted. Leaving the app, losing signal, or switching to a different task does not lose your progress — coming back and choosing Download again carries on from where it stopped rather than starting over.
Deleting a downloaded model does not remove the AI service that used it. The service stays configured exactly as it was; it simply cannot answer anything until the model is downloaded again.
Whether your device can run one
Grafida checks two things before it lets you download a model: whether your device has enough memory, and whether it has enough free storage.
Memory is the one that decides whether a model is offered at all. The figure Grafida checks against is the amount of memory Android itself reports, which is a little less than the number printed on the box — the operating system reserves some of it before anything else can use it. Against that reported figure, Qwen3.5 2B needs at least 6.75 GiB, and Gemma 4 E4B needs at least 12.75 GiB. A device below the figure its chosen model needs does not offer that model at all; there is nothing to press and nothing to work around, because no free space and no waiting changes how much memory the device physically has.
In practice that means Qwen3.5 2B suits a phone or tablet sold with 8 GB of memory or more, and Gemma 4 E4B suits one sold with 16 GB. Those figures come from measuring the whole application while it was actually generating an answer, on real devices, rather than from the size of the model file: running a model needs considerably more memory than storing it does.
Storage is checked as well, on top of the download itself, so a model download does not fill your device to its very last byte. Unlike the memory check, this one can simply be fixed — free up some space and the model becomes available again.
There is one further requirement, and a device that does not meet it is not offered local models at all: the processor must support a set of instructions that quantised models rely on to run at a useful speed. Every 64-bit Android device sold since about 2019 has them. Grafida checks before it starts anything, so a device without them is told plainly rather than being allowed to try.
Grafida does not try to guess whether your particular phone's processor is fast enough to be pleasant to use — there is no reliable way to know that in advance across the huge range of Android hardware, and a wrong guess would refuse phones that would have worked fine. The only question Grafida asks in advance is whether the model fits in memory at all.
What a local model cannot do
A local model runs entirely inside Grafida's own process and has no way of keeping a copy of a conversation anywhere else. Where a hosted provider can optionally store the conversation on its own servers so a follow-up message does not need to resend everything — see AI Services — a local model never does: every message Grafida sends is built fresh from its own record of what was said, every time. There is no equivalent setting to turn on, and nothing changes about reopening a saved chat later; see AI Chat for what reopening a saved conversation does and does not carry forward, which applies here exactly as it does to a hosted service.