Choose a GGUF for the phone, not for the download list.

A repository can publish base and instruct checkpoints, multiple architectures, vision projectors, several quantizers, and many bit levels. For conversation, choose the intended instruct model and the matching chat template. Confirm the runtime version supports the architecture; a newly released GGUF can be valid while an older app cannot load it. Read the model license and any gated access terms before downloading.

Multimodal models may need an additional projector file or a runtime-specific package. Reasoning models can require a new template or parser. Unsloth's Qwen3.5 work notes that a tool-calling chat-template issue affected quant uploads regardless of provider, which illustrates why a model that produces malformed output may need a template fix rather than a different quantization.

Lower-bit files reduce storage and weight memory, but they do not degrade every tensor equally. Unsloth Dynamic 2.0 selects quantization per layer and uses model-specific schemes. Its Qwen3.5 research reports more than 150 KL-divergence benchmarks and 9 TB of artifacts, and warns that attention tensors and some state-space outputs are particularly sensitive. It also reports that imatrix processing can improve quality metrics while costing roughly five to ten percent inference speed.

These findings do not mean one branded quant is always best. They show why a generic 'Q4 equals good' rule is incomplete. Compare the exact model, quantizer, file size, and real task results. At phone scale, start with a reputable three- or four-bit release when needed for fit, then move upward if memory permits and answer quality matters. Avoid two-bit files unless the size constraint is decisive and the target tasks have been tested.

The GGUF size is a lower bound for storage planning and an incomplete proxy for RAM. The runtime maps or loads weights, allocates compute buffers, and grows a key-value cache with context. Android and the rest of the app need their own memory. A phone advertised with eight gigabytes does not offer all eight to one process, and free memory changes while the device runs.

Use a short context during the first load, keep at least several gigabytes of storage headroom around large downloads, and test after other everyday apps have been used. If the process dies when conversations grow, reduce context before blaming the model. For a model that cannot fit reliably, move the same GGUF to a user-controlled computer and connect through a local server rather than repeatedly forcing the handset.

llama-bench distinguishes prompt processing from token generation and can emit machine-readable output. On a phone, record both because a model can generate acceptably after a long initial wait. Run multiple repetitions, keep thread count and context fixed, report the runtime commit, and note whether the device was cool. Do not compare a 512-token prefill number from one backend with a 128-token decode number from another.

Quality also needs more than perplexity. Unsloth explains that KL divergence can reveal distribution changes and that calibration data can bias both perplexity and KLD. Real-task evaluations may disagree with those metrics. For choosing a personal model, create a small private evaluation set—drafting, extraction, refusal, formatting—and compare answers blind. The most useful GGUF is the smallest one that remains stable and good enough for the user's actual work.