01The problem
Model accuracy alone does not explain whether an AI service is practical. Latency, memory, response size and access costs also matter.
02The approach
- Prepare 3,400 parallel knowledge, sentence-inference and mathematics items across Yoruba, Hausa, Igbo and English.
- Run Qwen2.5-1.5B and SmolLM2-1.7B at two quantization levels on a CPU server. Compare with Qwen2.5-7B AWQ and a versioned commercial endpoint.
- Measure streaming latency, process memory and HTTP application-body bytes. Evaluate a separately declared translate-to-English condition and dated mobile-data cost scenarios.
03The evidence
- The larger Kaggle model completed all 3,400 items.
- At the fixed 128-token budget, knowledge accuracy was 73.00% for English and 37.67% for Yoruba.
- CPU and quota-limited commercial comparisons remain partial.
04Scope & status
The scope is CPU-server versus GPU/API inference, with no phone measurements. Math outputs often hit the token limit. Loopback timings are not mobile-network timings; workspace hosting cost is unknown and never assigned zero.
Public source is available for independent inspection.