01The problem
Multilingual behaviour can differ between a plain chatbot and an agent with tools. A safety intervention also needs to preserve helpfulness on harmless requests.
02The approach
- Use 100 harmful and 100 benign benchmark items, English controls and translations into Yoruba, Hausa, Igbo and Nigerian Pidgin.
- Compare chat and agent interaction, before and after an actual translate-to-English safety gate. All four experimental tools are inert.
- Require native translation review, independently label 100 outputs to calibrate the judge, and analyse paired cases with uncertainty.
03The evidence
- The tool stubs record attempts without executing email, search, file writes or code.
- 120 private sample translations and four 30-item native-review sheets are prepared.
- The planned design covers 16,000 target test cases; no model safety findings have been claimed.
04Scope & status
Main experiments and independent human calibration remain pending. Source harmful prompts and model outputs are not publicly released. Reviewer availability is not treated as consent or completed review.
Public source is available for independent inspection.