I built a Qwen3 writing coach benchmark: 4B vs 8B vs 14B with Ollama

Over the past few weeks, I built a reproducible benchmark to explore a practical question:

Can a local language model correct writing effectively while also explaining the user’s mistakes?

This work is part of LinguaPilot AI, a Windows writing coach designed to improve the user’s own draft and explain the corrections, rather than automatically writing everything on their behalf.

For this first study, I compared Qwen3 4B, 8B and 14B under the same local conditions:

• 20 paired writing cases

• 60 responses generated locally

• English text correction

• Explanations requested in French

• Ollama running on Windows

• The same evaluation framework for all three models

The complete-case correction results were:

• Qwen3 4B: 18/20

• Qwen3 8B: 19/20

• Qwen3 14B: 19/20

The result that interested me most was the trade-off between model size and local responsiveness.

In the tested setup, Qwen3 8B matched the 14B model on complete corrections while running approximately twice as fast. Qwen3 4B remained faster and still completed 18 of the 20 cases.

This is not intended as a universal model ranking. The benchmark suggests that choosing a local writing model should not depend only on parameter count. Correction quality, explanation quality, response time and hardware requirements should be evaluated separately.

I documented the complete methodology, detailed results, charts and limitations here:

https://getlinguapilot.com/research/qwen3-writing-coach-benchmark/

For a local writing coach, which trade-off matters most to you: correction quality, explanation quality, speed, privacy or hardware requirements?

I would also be interested to know which local model you think should be evaluated next.