Qwen2.5 7B vs Qwen3 4B & 8B Writing Coach Benchmark with Ollama on Windows

This is a follow-up to my earlier Qwen3 4B vs 8B vs 14B writing-coach benchmark.

That first experiment suggested that a larger local model does not automatically provide a better practical trade-off for writing correction: Qwen3 8B matched Qwen3 14B on complete-case correction while requiring roughly half the measured cold-start time.

This second study asks a different question:

Can a smaller model from a newer Qwen generation reproduce the correction outcome of a larger previous-generation model?

I compared Qwen2.5 7B, Qwen3 4B and Qwen3 8B under the same local writing-coach evaluation framework:

• 20 paired writing cases

• 60 responses generated locally

• English text correction

• Explanations requested in French

• Ollama running on Windows

• Same frozen reference and evaluation rules for all three models

The complete-case results were:

• Qwen3 4B: 18/20

• Qwen2.5 7B: 18/20

• Qwen3 8B: 19/20

The most interesting result was the paired comparison between Qwen3 4B and Qwen2.5 7B.

They did not simply finish with the same 18/20 score. They succeeded on the same 18 cases and failed on the same 2 cases, with no discordant complete-case outcome across the 20 paired cases.

Local cold-start time changed the practical picture:

• Qwen3 4B: 23.99 s

• Qwen2.5 7B: 54.37 s

• Qwen3 8B: 60.24 s

In this tested setup, Qwen3 4B therefore delivered the same paired primary correction outcome as Qwen2.5 7B while averaging about 2.27× faster cold-start execution.

Qwen3 8B still provided the strongest correction coverage in this run, with 19/20 complete cases and 100% error-localization F1.

This is not intended as a universal model ranking. The result is specific to this controlled writing-coach workload and local configuration.

The main takeaway is that local model selection is better treated as a trade-off between correction reliability, explanation behavior, responsiveness and available hardware - not parameter count alone.

Full methodology, detailed results and limitations:

https://getlinguapilot.com/research/qwen2-5-vs-qwen3-writing-coach-benchmark/

I am particularly interested in the next practical question: should the next benchmark focus on RAM/VRAM usage, warm-start latency, a larger writing test set, or another local model family?