How to clean customer review data before training local language models

We tried fine-tuning a small open-source model on three years of raw support tickets to catch product sentiment trends early but the model learned to repeat typos, sarcasm and angry formatting instead of helpful patterns. Our initial evaluation runs scored terribly because half the training rows contained chatter or corrupted text exports.

We wasted two days trying learning rates before realizing the problem was garbage input data, not the model architecture.

Writing a Python script to filter out short messages strip HTML tags and normalize user names fixed the training output completely. Clean data matters more, than parameter count when you are adapting a model for a specific niche workflow.

What is your go-to preprocessing step when preparing raw user text for fine-tuning?