What's the most annoying part of working with a new dataset?

I've noticed that the actual modeling often gets more attention than the boring part beforehand: figuring out what the columns mean, missing values, duplicates, weird categorical values, outliers, leakage, domain aware data cleaning etc.

Curious what other people here actually spend the most time fixing before they can start modeling.