We run an automated pipeline that drafts articles for our product. During our first test attempts we let the model do almost everything, including the mechanical bits: fixing dashes and quotes, tracking which stage each article was in, creating SEO json, naming image files etc. It worked, but the quality was uneven, sometimes the model would miss some things, and, of course, it was rather expensive in terms of tokens. Then we changed our approach and started treating AI like a creative employee and providing it with "software" (python scripts) to help with any mechanical tasks that could be automated by code. The model only does the part that actually needs intelligence and creativity now. And honestly, the result surprised me - quality went up, because we now let it focus on what it's actually good at. As a pleasant bonus - the costs of running the pipeline dropped too. One of the first things we scripted was stripping the em-dashes the model loves to add but my co-founder has an allergy for
How do you improve the quality of your AI results, especially automated pipelines?
Solo builder here, using Cursor + Claude Code as basically my whole engineering team. Six months in I still don't have a good instinct for this: some days I let it just run with a whole feature and barely glance at the diff, other days I'm rereading every line like it's a junior dev's first PR.
Right now my rough rule is billing/auth code gets the paranoid review, UI and analytics panels get the "looks fine, ship it" treatment. But that's just a gut feeling, not a real system.
Anyone here have an actual framework for this, or is everyone just winging it based on vibes and how burned they've been before?
