When a data source changes, how do you decide which AI features to re-check?
Here is a scene I think many teams know. Someone renames a column, or a field switches from dollars to cents. The pipeline keeps running and nothing throws an error. A week later, someone notices that one AI feature, a support bot for example, is giving odd answers.
As far as I can tell, the hard part is that nobody knows which AI features actually depended on that field. The same table might feed three features, and each one cares about different columns. So the choice is to re-check everything or to guess.
What I’m trying so far is a short list for each feature of the fields it relies on, so a change points to the features to re-test. In my experience that only works while someone keeps the list updated.
Questions: When a data source changes, how do you find out which AI features are affected?
p/jot-2-0
p/tabstack
p/gatrends
p/runevr
p/google-cloud-platform
p/producthunt
p/okapi-2
p/rithub
p/flamigo
p/ski
p/dex-by-exmergo
p/replay-11
p/appxiom
p/verse-engine-by-ctruh
p/codedesign
p/many-displays
p/reso
p/dino-kids
Replies
I try to keep the model away from the fields in the first place. One rule in my setup is that a model only writes the body text. Anything like numbers, dates, paths, settings or JSON gets built by plain code, and validation never goes to the model either. So when dollars turn into cents, it's the code that breaks, and code at least throws an error you can see.