A Solo founder here to learn from makers who've shipped real products
by•
Hey everyone! I'm Muhammad Saud from Pakistan. I run a small software studio called LogicStack, where I build free dev tools and write about AI engineering.
I joined Product Hunt to learn from makers who've shipped real products and to get honest feedback on my work. Right now I'm deep into learning AI agent engineering myself currently wrestling with evals and context engineering, so if that's your world too, I'd love to connect.
Looking forward to being part of this community!
21 views
Replies
Welcome to Product Hunt! Evals and context engineering, same here. What helped me most with context was making every correction I give the agent end up as a file, so the next session already knows it. I've got 127 of those incident notes now, plus a memory folder per project with one fact per file. For the bigger calls I also run the answer past a second model. What are you using for evals?
Welcome Muhammad! One eval lesson from my product, where the output is a song: the plan a model returns is not the thing that was made. The music model hands back the lyrics it was asked to sing, but that is the plan, not what was actually sung. So the check is a second model that listens to the finished track and compares. I tried a smaller, cheaper model as that checker first, and it missed two chorus lines that had been merged into one, so the larger model does the listening now. My takeaway for evals in general: grade the artifact, not the model's description of it, and test the grader on a known bad case before trusting it.