ReplayAutonomous app testing for teams who ship at AI-speed
Promoted
Launching today
Open source feature flags as markdown files in your repo: the flag, the reason, the decision in one file, reviewed in a PR. One command, no account, so your coding agent can finish the install. A/B tests wait in the same files until you have traffic.
The simple approach sounds great early on but does the file structure stay manageable as the repo grows?
dif.sh
@amybradley Good question! We've found that it stays manageable as the overall footprint and file sizes are very small. There are also other mechanisms like the context file and surfaces directory that keep learnings so the old flag/experiments can be periodically cleaned out without any issues.
I like the RR based approach here because every flag change gets context. Could dif.sh also show a simple history of why a flag changed over time?
dif.sh
@manonbriffaut Yes, that can be stored in the individual flag .md file. We also added surfaces, so these learnings can be captured for every page/surface as well which I've found to be super helpful.
dif.sh
@hamza_afzal_butt right now it's the default git conflict behavior. We have found with the .md files that non-technical folks are able to handle those conflicts easily though.
Hmm...interesting. The key part is that coding agent has access to previous flag and A/B tests and their data. Right?
dif.sh
@bogdan_serbanescu Yep, it's a pretty big unlock. The skills make it easy for agents to build flags/tests, and then the context across previous tests and surfaces make for better recommendations and code.
I really like the idea of keeping the flag and the reasoning behind it in the same file. That kind of context usually disappears after a few weeks.
dif.sh
@thomas_bennett Agreed. A lot of flag/experiment tools handle the analysis, but usually they don't keep the actual story of what works, what has been tried, etc.
You asked for the critical version. Flags in git means flipping one takes a commit, a review and a deploy, and the reason teams pay for a flag service is the kill switch at 2am without any of that. For agent facing flags it's a fine trade, for a gradual rollout or an incident it isn't. Markdown also won't fix dead flags, since nothing gets reviewed when nothing changes and the flag nobody has touched in four months is still branching in prod.
dif.sh
@asadmalik901 Thanks for the input. Yeah there are definitely trade offs. We've found it's worth the improved every day performance (no api calls / extra hops) in not having the kill switch but it depends on the organization.
For dead flags, I actually think dif helps with this tremendously. The cli helps to find everything running and used with an agent it can find these pretty quickly.
Dial
Congrats on the launch. The "flags reviewed in a PR" part is the real hook for me - most flag tools bolt on a dashboard that nobody checks, and this just stays in the same review flow. Curious about one thing though: if two agents are working in parallel branches and both touch the same markdown flag file, does dif help resolve that conflict at all or is it just a normal git merge conflict at that point?
dif.sh
@galdayan Right now it's just a normal git merge. That's a good idea to add though, we'll note it!