How do you test a time-based rule? I broke mine on purpose seven ways
When a user is flagged by my safety layer, the app goes quiet for a while. I kept the whole rule in one pure function: it takes the user’s recent flags and the current time, and returns whether they’re paused, until when, and why. No database, no clock reads inside, and the lengths live in named constants (pause length, repeat window, minimum gap between two flags that count as a repeat).
The rule itself is small, but time rules go wrong at the edges: exactly at a boundary, two flags a minute apart, a timestamp with microseconds and a +00:00 offset. So I wrote 61 tests, then did something I now do for every rule like this: I broke the code on purpose, seven different ways, and checked that a test failed each time. I removed the minimum gap, counted flags I shouldn’t count, measured a repeat from the wrong flag, made the pause read fail closed, and so on. All seven were caught. A test that has never failed proves nothing.
Two things the design review found before users did:
I first started the “soft” period (gentler messaging after a pause) at the flag time. That meant it ended the moment the longest pause ended, which is the worst time to switch back to normal. I moved the start to the pause end. That quietly broke something else: the database query only looked back as far as the old rule needed, so a repeat flag would have fallen outside it while the soft period was still running. The look-back had to grow to cover pause plus soft period plus repeat window. A test now fails if I shrink it.
Contact-form flags must not count as repeats. The table holds only a user ID, a level, and a time, and I didn’t want a text column, so I added a single yes/no column instead.
One asymmetry I’m still unsure about: if the pause state can’t be read, the app fails open (nobody gets blocked), but if the classifier can’t run, it fails closed. My reasoning is that blocking everyone because a read failed is worse than missing a pause, while not knowing whether a message is safe to answer is different.
How do you test time-based rules: clock injection, property tests, mutation checks, something else? And do you fail open or closed when the state store is unreachable?
Replies
Breaking the code on purpose to prove the tests can fail is a habit more people should steal. Two edges from my side that pure functions with an injected clock do not catch. First, the clock itself can be wrong. A user's phone had its time zone set to UTC minus 7 while its clock showed Spanish wall time, so its idea of UTC ran nine hours ahead, and a freshly issued sign in token looked expired on the device. So every time rule of mine that matters runs on the server's clock, and the app only displays. Second, "which day" is a time zone question. A holiday offer in my app is shown by the user's local days, while the server accepts claims from a day before the start to a day after the end in UTC, so nobody in any zone is refused while the app still shows the offer. On fail open versus closed, your asymmetry sounds right to me: fail open on bookkeeping, closed on safety.