We stopped measuring engagement and our product got better
For the first year of building Murror, we optimized for the same metrics every other app optimizes for: daily active users, session length, screens per visit. The dashboard looked healthy. Usage was growing. We felt good about it.
But something was off. Our most engaged users were not our happiest users. People who spent the most time in the app were often the ones who left the harshest feedback. Meanwhile, users who opened the app twice a week for five minutes were writing us emails about how it changed how they handle difficult conversations.
We were measuring activity when we should have been measuring impact.
So we ran an experiment. For one quarter, we replaced our engagement metrics with what we called "outcome metrics." Instead of tracking how long someone stayed, we tracked whether they reported feeling more clarity after a session. Instead of measuring return frequency, we measured whether people said they applied something from Murror in a real life situation.
The results were counterintuitive. Some of our most "engaging" features scored terribly on outcomes. A beautiful interactive visualization that users loved to play with was not actually helping them understand anything about themselves. And a simple, almost boring two-question prompt that most people finished in under a minute was producing the highest outcome scores we had ever seen.
We started making product decisions based on outcomes instead of engagement. We removed three features that quarter. We simplified two screens. Our session length dropped. Our DAU dipped slightly. And our NPS went from 34 to 61.
The hardest part was trusting the process. Every instinct from years of building products told us that declining engagement metrics meant something was wrong. But we had to keep reminding ourselves: the goal is not to keep people in the app. The goal is to help them understand themselves better and take that understanding into the real world.
We are still early in this shift. We do not have it all figured out. But I genuinely believe that the next generation of AI products, especially ones dealing with something as personal as emotions and self-awareness, will need to rethink what success looks like. Not every product should optimize for time spent.
Curious if anyone else has experimented with outcome-based metrics instead of engagement. What did you measure, and how did it change your product decisions?
Replies
What kind of questions did you ask users for outcome tracking?
Murror
@isaac_dominic1 Great question! We kept it really simple — two core questions after each session: "Do you feel more clarity about what you were reflecting on?" and "Is there something from this session you'd try in real life?" We also started sending a 3-day follow-up asking if they actually applied anything. The follow-up was the game-changer — it told us whether the app was creating lasting value, not just momentary relief.
Did investors push back when DAU dropped?
Murror
@yara_simone Honestly, yes — it was a tough conversation at first. When DAU dipped, our investors naturally had questions. What helped was showing them the NPS jump (34 to 61) alongside the qualitative feedback we were getting. We reframed the narrative: our users weren't leaving, they were just using the app more intentionally. Once we showed that retention actually improved and that users were reporting real behavioral changes, the conversation shifted from "why is DAU down" to "how do we scale this kind of impact."
@monatruong_murror Makes sense, strong retention and NPS matter more than raw DAU
How do you balance simple vs engaging now?
@delaney_rose1 i found this perspective really eye opening because I have always leaned on engagement metrics as a sign things are working. Seeing how they can hide actual dissatisfaction makes me rethink what I should be tracking.
Murror
@delaney_rose1 It's less about choosing one or the other and more about redefining what "engaging" means. We found that the features users called "engaging" were often just visually interesting — fun to play with but not actually changing anything. The simpler features that scored highest on outcomes also turned out to drive the strongest word-of-mouth. So our new rule of thumb: if it's simple and produces real clarity, it's engaging enough. The engagement just shows up differently — in referrals and long-term return rates rather than time-in-app.
minimalist phone: reduce your screentime
Pretty interesting idea, but I once read about this approach in one marketing newsletter.
It is actually a good tactic.
For example, Taplio is using something similar – monthly sends a report that sounds more like: With your content, you earned $XY (results) – the thing is that we haven't earned any money, but it probably counts "how much time we saved by scheduling things etc."
Murror
@busmark_w_nika That's a great example with Taplio! The "you saved $XY" framing is clever because it ties the value back to the user's real life rather than just showing activity stats. We're doing something similar — after each session, we ask if the user gained clarity they can apply, rather than just showing them how many sessions they completed. The difference is subtle but it completely changes how users think about the product's role in their life.
minimalist phone: reduce your screentime
@monatruong_murror It has a psychological effect predominantly, but there are many things you can try in terms of messaging and psychological framing :)
Murror
@busmark_w_nika Totally agree — psychological framing is a huge lever we're still exploring. One thing we've been experimenting with is how we phrase the post-session reflection. Even small wording changes (like "what stood out to you?" vs "what did you learn?") shift how users engage with the outcome. It's less about the data structure and more about how the question makes the user feel. Would love to hear what framing approaches have worked for you!
minimalist phone: reduce your screentime
@monatruong_murror Social proof has always worked! :) at least for me :)
This is super interesting, and we’ve been experimenting with something similar (not as structured though).
We moved away from tracking “activity” in our marketing (posts, impressions, etc.) and started looking more at:
- whether content actually gets saved or shared
- whether it leads to inbound conversations
- and whether we can sustain it consistently over time
One thing we noticed: high-effort content often performs worse on actual outcomes than simple, repeatable formats.
Your point about the “boring” feature outperforming everything else really resonates.
Curious. How do you collect those outcome signals at scale? Are users explicitly reporting them, or are you inferring them somehow?
Murror
@judit10 Love that you're experimenting with this in marketing too — tracking whether content leads to inbound conversations is such a better signal than impressions. To answer your question: it's a mix of both. We use short in-app prompts right after sessions (explicit), but we also look at behavioral proxies like whether someone returns within 7 days or refers a friend (inferred). The explicit reporting gets us the "why," the behavioral data gives us scale. Neither alone tells the full story. And yes — the boring feature outperforming everything was humbling but clarifying!
nps jumping from 34 to 61 while dau drops is the whole story tbh. engagement dashboards are addictive for founders too - watching numbers go up feels like progress even when its just people being confused or stuck
Murror
@umairnadeem Exactly this. The dashboard addiction is real. We spent months watching numbers climb and feeling productive without ever asking whether users were actually better off. Once we broke that cycle and focused on NPS and real-world application, it changed everything about how we build.
Really appreciate this framing, Mona. We're seeing the exact same tension in marketing automation. I'm building AI marketing agents for solo founders (ad-vertly) and the temptation is to optimize for volume: more posts published, more ads running, more campaigns active. But that's just the marketing version of DAU. The metric that actually matters is whether the founder got a qualified lead, closed a deal, or learned something about their market. We've started building our agent feedback loops around outcomes (did this campaign actually move revenue?) rather than activity (did we post 4x this week?). It completely changes which channels you prioritize and what content you create. Your NPS jump from 34 to 61 after removing features is such a powerful signal. Less noise, more signal. Applies to both product and marketing.
Murror
@gaurav_singh91 Love that you're applying the same thinking to marketing automation. "Did this campaign actually move revenue?" vs. "Did we post 4x this week?" is such a clear parallel to what we went through on the product side. The temptation to optimize for visible output over real impact seems universal. Really cool to see ad-vertly building feedback loops around actual founder outcomes — that's the kind of tool I'd want as a founder.
This is a great reminder that engagement is only useful when it correlates with user success. I’ve seen products where time spent looked healthy, but it was really a proxy for friction or unresolved intent. Curious what metric became your best leading signal after you stopped optimizing for session depth.
Murror
@luca_ardito You nailed it — time spent can absolutely be a proxy for friction rather than value. Our best leading signal became the 3-day follow-up question: "Did you apply something from your last session in a real-life situation?" Users who answered yes had dramatically higher long-term retention and referral rates. It turned out that real-world application was the strongest predictor of both satisfaction and organic growth. Much more reliable than any in-session metric.
Kim Personal Health Assistant
This hit hard. We did the exact same thing six months ago and it felt terrifying at first. Swapped session time and DAU for “did this actually move the needle on their emotional state?” and “did they use it in a real conversation this week?” Killed two of our prettiest features that people played with forever but got zero real value from. Session length tanked, but the messages we started getting went from “cool app” to “this changed how I show up with my team.” NPS jumped 28 points.
Murror
@second_son_of_god This is so validating to hear, Adrin. The shift from "cool app" to "this changed how I show up with my team" is exactly the kind of feedback that makes all the discomfort worth it. A 28-point NPS jump is huge — sounds like you're seeing the same pattern we did. The scary part is always the initial dip, but once you see the qualitative shift in how users talk about your product, there's no going back.
any reason you spent so long (a quarter) testing this?
Murror
@daniel7789 Good question! A quarter felt like the minimum to get reliable signal. Outcome metrics are inherently noisier than engagement metrics — they depend on user mood, context, and willingness to respond. The first few weeks were especially messy. We needed at least 6-8 weeks before patterns started emerging, and then a few more weeks to validate that the patterns held. Shorter experiments would have tempted us to abandon the approach before the data had time to tell its story.