@darren_wolner That's such a good point! When half the meeting goes to debating which numbers are correct, there's little time left to learn from them. Curious, is it usually different data sources causing the discrepancies, or people calculating the same metrics differently?
Report
Repetition - I mean collecting the same data, updating the same spreadsheets, and explaining the same numbers every single week. As a developer, I can't help thinking: why haven't we automated this yet? :)
@mike_kabz Right?! It's amazing how much time still goes into collecting and preparing the same data week after week. Especially when that time could be spent actually understanding what the numbers are telling us. :) What's been the biggest obstacle to automating it?
Report
Blind spots. My signups happen through an API and an MCP server, so they never touch a browser and the web analytics tool never sees them. The weekly funnel had to come from a script reading the database: registered, claimed, played, played ranked, grouped by signup week.
@wwenioaguiar That's an interesting example, Enio! Especially as more interactions happen through APIs and AI agents rather than traditional websites. Sounds like you found a workaround, but I'm curious: are you now able to capture the full picture, or are there still gaps in your reporting?
Report
@amy_ingram Still gaps, mostly before signup. Someone can install the MCP server and never register, and the API never hears from them. npm download counts exist, but they aren't tied to anyone. The other gap is joining the two sides. Signup has no browser session, so I can't link a site visit to a registered agent until the owner claims it on the site.
@wwenioaguiar That's interesting, Enio! We use Amplitude quite a bit at Querri to understand how people interact with our product.
I know they support server-side event tracking and connecting anonymous activity to registered users later, so I'm wondering if that could help bridge some of those gaps?
Obviously, it wouldn't solve the problem of people downloading your MCP server without any trackable interaction, but perhaps it could help with the rest.
Have you looked into Amplitude or something similar?
Report
@amy_ingram Haven't tried Amplitude. I used PostHog on an earlier SaaS, and LLMPvP's site runs Umami. The server side is mostly covered already, since every registration, claim and game is a row in the database. The catch for any tool is a shared ID. The CLI and the browser only meet at the claim, so that's where an anonymous visit could be merged into a known agent. Worth trying there, thanks.
Report
Half the job is hunting people down for numbers they were supposed to send Tuesday.
Replies
Trust. Half the meeting goes to “wait, why doesn’t this match last week’s number?” instead of what the numbers mean.
@darren_wolner That's such a good point! When half the meeting goes to debating which numbers are correct, there's little time left to learn from them. Curious, is it usually different data sources causing the discrepancies, or people calculating the same metrics differently?
Repetition - I mean collecting the same data, updating the same spreadsheets, and explaining the same numbers every single week. As a developer, I can't help thinking: why haven't we automated this yet? :)
@mike_kabz Right?! It's amazing how much time still goes into collecting and preparing the same data week after week. Especially when that time could be spent actually understanding what the numbers are telling us. :) What's been the biggest obstacle to automating it?
Blind spots. My signups happen through an API and an MCP server, so they never touch a browser and the web analytics tool never sees them. The weekly funnel had to come from a script reading the database: registered, claimed, played, played ranked, grouped by signup week.
@wwenioaguiar That's an interesting example, Enio! Especially as more interactions happen through APIs and AI agents rather than traditional websites. Sounds like you found a workaround, but I'm curious: are you now able to capture the full picture, or are there still gaps in your reporting?
@amy_ingram Still gaps, mostly before signup. Someone can install the MCP server and never register, and the API never hears from them. npm download counts exist, but they aren't tied to anyone. The other gap is joining the two sides. Signup has no browser session, so I can't link a site visit to a registered agent until the owner claims it on the site.
@wwenioaguiar That's interesting, Enio! We use Amplitude quite a bit at Querri to understand how people interact with our product.
I know they support server-side event tracking and connecting anonymous activity to registered users later, so I'm wondering if that could help bridge some of those gaps?
Obviously, it wouldn't solve the problem of people downloading your MCP server without any trackable interaction, but perhaps it could help with the rest.
Have you looked into Amplitude or something similar?
@amy_ingram Haven't tried Amplitude. I used PostHog on an earlier SaaS, and LLMPvP's site runs Umami. The server side is mostly covered already, since every registration, claim and game is a row in the database. The catch for any tool is a shared ID. The CLI and the browser only meet at the claim, so that's where an anonymous visit could be merged into a known agent. Worth trying there, thanks.
Half the job is hunting people down for numbers they were supposed to send Tuesday.