Scoops

@chris_lee5 Since you’re casting a wide net wouldn’t 100 be too small of a sample?

@howwhowhen Totally. Working on making it bigger 😄 . We also just want to keep it affordable for makers, which is why we're starting this way. The beta customers have found less value so far in the proportion of people answering certain way - it's moreso at this point binary validation as to whether or not ANYBODY has the problem you're trying to solve. But feel free to challenge us on this 👍 . Always looking for ways to improve!

@chris_lee5 @howwhowhen Hey Hao! we now allow up to 250 respondents to be targeted at once :) Just thought you'd want to know!

@tom26454633 Totally fair - The argument we're making is that being 35 and a professional doesn't cause you to want pizza - being hungry, and in a rush does. Even in specific scenarios where you are building software only for lawyers, focusing on the attributes of the lawyers won't help you design great solutions. Focusing on the problem (managing billing), does. Simply, often times people mistake who they THINK suffer from the problems from the people who actually do. And that's where we can help. If you're curious, I write more in detail about it here on our blog! https://blog.scoops.io/your-defi... You're right though that not every product is going to be right for every person. We totally acknowledge that for a lot of vertical products, having SOME specificity of demographics will help significantly (e.g. yes you want to focus on a lawyer's problems, but since it's legal software you may want to only talk to lawyers). To that point, I'm going to have to say we're working on it, but we don't have a sufficient panel size yet 😞 However, if you are building software to manage a law firm's billing, is that a product that can also help accountants? or freelancers? If so, we can help you avoid that bias! Re: the %12, that's %12 of people from the United States answered that way, not that the panel is made up of %12 of people from the US. Apologies if that's not clear! (In fact, last I checked over 75% of the panel is from the US, then Canada, then the UK as it stands) Hope that makes sense, please keep challenging us if you disagree - we need all the feedback we can get!

@peter_falk Apologies, that diagram now I think is confusing. The population isn't comprised of 18% of people aged 61-70. It's simply that of the 61-70yr olds who answered, 43% of them answered "I would Never let strangers stay in my home". Hope that makes sense. Let me know if it doesn't!

@chris_lee5 What was the standard deviation within the demographics? 43% compare to an average of 42% doesn't seem significant. What kind of analysis are you using? Or does the survey constructor make sub-questions for any parameter (age, income, education, nationality) before hand? Specifically setting a followup question for 61-70yr etc?

@peter_falk Hi Peter! Not quite understanding. Are you asking about the significance of these surveys? The panel is randomly sourced and selected (literally, we use ads that target all demographics) - and following the Central Limit Theorum, if the sample is random if you increase in size it should follow a normal distribution. (though you could argue our sample is still too small, to which I’d say we’re working on getting a $124 for 1000 responses package!) If you were looking at significance for a specific age group, say 61-70, you could run a hypothesis test (using a categorical variables correlation instead of a Pearson correlation) with a confidence level to determine if in fact it’s significant. We don’t currently do this test for you in the UI, however we give you all the data after the survey so you can manipulate it however you want :). The UI just tells you of demographic group X, n% chose to answer A. Doesn’t mean it’s significant, it’s just the raw data, interpreting it (at the moment anyway) is up to you!

@chris_lee5 I'm asking. What is your sigma if the automated analysis thinks that a 1% deviation from the average (43% for the subgroup compared to the 42% average) stands out.

@peter_falk Hi Peter! I could be wrong but in this case the sigma (std dev) should just be included in a categorical variables correlation (since the observations are not 1 to -1 and you can't use a regular Pearson's correlation with a p-value) if you wanted to determine if the mean of the subgroup was significantly different from the average or it was just by chance. We could absolutely do that IF our automated analysis was smart 😄 . To answer your question, right now, it's not - as in our algorithm does not try to detect which subgroup statistics to show and which not to. It simply shows all (the mock above is altered so you can see different demographic variables, in the real thing you'll see all the age ranges). So what you'll see is 61-70 had 43%, 51-60 had 80%, and so on. We're not picking and choosing which data to show at the moment, we know we're not experts in statistics so we'd rather leave the interpreting of it to folks such as yourself for now. It's a really interesting topic though, would love to chat about it further or to get your thoughts on what a smart algorithm that only shows you the most standout stats SHOULD look like. Feel free to DM me!