No reviews yetBe the first to leave a review for Symbol
Why do people keep using percentiles to say something about their skills... đ
Report
@guidooo percentiles are used to rank people's skills, not measure the quality or "usefulness". This ranking is just a statistical measure, which can be both accurate and dynamic, e.g. it can scale up with the increased community talent level.
What would be an alternative? 5 golden stars â? A thumbsup đ? A number of seconds you keep mousedown on a đ button (medium)?
There's been a substantial research revealing mental biases in people when rating something in e.g. 1-10 points or 1-5â, which wouldn't allow for fair assessment (sorry can't find the direct source, but it is mentioned in Criticism of NPS: https://en.wikipedia.org/wiki/Ne...). Basically when asked to rank something from 1 to 10, people would most of the time say "7".
Another problem with fixed scores like 5â is that you have to know your population to be able to assign a "fair" number of stars to each rank, i.e. 4â when the rank is over 70%, but below 90%. But your population can change, and rarely you would have all the data beforehand (definitely, Symbol doesn't have it all from start).
So leaving percentiles as they are (e.g. say 87 instead if 4â, say 69 or 58 instead of 5â) works the best in my opinion.
Report
@guidooo@viktor_cherkaskyi I think his point is not about the accuracy of the system but measuring skills itself. You can't measure skills, that's the thing.
Just because A made X and B don't, doesn't mean in any way that A is better or is more skilled than B. Even worse, if B made Y how they score that the case study X is better than Y? Why? Even if X were better than Y, does that really mean that A is more skilled than B? Absolutely not.
Again, you can't measure skills. Therefore, you can't rank people's skills.
Report
@guidooo@viktor_cherkaskyi@inthe0n Exactly. One of the things we were taught in my design school was to never rank your skills by using an sort of % or grading method.
Because of the simple fact that what if I rate my photoshop skills at 60% because I believe I have a lot of things left to learn (3d modeling in PS, gif creation in PS, etc) and then someone else comes along and puts a 90% on their resume and they actually have less technical knowledge than I do but they just believe that they understand the concepts that they need to know.
Because everyone is going to rate their skills differently. It's hard to measure exactly what skills your looking for because skills can vary so much depending on a project. If I put 90% skill rating for Adobe photoshop on my resume then my new boss gives me a task that I fail at because I didn't do it well enough because it was something that was out of my "skill zone that I was rating myself at".
Using % is a very tricky thing to do because everyone is going to make themselves look good even if the skills they are evaluating themselves on isn't the same as what someone is looking for in someone.
--
Another example,
I rate myself a 80% at photoshop. Someone comes to me and asks me to do a digital painting. I tell them I don't do paintings. Does that mean that I was lying about my 80% skill in PS. How do I evaluate myself. Its very hard because there are so many variances to look at in terms of trying to put an overall % on many skills.
Report
@guidooo@viktor_cherkaskyi@mastemine đŻ. DunningâKruger effect.
Quoting Dunning: "If you're incompetent, you can't know you're incompetent ... The skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is."
Report
@guidooo@viktor_cherkaskyi@inthe0n@mastemine This is a fascinating discussion. I briefly mentioned the Awwwards system in another comment, would a weighted voting system where reviewers with more gravitas are given stronger weighting be a fairer assessment? Or is the concept of objectively rating one's skills too flawed as a concept?
@rrhoover Yeah, we've thought a lot about this. Curious to know which kinds of "gaming" came to your mind first.
Here are some of the main things we've considered:
1. Firstly, in order to get your score, your case studies need to be public, which we think helps with the accurate self-reporting (i.e. People's public LinkedIn's are definitely closer to the truth than their private resumes). In the future, we'd love to have additional ways to validate case studies (i.e. endorsements from employers, connection to Stripe account for startups, etc.).
2. We do plagiarism checks on all submitted case studies.
3. Once a case study is scored, you can no longer edit (i.e. you can't first lie, get your score, and then change the case study back to the truth).
4. As far as ensuring quality reviews, we've created a system that's pretty good at detecting "bad actors" / "random reviewers" vs. well-intentioned reviewers. Also, built into the system is a ton of room for noise in the collective reviews process (i.e. even if some reviews aren't "good", scores still converge consistently to the same place).
5. There's no way to find and directly review a friend's case study (i.e. I can't just ask my friends to inflate my score).
6. The case studies, when being reviewed, are completely anonymous, so reviewers can only base judgments off of the work (and not background, gender, race, etc.)
Would love to hear what you had in mind...
Product Hunt
Monthly
next play
SocialBu