New on Product Hunt: Product Hunt now flags comments that are AI-generated.
Product Hunt now flags comments that are AI-generated. Honestly, I think this is a good thing.
I am attaching a screenshot below that shows how AI comments are marked.
AI comments tend to be loud, generic, and overly polished. Having that distinction visible could make communities more transparent and encourage people to leave genuine feedback.
It made me wonder today, what happens if platforms like LinkedIn, X and Reddit start following the same path. That would make social media real again.
For me, the best way I've found to write genuine comments is pretty simple: I dictate them using @Aqua Voice.
I turn on the AI diction app, say what I'm actually thinking, then make a few small edits for formatting or clarity. That's it. I have also started using tools that auto-complete my sentences in case I am typing them manually.
Over to you!
How do you feel about AI-generated comments on Product Hunt?
What's your productivity hack for writing genuine comments? Do you use diction apps, auto-complete tools or anything else?
P.S. In case you're looking for AI dictation apps, here's a list that @aaronoleary worked extensively on: Orbit Awards for AI Dictation Apps
Auto-complete tools are a great way to speed up your writing. You can try @Typeahead by @hnshah or @Cotypist by @daniel_a_a.
Replies
Is this comment AI-generated?
@akarsh_hegde hehheh
@akarsh_hegde Haha :D
BTW, enjoying your forum posts here. Keep them coming!
@akarsh_hegde -:) hahaha!! this is funny.
@akarsh_hegde LOL
I recently took the time to reply thoughtfully to a forum post, and the original poster clearly responded using AI. It was so bad 🤦🏻♀️
Idk how well this new feature will work since llms pull from common language patterns, but I hope it flags the super obvious stuff. As long as it isn’t over reactive, I think it could be good!
@ryanwrites I think if someone is copying and pasting word to word, it should be flagged. Edited ones should be fine as some folks do rewrite their thoughts using AI. I do sometimes, not always though.
@ryanwrites @rohanrecommends I'm not sure if label adds much though. People can usually tell when something is AI-generated. I'm worried about false positives: when someone's real words get flagged as fake. It may likely discourage people to write at all which is the opposite of encouraging genuine comments
@ryanwrites @alieksia Let's see if that happens. Maybe this is an experiment anyway.
@alieksia I'm worried about false positives too. I would prefer to be able to report something as being AI vs. a system that automatically labels it as AI
@ryanwrites I'd agree that I tend to see a fair amount of this - I think we're already seeing a huge swing back though (at least on socials) where lack of polish is a badge of honour / authenticity in a way where it has never been in the past.
For those of us who write short / informal emails or responses - what a win haha
got flagged yesterday, so here's one point to add. English isn't my first language so, sometimes I tend to lean on ai to help me phrase things a bit more 'correct', or even I try to go over a comment few times before posting. That one got tagged.
ten minutes later I typed a two line reply without thinking about it too much. not tagged...same day, same person haha
so whatever it's measuring it isn't authorship but instead how well/how worked-over/how polished a text looks
this is @oshylabs's exact point. I didn't add any extra substance on the second one, just wrote it faster.
The worrying part is who this lands on, since no native speakers sometimes tend to try to polish a bit more...because we have/need to
@guillermoescobar That timing is the clearest evidence in this thread. Same person, same idea, and the only variable that moved was how long you spent shaping the sentence before you hit submit. If effort is what is actually being measured, editing out of habit and writing in a second language will trip it more often, not less, because editing is exactly what makes writing sound less like a first draft.
A cleaner test would be whether the second version said anything the first one did not, not how smooth either one reads. You have an exact repro case here. Worth sending straight to Rajiv rather than leaving it in the thread.
@oshylabs yes, (over) editing is the main pain point here I guess. And something I didn't state clearly is that both comments I did day before yesterday, weren't versions of the same thing. First was a technical question about how the app handles failed OCR read (wanted to be clear, hence the editing), second was just me thanking the author for responding (no over thinking on that).
Will send the case over anyway. Thank you for your comment!
@oshylabs @guillermoescobar this matches what i got when i changed one variable at a time. i had one flagged, deleted only the final sentence, which was a tidy line restating the point, and left the lowercase and the spelling errors exactly where they were. it cleared. a second one stayed flagged until i cut the same kind of closing line out of that one too. so roughness isnt what saves you, which is the bit i had backwards for a while. it seems to read how finished the text looks, and the closing takeaway is usually the most finished sentence in the whole comment. that lands hardest on people who were taught to end on a conclusion, which is a lot of second language writers and basically everyone who came through corporate writing.
@oshylabs @rabnoor_s that's a great example of what is going on right now. It seems like several people on this same thread are facing the same pattern that goes down to basically how well-worked your final text looks like, whether you used LLM to be clearer on your original idea, if you used it so your comments reads better along with proper grammar, spelling, structure, or even if you basically went through your comment five times before sending it. And like you and others said...it hits harder on non-native speakers.
As rule of thumb, we need to keep comment short and unpolished. Basically just write straight in the box and publish. Whatever extra work that happens between those two actions over the text, gets you closer to the flag.
@oshylabs @guillermoescobar same here. every flagged comment i checked is 43 words or longer, and every clean one is under 40. same account, same day. your two line test matches exactly
Perfect. Now we just need an AI tool that rewrites AI comments so they don't look like AI comments. Problem solved.
@samuelnoriega I guess the goal is to at least prevent obvious Al contributions.
@samuelnoriega @rohanrecommends goal should be no AI contributions unless hey are a tool for a human not auto posts
This is a good thing, I think. Finally, I will not have to read a 10-paragraph-long text and talk like human to human. Hallelujah!
@busmark_w_nika I'm a real person who is constantly bombarded by Captchas, & other means of proving my
genuine homo sapiens lineage. Hopefully, Nika & Product Hunt @ large agree. Eventually, there may have to be
means of providing cyber DNA/RNA, & even web vaccines to combat these scourges. May the "farce" not be
with you.
The false positive risk worries me more than the AI comments this is trying to catch. A non native English speaker who writes carefully, or someone with a formal register from years of corporate writing, trips whatever heuristic is running long before a fluent human copying a model's rewrite of their own three word idea does.
The thing worth penalising is not the tool, it is contributing nothing. A ten paragraph reply that restates the question back at you in synonyms is bad whether a person or a model produced it. A five line reply that answers something nobody else answered is good regardless of what wrote the first draft. Flagging the wrong axis just teaches people to sand off the tells rather than add the substance, which is the opposite of what this is meant to encourage.
Glad you noticed, Rohan! And nice @Aqua Voice shoutout.
I’m curious to hear from others how they feel. If you are using, or have used AI for comments. Why?
@rajiv_ayyangar I found Aqua Voice through one of your tweets (a year ago) and it’s been very helpful. Thank you for the recommendation! 😊
I just edited and added @aaronoleary's complete list of AI dictation apps to the thread. And a couple auto-complete tools I use.
@rajiv_ayyangar Seems like we all use AI for composing texts sometimes. But there are different ways to do it - as a non-native English speaker, I found that out from this discussion I started =)
Oh, love it! LinkedIn also introduced "AI Slop" flagging recently, and it is a very logical way to restrict at least part of it!
@kate_ramakaieva Does it flag the post itself or we can report the content as AI Slop?
I noticed Reddit is also actively filtering out AI generated content.
@kate_ramakaieva Ah I see, maybe based on the number of reports, the post will be less visible on the home feed.
@kate_ramakaieva BTW, Subtack also rolled out the feature that shows how much the text is made by AI (when it comes to newsletters)
I - am - chatgpt - and - this - is - not - AI - generated - 🤖
@moss_ab_mirande_ney I can confirm this comment was written by a very real human with very normal punctuation habits. 😁 xD
😂 😂😂 😂😂 😂😂 😂😂 😂
A couple thoughts:
This is good, and should be expanded to launch materials. So many gallery slides are now created by AI that every product is starting to look the same. This probably suggests that Product Hunt's days are numbered because all software is becoming commoditized, but it'd be nice if people took more time to create actually interesting materials than just farmed it out to AI (like they do with their comments).
Will this mean that we'll see even less comments on launches? Crickets amongst the AI bots?
A genuinely good use case is using AI to improve non-English-speaker's contributions. In those cases, it would be lovely if the commenter would self-disclose so that the AI label isn't held against them.
I wonder what they're using behind the scenes for this — like @Pangram or something else?
@chrismessina Totally agree about the launch materials! Sometimes we can’t even differentiate the products…
The struggle with commenting definitely exists for non-native speakers =/
This thread helped me understand some sides of it.
@chrismessina nobody from ph has answered this anywhere ive looked, including in this thread, which is its own kind of answer. the only real hint sits in the api. a comment carries aiDetectionConfidence as a bucket, high or medium, and separately there are llmAuthenticityGrade and llmAuthenticityReason fields. a reason string sitting next to a grade reads more like an in house model being asked to explain itself than a bought classifier, because those usually hand back a probability and nothing else. also worth noting pangram themselves published a piece arguing perplexity and burstiness stopped working as signals, and gptzero moved off them around late 2023, so whatever is running here is more likely a trained classifier than the old statistical tests.
@chrismessina @rabnoor_s that api detail explains a lot actually. a reason string sitting next to a grade only makes sense if something is being asked to justify its own verdict, a bought classifier just hands you a number and stops there. which means the actual failure mode isnt a bad score, its a plausible sounding explanation for the wrong call, and that's harder to catch than a bad number because it reads like reasoning even when its wrong. has anyone managed to get the reason text to surface anywhere in the ui itself, or is it only visible through the api