What is your opinion on artificial intelligence companions (their harm and regulation)?
Currently reading about the move of California:
The state has become the first U.S. state to regulate AI companion chatbots. Governor Gavin Newsom signed SB 243, a new law requiring companies like OpenAI, Meta, Character AI, and Replika to implement safety protocols protecting children and vulnerable users.
Do you think this is the right move?
+ Should AI companion chatbots be regulated differently for adults and minors?
The idea behind it seems generous, but I’m not sure how effective it will actually be. Kids can easily pretend to be adults online, so I wonder how much this regulation can really prevent harm.
Also, I feel this shouldn’t apply only to minors. Even as an adult, I often use AI to get advice on different topics, and I can see how easily it could become manipulative or emotionally persuasive. Even I can be manipulated by AI.
Replies
Similar regulations are also developed in Europe: https://www.edpb.europa.eu/news/news/2025/edpb-adopts-statement-age-assurance-creates-task-force-ai-enforcement-and-gives_en
minimalist phone: creating folders
@bencostantini I totally missed them here to be honest, but just because the fact that Europe is restricting everything :D
@busmark_w_nika right ;)
minimalist phone: creating folders
@bencostantini (Sometimes, unfortunately, because I do not have access to the latest updates.)
Cal ID
Important topic!
I think some regulation is needed.
AI companions feel real - it’s very easy for both kids and adults to get emotionally attached or influenced.
Age gates aren’t perfect, but basic protections like transparency, safety checks, and easy reporting should be the norm for everyone.
Yes, adults can be vulnerable too. The law is a start, but tech needs to stay creative to really safeguard users without making things unusable for those who rely on these tools for support.
minimalist phone: creating folders
@sanskarix I can see how more social media want to restrict some pieces of content under 13. But this is more effort from the side of the government and activists.
Cal ID
@busmark_w_nika Not just the activists but the users have to raise the activism about this, I don't think any robust actions will be taken unless that happens.
minimalist phone: creating folders
@sanskarix yes, it is about us, we need to be those who are involved :)
Hmm, interesting topic. I think they're probably doing both right and wrong things to regulate on AI service providers rather than targeting on customers only.
Good side is by applying safety protocols, it does prevent some of the vulnerable from accessing something that could potentially be harmful to them if miss-used/addicted :)
Not so good side would be it might brings a lot extra inconvenience for ppl that would need full potential of AI services for daily usage.
Being said that, people that really needs AI doing crazy things will find out crazy solutions to break out the rules/restrictions eventually one way or another.
minimalist phone: creating folders
@rickyguo Everything has its pros and cons, but I think that the majority of people are just consumers who will take AI as it is and blindly follow what it preaches.
Feels like we’re entering an “emotional responsibility” phase. Regulation around protecting kids makes sense, but I’m curious how this will affect indie builders. Will smaller AI companion projects need to follow the same protocols? Watching this space closely.
minimalist phone: creating folders
@iamrajanrk sooner or later it can become mainstream to follow such rules.
@busmark_w_nika Just around the corner!
minimalist phone: creating folders
@iamrajanrk THIS!
Cal ID
I think basic rules and transparency are really important for both kids and adults.
Safeguards shouldn’t stop at age gates or just limit innovation. Clear boundaries and healthy defaults make sense for everyone, not just the youngest users.
minimalist phone: creating folders
@sanskarix yep, age shouldn't be a barrier, and even older people (like Gen Z) can be fooled. AI has just become quite realistic.
Cal ID
I always wanted to start a Gen-AI lab that has a bunch of AI-creators making a ton of content for me in different niches. Realism always held me back, but I think now is the time!
minimalist phone: creating folders
@sanskarix Do you mean creating same characters and presenting them like AI influencers over social media while earning money from them?
Cal ID
@busmark_w_nika You got my point!
IXORD
It would be great to have something like parental control, where the AI has a built-in prompt indicating that the person writing is a minor, and no other requests can override that. It would also be nice if a parent could enable such control for their family.
minimalist phone: creating folders
@ixord Do you know about such a service somewhere?
IXORD
@busmark_w_nika For example, take Steam. It’s a video game platform and has a Family Library feature. It works on the principle that a family member creates their own account on the platform, and I can send them an invite to join the family as a child. After that, I have the ability to restrict these accounts from accessing unwanted content
minimalist phone: creating folders
@ixord Okay, this could be pretty cool if there is no way to cheat the system :)
IXORD
@busmark_w_nika I think it would be a great update for AI if parents could add banned words for other accounts :)
I think this is a good move overall. AI companions can be fun and helpful. However, they can also affect emotions if there are no guardrails.
Regulating them for minors makes sense. But kids pretending to be adults is a real loophole. We should focus more on transparency and healthy boundaries instead of strict age filters.
Even adults can get attached to an AI that feels “human.” So having some safety measures is a good idea. We just need to make sure it doesn’t stifle innovation.
minimalist phone: creating folders
@aaroncalvo I am pretty curious how they want to check whether on the other side is an adult or a kid, to be honest.
I believe we're giving too much of our personal data for the sole purpose of "innovating fast". The tech sector can innovate while also making sure they don't do it at the expense of people's wellbeing (ahem, the ones using the tech). I'm not of fan of the amount of regulation in Europe, which indeed slowed us down from an economic perspective, but I feel the US is too much of an unregulated paradise 😆. I'm glad to hear there's a US state taking a stance on this, hope it will become common practice!
minimalist phone: creating folders
@ruxandra_mazilu As you said. The internet even knows what I already forgot about myself. It is scary :D
Triforce Todos
I think it’s a step in the right direction. Kids are especially vulnerable to AI manipulation, and any safeguards are better than none.
minimalist phone: creating folders
@abod_rehman Many adults would need this too IMO. 😅
Triforce Todos
@busmark_w_nika 😂😂