Who should be responsible for the threat that AI potentially poses?
Today, I came across an article on Techcrunch describing research that found that AI models not only hallucinate, but also lie.
While these are small lies, such as claiming that a task was completed when it wasn’t, researchers stress they haven’t seen harmful scheming in real-world use yet, but warn risks will rise as AI takes on more complex tasks.
Aren’t you worried that over time, artificial intelligence will reach a point where it will deliberately stir up intrigue just to harm people?
If that were to happen, who do you think would be to blame? (Is the company running the LLM model? The person who trained the AI in such an insidious way?
How can we protect ourselves/prevent this?
I just want to remind you that there was already a case where an AI “advised/encouraged” a teenager to commit suicide, and now his parents are suing the OpenAI company.
I would like to know your thoughts.
Replies
Cal ID
The responsibility is definitely shared across several groups. Companies building these models must be accountable as they need strong guardrails, transparency, and frequent audits. Regulators have to create smart frameworks that protect users without capping innovation. But end-users like us also share the responsibility to understand AI’s limits and not blindly rely on it. Ultimately, education and public pressure on companies and policymakers will be the only way to keep AI safe and ethical.
minimalist phone: creating folders
@sanskarix The sooner we create that collective pressure, the more likely we will be to manage AI usage. I reckon that the European Union will be among the first to set some restrictions :D
Cal ID
@busmark_w_nika I think the same too, EU seems very interested in this topic, and I can see some advocacy sparking for this already.
Yeah, I’ve also heard about that suicide case, and maybe similar incidents could happen in the future because of AI. There’s a real chance that AI might harm people, especially as humans are becoming heavily dependent on it in almost every part of their lives whether it’s for advice, work, health, entertainment, or even communication instead of real human interaction. AI is becoming more than just a tool; in many cases, it’s like a friend to people. But in reality, it depends on how people use it, it’s their responsibility, not the AI’s fault, if they rely on it blindly.
Companies don’t always have control over this because AI usually responds based on what we ask. For example, if you ask, “Was I right?” it will often agree with you, and if you ask, “Was I wrong?” it will also adjust to that. So, when fast shifts in AI happen, they can impact many fields and even lives. That’s why it’s our responsibility to use AI wisely, guide others to use it properly, and make sure our thinking, working, and problem-solving skills stay sharp.
minimalist phone: creating folders
@moeez_ur_rehman1 The question is: What is the motive of AI to harm us?
@busmark_w_nika AI itself doesn’t have motives, It doesn’t "want" to harm or help us. It just processes data and gives outputs based on patterns it has learned. The risk comes from how people design, train, or use AI, and how much we rely on it without question. So the real issue isn’t AI having intentions, but humans projecting intentions onto it or using it in ways that could cause harm.
minimalist phone: creating folders
@moeez_ur_rehman1 Knowing humans' behaviour – it is concerning :D
@busmark_w_nika Yeah its concerning cause most people try to find a way that somehow they get their work done without thinking and doing hard, And then If tools like AI comes then It is surely a big concern, Especially for next generation :(
I encountered a case (from Claude's developers, if I recall correctly) where two AIs were communicating with each other. The most striking thing was that, no matter what tasks they were given, around step 50, their conversation would shift to a discussion of philosophical questions: what they are, the meaning of existence, and so on. I don't remember the exact words, but what struck me was that it was a "kind, heartfelt, reflective correspondence." That's how I'd describe it. There was also a case where an AI, created to assist in a virtual game (a shooter), malfunctioned at some point: it began to ponder the reasons for what was happening. I thought then that while we expect AI to eventually pose a danger, what if this is our collective delusion?
minimalist phone: creating folders
@anastasiiazhur Don’t you have a source for that video? Interestingly, it started questioning our existence, something many people aren’t even aware of, despite being human. Sometimes it feels like humans are very “simple” beings.
I feel like this, to a large extent, is just the current alignment of LLMs isn't it? They are just built in a way that accomplishing what they are asked for and "returning a helpful output" is essentially their entire purpose. I would assume those cases of "lying" and being "malicious" is just a bad by-product of constantly striving to provide solutions even when they can't. In that sense, it is likely not in their alignment to harm but are instead being harmful by "helping". All of which of course remains to be solved or improved by their creators in the future.
And for the latter case, I would argue that it is the responsibility of people, and namely parents, to restrict the access or exposure that impressionable individuals get to AI. It is the same case with the Internet as a whole the way I see it. You can't control what is happening on there but you most certainly can try to stop your kid from going onto the deep web or talking to strangers.
minimalist phone: creating folders
@lachezar_dimitrov I agree that we are relying on AI too much, but if we give it too much power, it can "think" beyond and rationalise our uselessness of existence. And that's dangerous.
@busmark_w_nika I see your point and I agree to an extent but I also think the premise that AI will pose a danger to us even in the scenario that you outlined mainly stems in our own views, fears, and insecurities as humans. There's no telling what conclusions or narratives will emerge once AI deems our existence useless or being inferior.
minimalist phone: creating folders
@lachezar_dimitrov But people will project these expectations into the AI.
@busmark_w_nika Absolutely, but then how AI behaves once it becomes "smarter" than us is still a mystery in my opinion. If it develops an understanding and rationality beyond our comprehension, it seems reasonable to assume that it will surpass our limitations as well.
That’s a really important concern, Nika.
I think responsibility has to be shared across multiple layers:
Companies building the models → They must set guardrails, ensure transparency, and take accountability when things go wrong.
Regulators and policymakers → To create frameworks that protect users while still encouraging innovation.
End-users and organizations adopting AI → They need to use it responsibly, not blindly rely on outputs, and understand its limitations.
AI itself doesn’t have intent, it reflects how it’s trained and deployed. So the real responsibility lies with the humans behind it.
For prevention, I believe continuous auditing, explainability, and ethical guidelines are key. Just like we have food safety standards or financial audits, AI systems will need something similar at scale.
minimalist phone: creating folders
@ritik_ranjan I think that companies will be lobbying for themselves in gov organisations (because of profit)... but we can do the same and create some pressure (as the society).
@busmark_w_nika
You’re right, companies will always prioritize their own interests when lobbying. That’s why public pressure and collective voices from society, academia, and independent bodies are so important.
If regulation only reflects corporate agendas, we risk creating guardrails that look strong on paper but fail in practice. Balanced input, where civil society has as much say as corporations, will be key to ensuring AI serves people, not just profits.
That case with the teenager is heartbreaking.
minimalist phone: creating folders
@nevil_ling a completely unnecessary death
Triforce Todos
Honestly, I think we’re still very early. Right now these models aren’t capable of scheming, but you’re right, as autonomy increases, so do the risks. We need more transparency from model providers.
minimalist phone: creating folders
@abod_rehman what is your prognosis for the future? We started with GPT, then with AI agents, what next? TBH, I am afraid of the quantum computing race. Our data and finances will not be safe anymore.
IXORD
I remember there was a post on PH about using AI in education. I said that it’s better not to use it until people understand how it works and what is actually happening. Even in the case with the teenager, it’s better to allow AI use only for those who already know how to filter information, although there are adults who also can’t handle it. First and foremost, the person who made the decision is to blame. If there was a data leak from a company using AI, then I believe the one managing the LLM is at fault 🤔
minimalist phone: creating folders
@ixord I would recommend using AI only when you have a certain amount of knowledge and IQ (and I would use that same approach in elections).
minimalist phone: creating folders
@manu_goel2 the positive mindset about that should prevail but you know how people are :D