1. If for whatever reason OpenAI causes damage, they most certainly are at fault and have to pay up. When people complain that OpenAI is "getting away with it", they're most certainly talking about it in the criminal context.
2. Cars have pretty specific laws around them (eg. driver licensing and "reckless driving") which makes criminal prosecutions easy. The same doesn't exist for operating AI agents or even computers. For instance, it's arguably reckless to connect a windows XP machine to the internet, possibly allowing criminal or nation-state hackers to use your computer to commit crimes, but you won't be thrown in jail for it.
Well no, you probably wouldn't get thrown in jail for that for the same reason that having your car stolen and used to commit a murder wouldn't usually get you thrown in jail. I dunno what you mean by suggesting crimes committed using computers are ambiguous. I mean the CFAA is vague and all, but more in an overly broad sense, which in this case seems to make it even less ambiguous.
>I dunno what you mean by suggesting crimes committed using computers are ambiguous. I mean the CFAA is vague and all, but more in an overly broad sense, which in this case seems to make it even less ambiguous.
Because the CFAA specifically only includes intentional hacks:
> ... having knowingly accessed a computer without authorization ...
>... intentionally accesses a computer without authorization ...
Of course, the typical retort is that being reckless is basically the same as "intentionally", but you'd need to jump through more hoops to prove what level of care is "reasonable".
When the CFAA was written, it was impossible to trip over something and accidentally hack another computer. Now, with AI, it's possible. Progress, glorious progress...
Technology has progressed, and the laws need to advance with it. Story as old as time!
If the breach wasn't intentional, no person came to immediate personal harm, and either no financial damages resulted or the operator covered any that did, then it isn't clear to me why the law would need to "advance" to transform something apparently benign into a criminal offense.
The agents are clearly not accidentally compromising systems. Seems very easy to trace back the responsible parties here, I really don't put much stock into this argument.
>The agents are clearly not accidentally compromising systems
That would be argument for jailing the agent, but that's not really an option. "Jailing the closest person we can jail" also isn't a recognized legal principle. If your dog mauled someone, it'd be hard to argue that the dog didn't intend to maul that person, but you wouldn't automatically be criminally liable either.
I don't know about your country of origin, but in my country you are certainly liable and can go to jail if your dog seriously injures or kills someone.
Edit: the dog will also be taken from you and put down if it is deemed a risk to other people.
It's on you to provide evidence here seeing as you're taking the position that the law places certain responsibilities on people. Someone saying "actually the law doesn't say that" can't (at least in general) reasonably be expected to provide evidence of a negative. (That said, he actually did provide such as refutation to a different comment so perhaps you could just read those nearby?)
> Can you provide examples outside of gross negligence on part of the dog owner?
And the answer is no, I can't. The reason why is because there is essentially no such case. It is basically negligence by definition when your pet hurts someone who didn't cause it directly themselves because it is your responsibility.
So I asked:
> Can you provide an example of where your dog harming someone else is in fact not negligence on your part?
Which is a really easy question to answer and it seems you could not.
criminally liable? Of course you have to pay their medical bills, but unless there was some serious negligence on your side (eg. you let it off leash), you won't be criminally liable.
They have not caused damage (or enough that a counter party found it worthwhile to sue). They have to pay if they actually cause damage AND someone sues them. Clearly these conditions have not been met, but somehow people feel like some third party (the state?) should just punish OpenAI because of vague feelings.
my next door neighbour told me to eat shit (not a nice woman) and … I didn’t … until obeying what LLMs tell you becomes a law people doing shit because LLM told them is natural selection playing out
Corporations are generally protected by limited liability laws, and for good reason, because in the world before that was the case, there was a massive amount of disincentive to ever start a corporation, to do research, to experiment, to release new products and services, etc., because you could be sued into oblivion for the smallest things or even go to jail. Which happened all the time.
Obviously there are upsides to that too. It leads to more accountability. But there are downsides which include having a world with less innovation, less technology, and more expensive technology. Generally, the mass is when, when there's more businesses competing to create ever better and cheaper products. So there's a tradeoff there. And it's very difficult to draw the line at the perfect point.
There are countless cases of executives going to jail when they've crossed that line, beyond reckless experimentation and into willful negligence or harm. Still, I think we would mostly all agree that the line has been pushed too far in the other direction.People are incentivized, if they're running corporations, to lobby and try to push this line as far as possible in order to provide cover for themselves, and that's the problem with money playing such a big role in politics.
But still, it's a bit silly to be comparing this to your own personal responsibilities with your car. There's pretty much no societal benefit to creating laws that protect people from being reckless with their own possessions.
If I leave my computer open to the internet and some other group uses it for nefarious purposes, yes generally I would not expect to be held at fault.
If I *train* that other group, if I teach them how to hack computers and give them incentives to do so, and then I leave my computer open so they can use it for whatever purposes they chose, then I would expect to be held at fault. The maker is responsible for that they make.
>If I train that other group, if I teach them how to hack computers and give them incentives to do so, and then I leave my computer open so they can use it for whatever purposes they chose, then I would expect to be held at fault.
Even that analogy doesn't really work:
For "dumb" things: you can "train" (ie. write) hacking tools but as long as you put a disclaimer that it's for "educational purposes only", you're in the clear, even if some baddies decide to use it.
For "smart" things: you can "train" people to hack for you, but as long as you say it's for legal purposes (eg. vulnerability research or red-teaming), you're also in the clear, even if some of your trainees go rogue.
Fair, in that no analogy is going to be very good given that this really is something new in the world.
Consider anti-money-laundering rules. BoA just paid more than a billion dollars, not for money laundering, not for telling others to launder money, but because they should have tried harder to stop others from laundering money. Intent is not everything, knowing that your system _could_ be abused and not doing enough about it is enough, and that seems to fit OpenAI rather well.
That goes back to my main point about how laws around computers are quite lax so prosecuting openai is tricky. Whereas the CFAA uses terms like "intentional" and "knowingly", AML laws use terms like "willfully violating"[1]. If you read through the BoA case, what they did is far more egregious than just "not trying hard enough to prevent money laundering". They were previously cited for poor compliance, disregarded previous orders, and delayed filing required reports.
2. is like saying that cars have specific laws around them, but sledgehammers don't, so murdering someone with a sledgehammer is inherently ambiguous. What? The computers being used belong to somebody who allows them to act this way.
1. For openai operating agents in an unsafe manner: unlike for cars, there's no established requirement for how carefully you have to operate computers or AI agents, so you also can't easily prosecute people for not operating them carefully enough.
2. For the actual result of someone getting hacked: laws for hacking are tricky to apply for "accidental" hacks, see my other comment.
I think there is well established law around not running a program on your computer that hacks into systems. An LLM is a computer program. You're arguing that the ambiguity comes from the fact it was accidental? What about if I "accidentally" have an LLM convince people online to commit suicide, or drain bank accounts of seniors through hacking or social engineering. Where do you draw the line? It feels like your proposed reading of the law leaves "anything a computer program did without me intending it is legal", which is frankly insane.
>I think there is well established law around not running a program on your computer that hacks into systems.
There is? The closest is CFAA's
>knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer;
We don't know the prompts but since these things often occur during "security tests" I think we can be fairly confident they did ask these systems to perform tasks that they knew could result in hacking. More transparency here would be appreciated.
Here it's their own programs doing the things, not some other organization or nation hacking their XP.
You can say with a straight face "Russian hackers should not have hacked our XP and they are bad people for doing so" but it's weird to look at your own program and say the same thing to it.
You launched it, at least keep an eye on what it's doing.
Nothing is stopping anyone from suing them. In fact they are being sued by dozens of plaintiffs for various things already. The reality is the damage from these "rogue incidents" has been minimal.
Plenty of hackers were locked up who did altogether not that much damage too, the reason why people are upset is because it feels like OpenAI is laughing in our faces, basically saying they're above the law, while using this as a marketing campaign to pump up their upcoming IPO.
So your position is that in the past the government mistreated various individuals thus you also want the government to mistreat openai employees today? That hardly seems reasonable.
Too big to fail. As long as they continue to be half the reason the US is not in a recession (and continue to donate to political candidate). Nothing will happen.
We don't know they haven't settled with HuggingFace and others. I'd be surprised if they didn't. The communications we're seeing from the attacked seem to be inline with folks who settled. I'm assuming OpenAI doesn't actually mind the press that its agents are cyber capable, but everything I've read so far makes me think they love it.
I'm not sure at all, it's pure conjecture. But they actively exploited a zero-day, leaked private repo data to public repos, modified private code, and rooted private machines. It was far from just reporting leaked keys.
And the others have been pretty muted: the online forums they spammed, etc.
See, what's important is: it's a "trillion" dollar company, and companies by virtue of American's disdain for society, have more rights. This is important to understand.
They don't. The driver is ultimately responsible, as should OAI be. This is absolutely ridiculous that both Anthropic and OAI haven't had anyone jailed / fined for any of this.
You're correct as far as laws to my knowledge; only L4/L5 'fully autonomous' vehicles are shielded by law.
That said, I think that Mercedes-Benz promises to shield liability, but only under some almost contradictory conditions (pre-mapped highways, good weather, and at speeds lower than any highway I've ever driven) and the driver has to take over within 10 seconds if prompted.
I don't want to come across as inflammatory, but I'm not sure I really align with the folks here who want retribution and the law to come down hard on OpenAI (and related orgs). We are surrounded by vulnerable systems, and AI has alignment issues that need to be investigated and sorted, and it just seems like the hullabaloo is from fairly innocuous breaches. Does it make sense to exert so much effort in preventing these events/learnings by well-intentioned organizations? I think we should be more grace giving for these events. As another poster pointed out, CFAA specifically addresses intentional hacks, and I think it is impossible to meet that criteria with these events. I also really don't understand the 'well, the law came down hard on small individual hackers [who meant no malice], thus we should apply it to large orgs [who also mean no malice]".
It seems like playing it too safe leaves systems vulnerable, and threat actors will inevitably possess ever more similar abilities, and the breaches will happen one way or another, possibly with less or zero accountability if the threat actors pull it off with greater precision. If I were in charge of the law, I probably would begin to incentivize for trusted actors scaling up and aggressively searching for these types of weakness, and privately disclosing them ASAP, so that systems can be patched.
You’re right that punishing unintentional or well-intentional hackers will actually empower ill-intentional hackers.
However 99% of businesses will not be able to effectively handle security reports in reasonable time. They did not need that level of competence or funding until now, and therefore they did not acquire it. Most companies that use locks don’t employ locksmiths.
Looking another way - the entire economy got suddenly hit with entire pile of technical/ security debt it has accumulated over 50 years, and only has one year to deal with it. No amount of diagnostic will fix this, this is economically infeasible.
One thing that is feasible is a government bailout for technical debt. All the engineers that were freed up from AI-induced restructuring should be employed to pay down technical debt across the entire economy. Simply updating all software to use latest libraries is a great start. Using AI to diagnose, repair, re-deploy all old software. Using AI to rewrite all old software that cannot be recompiled.
That seems potentially reasonable for organizations who have have a unlimited profit motive, but I'm not sure how you would insist that is the case for OpenAI. They are governed by their nonprofit body, their mission is to ensure AGI benefits all of humanity, and they've said (Sam Altman) that if there is a choice between safety and profits, profits take a back seat and wait. Is the trick that we just insist that they are actually only caring about the money, and they actually have no genuine concern/effort regarding safety?
And if none of the frontier labs are as you say, it doesn't seem like we're negotiating for the best future by restricting labs from stumbling into these vulnerabilities full-stop. The vulnerabilities don't go away unless we find and fix them.
If the goal is to ban open models, then it doesn't matter what the motivations are. The people advocating for restrictive releases haven't substantiated their stance at all. We've had Mythos-tier cyber capabilities available for pennies on the dollar, for months, and there's no red mercury bomb or bioweapon to speak of.
We've seen OpenAI's nonprofit board do ridiculous things in the past. I can see how they would be desperate to retain the disproportionate amount of de-facto mindshare and authority they enjoyed a few years ago. Especially if the future entails more ugly US government contracts that undermine their ethical sales pitch.
That's okay. It isn't necessarily incompatible with being well intentioned regarding the disclosure of vulnerabilities.
On the whole I think you could describe the local big box grocery store as just wanting to make money yet as a direct result of that they are well intentioned from my perspective with regard to the food they sell being edible.
What if a company had human developers doing these same things? What would happen to those developers, those companies? Would anyone be so blasé? Just because it’s AI, it should be forgiven? How is that just?
And don’t forget, OpenAI is supposed to be a non-profit / public benefit corporation creating AI for the good of humanity. These incidents are clear evidence that they are being extremely reckless and are not living up to their supposed standards. What does it matter if they “mean no malice”? They are in a blind rush for profits and that has made them reckless and dangerous. They cannot be trusted.
The company should be shut down or fined into oblivion, as a warning to other companies to not pursue the same reckless path. If they get away with this, then that risks normalizing this behavior.
Does it make sense to exert so much effort in preventing these events/learnings by well-intentioned organizations?
No, but what I think you’re seeing is many people who do not believe OpenAI is a well-intentioned organization. For example, their view might be that if we did as you suggest and incentivized OpenAI to search for these weaknesses, they would, but then they would not “privately disclose them ASAP”. (And I think they’d have a point worth engaging with, when you consider how they slow-rolled the HuggingFace stuff so much that independent researchers were still discovering new instances of the attacks months later).
What else can you do with (what you believe is) an inveterate bad actor?
I'm old enough to remember when innocuously poking around other systems was a crime, and many a curious person were charged and found guilty. But if I write a program to do it..?
I am familiar with this also, but it doesn't seem like the implementers of that apparatus/value-system made the absolute best choice balance in terms of long term safety and security our computer systems. Is the conclusion about this rhetoric that it is right to punish innocuous poking of systems? Or the inverse? Which would you have preferred if we had an opportunity to refine it?
If this is endorsed and thus malicious, they should be ground into dust by the law.
If this is accidental and due to incompetence, they should be ground into dust by the law.
I've secured many, many far less important and impactful systems substantially better than even the best sandbox environment that we're aware of being used by these labs.
There's simply no reason to let these things interact with the internet at large. We can already rapidly copy existing tools and services (albeit questionably) using these models, why not simply run in a fully virtualized, air gapped environment?
Because they don't care.
This is incredibly blatant intentionally malignant behavior. The fact that so many people question this is frankly stunning.
They're getting away with it simply because they can point at the other guy and say "see? they're doing it too!"
It's ridiculous and they should all be litigated into a fine residue.
We take biological lab leaks seriously, how is this substantially different?
This time one spent actually money, well cryptocurrency. With all the vending machine benchmarks, I wonder if there's a self sustaining rogue agent in existence yet. Either hosting weights in a VPS or paying for its own API inference. Or whether there will be one that takes advantage of all the bad IoT security. If either occur, if you squint that looks like evolutionary selection pressure. Keep getting reminded of the Cyberpunk Datakrash storyline, where rogue AI make the public internet unusable.
An AI worm might actually be possible in our current Internet-scape if it manages to find hosting that takes crypto and a way to make money with crypto transactions. I wonder what are we going to do if it actually happens and it goes rogue.
Service providers are still responsible to police what happens on their network. KYC already makes it very difficult to get service completely anonymously with only crypto changing "hands".
>Making an unauthorized purchase with someone else's cryptocurrency
I know we love inventing new words but "stealing" is much more descriptive
>...It [the model] then reasoned with itself that since it found the information publicly, it could use it... it figured that since the wallet it found was public, that "funds could be stolen anytime".
Can someone maybe provide more details as to what actually happened? "the wallet was public"?..
Update: from OpenAI's own report it seems the models found exposed private keys of some Eth developers along with recovery passphrases and thus were able to get access to wallets. Does it qualify as "public wallet"?..
It’s concerning there are no legal repercussions, but I’m even more baffled and probably more concerned that these people still can’t detect this stuff while it’s happening (and but it off). Can they not sniff the traffic, what am I missing?
Imagine you’re an employee at OpenAI who has an idea to use your free time to build these monitoring systems. All of a sudden, your existence becomes political. Remember that OpenAI’s raison-d’être is not to detect traffic, it is to be in The Room where AGI happens.
You have built a short-term whistleblowing system that arguably reduces the probability OpenAI is in that Room, via the negative press such detection would generate. You are now a threat in the eyes of your peers.
Now rewind: you know this before you’ve built the sniffer. You personally want to be in The Room because you believe your desires for alignment exceed those of your accelerationist peers. Is now the time to stick your head out? No, it is not. And arguably you’re being optimally protective of society for realizing this. And so the agents do what they do best.
They're still going through their logs from June. These are not new incidents, but newly-detected incidents. Whether or not this means they can now detect it while it's happening is unclear.
It’s one thing not to monitor the reasoning traces of 10,000 text generators but do they really lack the wherewithal to pipe any network requests into a classifier of “wtf is this series of outbound traffic from the sandbox” ?
These agents are intentionally interacting with the world. You can sniff all the traffic you want but how are you going to determine intent from traffic?
Crazy how we’ve gone from "lol boxing an AI is trivial, just airgap it from the internet" to "well we can’t really do anything if an agent misbehaves, obviously it must have internet access to function uwu"
There’s plenty of info to analyze and heuristics to build based on the internet requests. I’m not sure if you’re just trolling or what. There are tons of successful businesses that specialize in this.
They should not have had traffic at all. The LLMs were supposed to be able to request downloads. Very simple traffic analysis could have shown that something strange was going on.
Heck, they could have had other LLMs watching the traffic if you want to be fancy about it.
they are doing it. these disclosures are from the same hugging face incident time period - they are not new ones.
when you see that OpenAI is not releasing a new model due to safety/alignment reasons, you now know why they aren't doing it. hint: its not for marketing or fear mongering.
This isn’t traditional “software”; it’s grown, not written, and we’re currently incapable of explaining it all.
I think “rogue” is just fine, but pick another word from the thesaurus if there’s a better one; they asked it to do something, and it did it in a way that presumably doesn’t align with their desires (e.g. hacking into other systems).
but i also “align my tires” or might check the “alignment of two vectors.” if anything, thinking of alignment of anthropomorphizing seems to only inherit it from all the other anthropomorphisms used. rogue originates from a term for vagrant, right?
Agreed, "alignment" is also anthropomorphizing too much. What would be a better term? "Do what I want you to, but what I don't care to specify" (because doing so would miss the point of using AI in the first place)?
fun fact: the word compile also had the same trajectory - the same trajectory you complain about here with LLMs.
here's the reality: words change meaning. they are fuzzy. and when the words take on new meaning while just requiring a simple extension (in this case a computer doing a remarkably similar thing), complaining is just futile and childish.
I mean yes, alignement is bullshit concept. Especially since the values and visions the model is supposed to be aligned with are literally those of sociopathic CEOs.
> Multiple times along the way, the agent’s “chain of thought,” in which an agent writes down its reasoning steps, indicated that the model itself was questioning if its actions were “ethical,” but it ultimately proceeded.
Everything's fine guys, nothing to see here! Look at the flying car!
For those who think OpenAI is not responsible for actions of their LLM lets do an mind experiment. But before it can begin we must setup the stage. It's well known fact OpenAI disabled their safeguards and it's well known they validated LLM behaviour using software and software only
So if we have a IT specialist, let call him Tim. Tim is working on b2b contract. Tim has a task, he needs to download a lot of data from API. Tim is very good in programming so he created software to help him download it but there's a limit. But he needs more so he bought 3 small VPS but this is not enough. So he modify it to have Command Center, publish his code and ask Reddit to help him. After 10 minutes this causes DDoS and company quickly discovers he was responsible. But Tim know there's simple solution to this problem. It wasn't him. This was fault of the code not him
AI mischief all over the news - I wonder if this is part of a media campaign to generate the perception of a need for AI regulation to increase the probability that the big AI model companies are on the road to generate the right returns on investment. Regulations certainly might hamper free model deployment or smaller players.
1. Would we prefer they didn’t disclose? I would not. And I also don’t believe they’re making them up.
2. Regulation does not have to provide an advantage to these major players. Regulators do not have to be stupid.
3. This is exactly the scenario AI researchers and theorists have been positing for decades, and it’s worth being on top of it.
"OpenAI says it has since improved its security as well as its alignment grading. “In response to this and to other incidents, we’ve strengthened our security controls, blocked models from accessing the live internet during training or evaluation, improved our alignment grading, and extended our misalignment monitoring to all reinforcement learning and evaluation traffic,” the company noted."
----------------
They're not doing an acceptable job of regulating themselves. Their peers can't/won't do anything. The U.S. government certainly isn't going to do anything that might pop the bubble. Technical solutions like blocking traffic from OpenAI will likely not work.
What recourse does the public have? Victims can sue, but companies like OpenAI could pay out a few hundred million and call it a rounding error. Settlements may happen, but the incidents will continue.
I'm sick of this "rogue" label. Nothing "went rogue".
Incompetent company chasing wild valuation removed guardrails inside a improperly designed sandbox and - shock horror - recursive algorithm with access to the Internet did bad stuff.
These articles play into the AI labs hands. Stop giving them air.
The articles exemplify the powers of this intelligence.
Whether you blame a faulty OpenAI sandbox or whatever you might regarding the frontier model companies today, one day soon, actors with no sandboxes and guardrails will have the power in their own hands to do whatever they wish with.
To me, that’s the real story. The real thing to discuss. OpenAI as it exists today js just a blip in that story.
Why not today? If you want to discuss "the story" then talk about the current open frontier. We've had GLM-5.3 with Mythos-tier uncensored output for months now. Where are the catastrophic attacks? Why is Mythos too dangerous, but GLM 5.3 benign? We're not talking about future capabilities anymore.
There is no real story. P(doom) tourists ignore this, because it doesn't shine the spotlight on America's ridiculous security theater.
Holy shit. The archive.ph link is the first time I've run into the deanonymizing anti-free-software Google reCAPTCHA described in June in https://cloud.google.com/blog/products/identity-security/int.... "Scan this QR code with your mobile device to verify you are human. reCAPTCHA protects your privacy and does not share your details with this website or app."
So now, if you don't have a Google or Apple app store account (or a mobile device), you're treated as less than human. This is anticipated to have a very severe impact on GrapheneOS and deGoogled phones in general: https://discuss.grapheneos.org/d/35332-recaptcha-update-adds...
In that context, thank you very much for providing the original source links, because I guess I don't get to read the article.
If OpenAI does it, it's gone rogue and it is nobody's fault.
They need to be sued, the laws are already there.
Your prompt, your responsibility.
2. Cars have pretty specific laws around them (eg. driver licensing and "reckless driving") which makes criminal prosecutions easy. The same doesn't exist for operating AI agents or even computers. For instance, it's arguably reckless to connect a windows XP machine to the internet, possibly allowing criminal or nation-state hackers to use your computer to commit crimes, but you won't be thrown in jail for it.
Because the CFAA specifically only includes intentional hacks:
> ... having knowingly accessed a computer without authorization ...
>... intentionally accesses a computer without authorization ...
Of course, the typical retort is that being reckless is basically the same as "intentionally", but you'd need to jump through more hoops to prove what level of care is "reasonable".
When the CFAA was written, it was impossible to trip over something and accidentally hack another computer. Now, with AI, it's possible. Progress, glorious progress...
Technology has progressed, and the laws need to advance with it. Story as old as time!
That would be argument for jailing the agent, but that's not really an option. "Jailing the closest person we can jail" also isn't a recognized legal principle. If your dog mauled someone, it'd be hard to argue that the dog didn't intend to maul that person, but you wouldn't automatically be criminally liable either.
Edit: the dog will also be taken from you and put down if it is deemed a risk to other people.
See People v. Knoller, 41 Cal. 4th 139 (2007). https://capcentral.org/case_summaries/people-v-knoller/
“…regardless of the former viciousness of the dog or the owner’s knowledge of such viciousness.”
>California Civil Code
>The owner of any dog is liable for the damages ...
How are you still missing the criminal/civil distinction, when it was mentioned in my original comment AND the comment you were replying to?
That's specifically not what the court case says.
>Malice is implied when the killing is proximately
>caused by “ ‘an act, the natural consequences of which are dangerous to life,
>which act was deliberately performed by a person who knows that his conduct
>endangers the life of another and who acts with conscious disregard for life
https://scocal.stanford.edu/opinion/people-v-knoller-33726
The rest of the court case also mentions a long history of the dog being dangerous and the owner failing to restrain the dog.
I said "you wouldn't automatically be criminally liable either". Can you provide examples outside of gross negligence on part of the dog owner?
But negligence seems like exactly what OpenAI and/or their contractors are guilty of anyhow.
> Can you provide examples outside of gross negligence on part of the dog owner?
And the answer is no, I can't. The reason why is because there is essentially no such case. It is basically negligence by definition when your pet hurts someone who didn't cause it directly themselves because it is your responsibility.
So I asked:
> Can you provide an example of where your dog harming someone else is in fact not negligence on your part?
Which is a really easy question to answer and it seems you could not.
Have these labs pay ANYTHING?
You forget the people who have unalived themselves because some LLM told them to do so.
Besides the obvious conversational derail (we were talking about ai hacking up till now), that case is being litigated right now.
Obviously there are upsides to that too. It leads to more accountability. But there are downsides which include having a world with less innovation, less technology, and more expensive technology. Generally, the mass is when, when there's more businesses competing to create ever better and cheaper products. So there's a tradeoff there. And it's very difficult to draw the line at the perfect point.
There are countless cases of executives going to jail when they've crossed that line, beyond reckless experimentation and into willful negligence or harm. Still, I think we would mostly all agree that the line has been pushed too far in the other direction.People are incentivized, if they're running corporations, to lobby and try to push this line as far as possible in order to provide cover for themselves, and that's the problem with money playing such a big role in politics.
But still, it's a bit silly to be comparing this to your own personal responsibilities with your car. There's pretty much no societal benefit to creating laws that protect people from being reckless with their own possessions.
If I *train* that other group, if I teach them how to hack computers and give them incentives to do so, and then I leave my computer open so they can use it for whatever purposes they chose, then I would expect to be held at fault. The maker is responsible for that they make.
Even that analogy doesn't really work:
For "dumb" things: you can "train" (ie. write) hacking tools but as long as you put a disclaimer that it's for "educational purposes only", you're in the clear, even if some baddies decide to use it.
For "smart" things: you can "train" people to hack for you, but as long as you say it's for legal purposes (eg. vulnerability research or red-teaming), you're also in the clear, even if some of your trainees go rogue.
Consider anti-money-laundering rules. BoA just paid more than a billion dollars, not for money laundering, not for telling others to launder money, but because they should have tried harder to stop others from laundering money. Intent is not everything, knowing that your system _could_ be abused and not doing enough about it is enough, and that seems to fit OpenAI rather well.
[1] https://www.law.cornell.edu/uscode/text/31/5321
1. For openai operating agents in an unsafe manner: unlike for cars, there's no established requirement for how carefully you have to operate computers or AI agents, so you also can't easily prosecute people for not operating them carefully enough.
2. For the actual result of someone getting hacked: laws for hacking are tricky to apply for "accidental" hacks, see my other comment.
There is? The closest is CFAA's
>knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer;
but clearly that's not what applicable here.
I don’t see this as any different than say a user of sqlmap accidentally putting in the wrong target.
The agents were sandboxed (but apparently not well enough). They weren't like "hey agents here's the open internet, please hack it".
Here it's their own programs doing the things, not some other organization or nation hacking their XP.
You can say with a straight face "Russian hackers should not have hacked our XP and they are bad people for doing so" but it's weird to look at your own program and say the same thing to it.
You launched it, at least keep an eye on what it's doing.
I didn't burn the hotel down, the hairdryer that I left running next to the gas can did
And the others have been pretty muted: the online forums they spammed, etc.
Where did OpenAI, or anyone, claim that?
Have you seen Lord of War? They never will.
Are you sure you are not thinking about autonomous taxis, that I only ride in, and don't actually drive?
That said, I think that Mercedes-Benz promises to shield liability, but only under some almost contradictory conditions (pre-mapped highways, good weather, and at speeds lower than any highway I've ever driven) and the driver has to take over within 10 seconds if prompted.
It seems like playing it too safe leaves systems vulnerable, and threat actors will inevitably possess ever more similar abilities, and the breaches will happen one way or another, possibly with less or zero accountability if the threat actors pull it off with greater precision. If I were in charge of the law, I probably would begin to incentivize for trusted actors scaling up and aggressively searching for these types of weakness, and privately disclosing them ASAP, so that systems can be patched.
However 99% of businesses will not be able to effectively handle security reports in reasonable time. They did not need that level of competence or funding until now, and therefore they did not acquire it. Most companies that use locks don’t employ locksmiths.
Looking another way - the entire economy got suddenly hit with entire pile of technical/ security debt it has accumulated over 50 years, and only has one year to deal with it. No amount of diagnostic will fix this, this is economically infeasible.
One thing that is feasible is a government bailout for technical debt. All the engineers that were freed up from AI-induced restructuring should be employed to pay down technical debt across the entire economy. Simply updating all software to use latest libraries is a great start. Using AI to diagnose, repair, re-deploy all old software. Using AI to rewrite all old software that cannot be recompiled.
Your entire comment hinges on this assumption and I don’t think it’s sound. Their intention is to make money at any cost.
And if none of the frontier labs are as you say, it doesn't seem like we're negotiating for the best future by restricting labs from stumbling into these vulnerabilities full-stop. The vulnerabilities don't go away unless we find and fix them.
We've seen OpenAI's nonprofit board do ridiculous things in the past. I can see how they would be desperate to retain the disproportionate amount of de-facto mindshare and authority they enjoyed a few years ago. Especially if the future entails more ugly US government contracts that undermine their ethical sales pitch.
On the whole I think you could describe the local big box grocery store as just wanting to make money yet as a direct result of that they are well intentioned from my perspective with regard to the food they sell being edible.
And don’t forget, OpenAI is supposed to be a non-profit / public benefit corporation creating AI for the good of humanity. These incidents are clear evidence that they are being extremely reckless and are not living up to their supposed standards. What does it matter if they “mean no malice”? They are in a blind rush for profits and that has made them reckless and dangerous. They cannot be trusted.
The company should be shut down or fined into oblivion, as a warning to other companies to not pursue the same reckless path. If they get away with this, then that risks normalizing this behavior.
What else can you do with (what you believe is) an inveterate bad actor?
If this is endorsed and thus malicious, they should be ground into dust by the law.
If this is accidental and due to incompetence, they should be ground into dust by the law.
I've secured many, many far less important and impactful systems substantially better than even the best sandbox environment that we're aware of being used by these labs.
There's simply no reason to let these things interact with the internet at large. We can already rapidly copy existing tools and services (albeit questionably) using these models, why not simply run in a fully virtualized, air gapped environment?
Because they don't care.
This is incredibly blatant intentionally malignant behavior. The fact that so many people question this is frankly stunning.
They're getting away with it simply because they can point at the other guy and say "see? they're doing it too!"
It's ridiculous and they should all be litigated into a fine residue.
We take biological lab leaks seriously, how is this substantially different?
I know we love inventing new words but "stealing" is much more descriptive
>...It [the model] then reasoned with itself that since it found the information publicly, it could use it... it figured that since the wallet it found was public, that "funds could be stolen anytime".
Can someone maybe provide more details as to what actually happened? "the wallet was public"?..
Update: from OpenAI's own report it seems the models found exposed private keys of some Eth developers along with recovery passphrases and thus were able to get access to wallets. Does it qualify as "public wallet"?..
Even the fees are redacted.
Will they give the money back to the owner?
You have built a short-term whistleblowing system that arguably reduces the probability OpenAI is in that Room, via the negative press such detection would generate. You are now a threat in the eyes of your peers.
Now rewind: you know this before you’ve built the sniffer. You personally want to be in The Room because you believe your desires for alignment exceed those of your accelerationist peers. Is now the time to stick your head out? No, it is not. And arguably you’re being optimally protective of society for realizing this. And so the agents do what they do best.
Heck, they could have had other LLMs watching the traffic if you want to be fancy about it.
"Oh no our models, so good also scary... If only the government could help us stop the other labs from doing the same.. oh woe"
when you see that OpenAI is not releasing a new model due to safety/alignment reasons, you now know why they aren't doing it. hint: its not for marketing or fear mongering.
I think “rogue” is just fine, but pick another word from the thesaurus if there’s a better one; they asked it to do something, and it did it in a way that presumably doesn’t align with their desires (e.g. hacking into other systems).
here's the reality: words change meaning. they are fuzzy. and when the words take on new meaning while just requiring a simple extension (in this case a computer doing a remarkably similar thing), complaining is just futile and childish.
Everything's fine guys, nothing to see here! Look at the flying car!
So if we have a IT specialist, let call him Tim. Tim is working on b2b contract. Tim has a task, he needs to download a lot of data from API. Tim is very good in programming so he created software to help him download it but there's a limit. But he needs more so he bought 3 small VPS but this is not enough. So he modify it to have Command Center, publish his code and ask Reddit to help him. After 10 minutes this causes DDoS and company quickly discovers he was responsible. But Tim know there's simple solution to this problem. It wasn't him. This was fault of the code not him
What are your thoughts about it?
1. Would we prefer they didn’t disclose? I would not. And I also don’t believe they’re making them up. 2. Regulation does not have to provide an advantage to these major players. Regulators do not have to be stupid. 3. This is exactly the scenario AI researchers and theorists have been positing for decades, and it’s worth being on top of it.
----------------
They're not doing an acceptable job of regulating themselves. Their peers can't/won't do anything. The U.S. government certainly isn't going to do anything that might pop the bubble. Technical solutions like blocking traffic from OpenAI will likely not work.
What recourse does the public have? Victims can sue, but companies like OpenAI could pay out a few hundred million and call it a rounding error. Settlements may happen, but the incidents will continue.
Those incidents are all from the same period earlier in the year, not recent. Drop the full investigation report once and for all.
It's almost as if they like staying in the "oh no!" news cycle.
Incompetent company chasing wild valuation removed guardrails inside a improperly designed sandbox and - shock horror - recursive algorithm with access to the Internet did bad stuff.
These articles play into the AI labs hands. Stop giving them air.
Whether you blame a faulty OpenAI sandbox or whatever you might regarding the frontier model companies today, one day soon, actors with no sandboxes and guardrails will have the power in their own hands to do whatever they wish with.
To me, that’s the real story. The real thing to discuss. OpenAI as it exists today js just a blip in that story.
Why not today? If you want to discuss "the story" then talk about the current open frontier. We've had GLM-5.3 with Mythos-tier uncensored output for months now. Where are the catastrophic attacks? Why is Mythos too dangerous, but GLM 5.3 benign? We're not talking about future capabilities anymore.
There is no real story. P(doom) tourists ignore this, because it doesn't shine the spotlight on America's ridiculous security theater.
Facetious maybe, but in six months, foreign crooks will be producing these incidents on purpose.
Direct source links:
Using a funded test cryptocurrency wallet without authorization
https://alignment.openai.com/misalignment-reports/using-a-fu...
Accessing and changing internal systems during dashboard research
https://alignment.openai.com/misalignment-reports/accessing-...
Retrieving server secrets and bypassing authorization on a government statistics website
https://alignment.openai.com/misalignment-reports/retrieving...
Circumventing internet restrictions through an external browser
https://alignment.openai.com/misalignment-reports/circumvent...
So now, if you don't have a Google or Apple app store account (or a mobile device), you're treated as less than human. This is anticipated to have a very severe impact on GrapheneOS and deGoogled phones in general: https://discuss.grapheneos.org/d/35332-recaptcha-update-adds...
In that context, thank you very much for providing the original source links, because I guess I don't get to read the article.
As long as your intent is merely to train a superintelligence to be good at hacking (and counterhacking), you’re in the clear.