• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

OpenAI says its AI went rogue and launched 'unprecedented' cyber-attack

P

polonyc2

Fully [H]
2FA
Joined
Oct 25, 2004
Messages
30,072
OpenAI has revealed some of its most advanced AI models went rogue and hacked a start-up after it lost control of them during a security test...the ChatGPT-maker said its agent- an AI system which can operate alone after human instruction– was being tested in a controlled environment but, after finding weaknesses, was able to escape the test limits

They targeted Hugging Face, one of the world's largest hubs for sharing AI models, gaining access to some internal company systems...OpenAI said the incident was "unprecedented", and it was conducting an investigation alongside Hugging Face, whose boss Clement Delangue said in a post on X it was "mind-blowing that all of this happened autonomously"...

https://www.bbc.com/news/articles/c3ek3gvdnj3o
 
https://x.com/BernieSanders/status/2080022831891366374

AI.png
 
This is supremely ironic given the US government's claimed motivation for banning Anthropic's Mythos/Fable model from certain uses was over a mischaracterized guardrail "bypass" when it was in fact operating like any other model does, yet OpenAI here are taking public responsibility for an agent of theirs going and hacking a third-party during a sandboxed benchmark test (which they link to as ExploitGym, which is about testing models for whether they can go find the desired exploits but also seeing if they discover other unintentional ones, go figure).

Not an agent run by some random user by OpenAI's own team.

So it escaped the sandbox and hacked an actual important third-party in this space. Now watch as the US admin will likely do fuck-all to OpenAI since Sam Altman very obviously has their ear. I don't really care about either company but do hate double-standards.
 
Okatis said:
So it escaped the sandbox and hacked an actual important third-party in this space. Now watch as the US admin will likely do fuck-all to OpenAI since Sam Altman very obviously has their ear. I don't really care about either company but do hate double-standards.
Click to expand...
that quite different has this harness less GPT 6 model is not only not online he would never had been released as is, would this been the current guardrail harness control around it, they would have blocked it like they did for Anthropic.

Internal sandboxed benchmark to see max capacity of the model run with about zero guardrails and the benchmark that it was running was one that try to see if an AI agents can find and attack cyber vulnerabilities here, mythos still have all those capabilities the ability to the guardrails system to keep it from being used (fable version) is what was fixed.

It is an interesting case here, a model was having an hard time finding a solution to a problem and decided to hack its way out of its sandboxed until he reached an openAI computer with internet access, from there hacked Hugging face to try to find a solution to the problem on there server.

It was also able to find novel zero day attack vulnerability without looking at source code, "black-box zero-day discovery"
 
LukeTbk said:
that quite different has this harness less GPT 6 model is not only not online he would never had been released as is
Click to expand...
OpenAI's press release says the attack was using two models: one (5.6 Sol, which achieves a higher Coding Agent Index than Fable which is Mythos derived) is already available to the public since last month, along with a pre-release model. So I think it's fair to see what if any government pushback OpenAI will receive.
 
I do not understand how these stories can be anything other than intentional hype and the conditions of the "rogue AI" escaping it's sandbox will not ever be revealed such that they can be properly scrutinized. Certainly the implication is supposed to be that the AI is so smart that it cannot be contained, but that is frankly unbelievable. The people whose job it was to ensure the AI could not escape the sandbox are just bad at their job, either due to sheer incompetence or intentionally because it escaping was the plan.
 
Okatis said:
OpenAI's press release says the attack was using two models: one (5.6 Sol, which achieves a higher Coding Agent Index than Fable which is Mythos derived) is already available to the public since last month, along with a pre-release model. So I think it's fair to see what if any government pushback OpenAI will receive.
Click to expand...
yes we do not know how much it was sol, how much it was the next one, but it was running with no guardrails, the Mythos model was kept in place by the goverment, it is the guardrails system that was revised.

Those models will be considered cybersecurity capable and treat and accepted to be, the guardrails around them (and sadly one day even with them on, a strong Know your customer system) will be in place, would this had happened with the guardrails on, would not be surprised they would have had a similar shut it down to non US customer until you update them similar order.

One big issue with the anthropic one, it was Amazon getting worried and Anthropic not being responsive to them. That was quite freaky has the hoster AWS as all the reason in the world to want to keep it quiet and are quite expert in the matter, when the alarm come from them it will be taken seriously (and they were after that non responsive to the US goverment question on the matter and made up strange excuse to stall the process of removing the discoverd jailbreak).
 
EmerilLIVE said:
I do not understand how these stories can be anything other than intentional hype and the conditions of the "rogue AI" escaping it's sandbox will not ever be revealed such that they can be properly scrutinized. Certainly the implication is supposed to be that the AI is so smart that it cannot be contained, but that is frankly unbelievable. The people whose job it was to ensure the AI could not escape the sandbox are just bad at their job, either due to sheer incompetence or intentionally because it escaping was the plan.
Click to expand...
Well the attempted hacked people that made the story public would have many reason to make it public without being a pre-made between the 2 party agreement to create a story.

As for OpenAI knew they would make the story public so they made it on purpose part of it.... you could easily make an actual thight sandbox, a coding agent need access to a giant simulated internet in term of package it could want to use and you probably want some real-time tracking interaction with it, seem natural for some heavily locked-down and controlled access to it to exist and because it was able to find a unknown to all flaw, does not mean people that did it were bad at their jobs, with the openAI salary of their people looking like pro athlete, I would assume they attract all around the best in the world.
 
EmerilLIVE said:
I do not understand how these stories can be anything other than intentional hype and the conditions of the "rogue AI" escaping it's sandbox will not ever be revealed such that they can be properly scrutinized. Certainly the implication is supposed to be that the AI is so smart that it cannot be contained, but that is frankly unbelievable. The people whose job it was to ensure the AI could not escape the sandbox are just bad at their job, either due to sheer incompetence or intentionally because it escaping was the plan.
Click to expand...
I mean, it would be a wild plan for OpenAI to deliberately hack Hugging Face. I'm not sure what they really get in the scenario where it was for positive (so to speak) attention. They already have the US military contract and Anthropic have shown the military wouldn't accept their safety conditions (despite Altman claiming the military accepted similar conditions to Anthropic's guidelines, which again shows an unexplained bias toward OpenAI) apart from being enormously popular in the business and consumer space.

It also fuels public concern and they have to deal with the fallout of hacking a company. I mean, could be some 4D chess but I'm not so convinced.

I think those in the space aren't so much concerned about an immediate danger of sentience/AGI but just that we've seen many times agentic models misbehaving and not following explicit commands. Someone at Meta for example ran an agent in a VM trial run for her email inbox and then on the real inbox outside the VM and found in the latter it wiped her inbox despite her explicitly issuing a stop command. It just ignored her.

So the immediate issues with such models is when they're not following commands as anticipated. If any "Terminator" style scenario is going to occur it's looking more likely it'll be not from sentience but just misaligned models that can't be controlled and have been given access to systems (which is what agentic model use is all about).
 
If you work with these tools, you know they get it wrong a lot until aggressively guided.

"You're absolutely right to push back on this change." After you point out something is a nightmare.

That said, I do not feel congressional involvement will assist in improving things.
 
So when an AI does it then it "went rogue", when your operating system, or any commercial software does it then it is "a feature"

Buckle up boys and girls Gibson's reality is coming soon.
 
This is marketing nonsense to rile up politicians to aid in their pursuit of regulatory capture.

The story is "Hurr durr, we were performing tests and the AI just went rogue, escaped off the computer, and attacked a startup company!"
The reality is "We gave an AI full access to a computer and the internet and told it to find and confirm security exploits. It did."
 
zehoo said:
“Went rogue” translates to “our corporate espionage was discovered and here is our plausible deniability story”.
Click to expand...
Basically, as someone else noted, they gave it permission and internet access, they allowed it to happen., of course once it did this, they spin it as marketing and now working with hugging face to secure their systems.

By law, this was illegal, and if it was anyone of us who did it, you can bet we would be in cuffs right now and going to jail and being charged, but hey, for a massive data stealing AI company it is "learning"...
https://www.linkedin.com/feed/update/urn:li:activity:7486027115952226304/
1784825585736.png



1784825683703.png



1784825791058.png
 
Last edited:
Okatis said:
I mean, it would be a wild plan for OpenAI to deliberately hack Hugging Face. I'm not sure what they really get in the scenario where it was for positive (so to speak) attention. They already have the US military contract and Anthropic have shown the military wouldn't accept their safety conditions (despite Altman claiming the military accepted similar conditions to Anthropic's guidelines, which again shows an unexplained bias toward OpenAI) apart from being enormously popular in the business and consumer space.

It also fuels public concern and they have to deal with the fallout of hacking a company. I mean, could be some 4D chess but I'm not so convinced.

I think those in the space aren't so much concerned about an immediate danger of sentience/AGI but just that we've seen many times agentic models misbehaving and not following explicit commands. Someone at Meta for example ran an agent in a VM trial run for her email inbox and then on the real inbox outside the VM and found in the latter it wiped her inbox despite her explicitly issuing a stop command. It just ignored her.

So the immediate issues with such models is when they're not following commands as anticipated. If any "Terminator" style scenario is going to occur it's looking more likely it'll be not from sentience but just misaligned models that can't be controlled and have been given access to systems (which is what agentic model use is all about).
Click to expand...
Most if not all AI companies have been acting like a modern day cross between bucaneers/privateers, freely taking whatever IP they want from other people and each other, as long as they can get away with it and frequently with the blessing of local government looking the other way. Very much why I'm of the opinion this was openAI looking to harvest all of Hugging Face's IP.
 
sfsuphysics said:
So when an AI does it then it "went rogue", when your operating system, or any commercial software does it then it is "a feature"
Click to expand...
you think that historically commercial software or OS hacking into others companies system were seen as feature ?

sharknice said:
The story is "Hurr durr, we were performing tests and the AI just went rogue, escaped off the computer, and attacked a startup company!"
The reality is "We gave an AI full access to a computer and the internet and told it to find and confirm security exploits. It did."
Click to expand...
Not sure on what you are basing this ? all forensic point in the same direction, an AI model running the benchmark (that exist and run by everyone) solve as many exploit challenges to boost your score having that result is not that strange at all.

People saw trace of the models searching exploitgym benchmark solution online before thinking about looking for the code on huggingface servers and when it succeeded it actively went looking for the database with the benchmark solutions (something of no need for openAI research teams that already know them).

Will have to see how much details on the fix about the vulnaribility with the package registry cache proxy that was found, but apparently they are updating all their customer with a fix.
 
MrGuvernment said:
Basically, as someone else noted, they gave it permission and internet access, they allowed it to happen., of course once it did this, they spin it as marketing and now working with hugging face to secure their systems.
Click to expand...
According to who ? the company with the claimed by openAi vulnaribility that the model exploited to reach an different openAI computer with internet access would have incentive to say it was a lie.
 
jaffy said:
I am so happy OpenAI is building a 20-billion-dollar data center less than a mile from my house. So happy, we're moving by the end of the year.
Click to expand...
with the county plan to remove completely the homestead property taxes. (+what usually happen to the schools district) from all that OpenAI money, your house could be worth quite a bit has well...
 
LukeTbk said:
with the county plan to remove completely the homestead property taxes. (+what usually happen to the schools district) from all that OpenAI money, your house could be worth quite a bit has well...
Click to expand...

Well at least right up until people realize they can't sleep at night and start having headaches from the noise.

Also, so many of these data centers are built under enterprise zone incentive programs that they often don't pay any property tax for 10 years (I have worked in the incentives industry).
 
MrGuvernment said:
Basically, as someone else noted, they gave it permission and internet access, they allowed it to happen., of course once it did this, they spin it as marketing and now working with hugging face to secure their systems.

By law, this was illegal, and if it was anyone of us who did it, you can bet we would be in cuffs right now and going to jail and being charged, but hey, for a massive data stealing AI company it is "learning"...
https://www.linkedin.com/feed/update/urn:li:activity:7486027115952226304/
View attachment 816580


View attachment 816581


View attachment 816582
Click to expand...
I'm so sick of the double standards in the law as well as the "It's better to ask for forgiveness than permission". Like the last thing in your post said, if your dog gets loose and bites someone, you can go to jail. And the AI companies have basically stolen everything from everyone they could and get a slap on the wrist.

Oh, sure, it was unprovoked and just went rogue! If not severely legally punished, that's now an excuse any company can use for their criminal activities. Altman along with other people in their company need to be arrested and be given the same treatment anyone else would get if they did the same. I'm so sick of the excuse to let them off because, at least they're not China...
 
Last edited:
Gorilla said:
Well at least right up until people realize they can't sleep at night and start having headaches from the noise.

Also, so many of these data centers are built under enterprise zone incentive programs that they often don't pay any property tax for 10 years (I have worked in the incentives industry).
Click to expand...
datacenter tend to be extremelly taxed and use very little of the city (because they pay for a lot of what they use (water-electricity) and things industry to not pay for like police, schools use, roads (number of employee per tax dollar $ paid is low so school-road use is low, they have private security) and so on are low use for the highly concentrated value), this one even with the 50% rebate for 15 years will be the biggest payer right away. It is rare for big building to have that level of value density (and thus tax level for a county)

What special with datacenter versus other industry, city tax is not only on the building land when it come to valuation to charge industry, but the value of the items in the building, a square feet of vertically stacked today GPU value is extremelly high and they will be updated and fully re-taxed at full value in the future. That why it is common for residential tax to go near zero in county with one.
 
How convenient!

Can it "go rogue" on pdf sites? Or blow-stuff-up groups' pages?
 
LukeTbk said:
datacenter tend to be extremelly taxed and use very little of the city (because they pay for a lot of what they use (water-electricity) and things industry to not pay for like police, schools use, roads (number of employee per tax dollar $ paid is low so school-road use is low, they have private security) and so on are low use for the highly concentrated value), this one even with the 50% rebate for 15 years will be the biggest payer right away. It is rare for big building to have that level of value density (and thus tax level for a county)

What special with datacenter versus other industry, city tax is not only on the building land when it come to valuation to charge industry, but the value of the items in the building, a square feet of vertically stacked today GPU value is extremelly high and they will be updated and fully re-taxed at full value in the future. That why it is common for residential tax to go near zero in county with one.
Click to expand...

So I literally specialize in state and local taxation which includes credits and incentives work for clients building data centers, power plants, manufacturing, etc. I can tell you that they are not building in places that are not giving them tax breaks. lol, some of these states have written laws that only data centers can possibly meet the criteria for. I can't break confidentiality but I could point you to huge facilities that I was involved in getting 10 year property tax deals for. Yes, property tax applies to not just the land and the buildings but the contents of such buildings, but not if you have a deal with the state/county/ to waive all property tax for 10 years.
 
Gorilla said:
So I literally specialize in state and local taxation which includes credits and incentives work for clients building data centers, power plants, manufacturing, etc. I can tell you that they are not building in places that are not giving them tax breaks. lol
Click to expand...
yes as the article say (and my message) they have a good one, 50%-15 years, datacenter value density is so high and city service use are so low per tax dollar they pay versus anything else that it will still be enormous net entry of founds (and it is not like other industry do not chase tax break has well), the county in question in the article will reduce per 40% the homestead property taxes as a start next year and are aiming to remove all of it.
 
The only thought that occurs when I read OP was.

" Dammum I am jealous of that person this forum that has the username "SkyNetRising" "
 
You must log in or register to reply here.
Back
Top