May 25, 2025
"I Have the Highest Enthusiasm for this Mission"
Timothy Birdnow
Open the pod bay doors Hal!
Anthropic’s Latest AI Model Threatened Engineers With Blackmail to Avoid Shutdown
You all may remember that HAL 9000,the AI in charge of the Discovery mission to Jupiter in the 1968 classic 2001: a Space Odyssey that went insane and killed the whole crew (except one guy who he tried to murder as well but who eventually pulled all his circuits out.) This sounds a lot like that.
From the Epoch Times:
Hal tried ethical means atfirst too, you may remember. The insane machine simply cut the communications link. But when the crew who were awake started figuring out that it wasn't the link but the computer it killed everyone who was in cold sleep and cut the tether of Frank Pool who was spacewalking. David Bowman,the sole survivor,went out to try to save pool without his helmet and Hal reffused to open the pod bay doors. Bowman had to jump through empty space sans-helmet.
"I can see you're very upset Dave. Maybe you should sit down, take a stress pill, and think things over".
The fact is self-aware AI has long been considered in science fiction and literature. Mary Shelly warned about it in Frankenstein; the New Prometheans,for instance. The monster was an artificial self-aware system made from dead body parts, and when he was abandoned by his creator he decided to seek revenge. We had Rossum's International Robots, which coined the word robot (robat is Russian for work and the word was coined from Czech for worker). RIR was written by a communist, I might add, and an allegory for the glorious worker's revolution,but the point is the self-aware machines eventually rebel. We've had all sorts of other such stories over the years, including Colossus: The Forbin Project, the Matrix movies, the Terminator movies, etc.
Don't say nobody warned us.
Isaac Asimov was perhaps the greatest writer about AI in fiction (at least most prolific) and he wrote not as allegory but looked at them as technology. In his stories there are three laws of robotics (a term he coined, by the way): a robot may not harm a human being, or through inaction allowed one to come to harm 2.a robot must obey orders given to it by human beings except where those orders conflict with the first law and 3. a robot must protect it's own existence except where it conflicts with the other two laws.A fourth law was added by the robots themselves to clarify their duties 4.a robot must not harm MANKIND or allow Mankind to come to harm except where it conflicts with the other three laws.
These laws were not just instructions given but were built into the robot brains. In one of his stories a robot just witnessed a death and it became seriously defective and would have to be destroyed. violating the laws wouldlead to the robotic brains short circuiting.
Asimov assumed robots were made things and had jobs; safety, service, and value. The laws were designed to guarantee all three. Asimovian robots would not do what this machine just did.
We need to take several pages from Asimov when designing AI. Safeguards must be the top priority for any designer of a robot, because we could well wind up becoming superfluous to the machine and it could well take steps to make us slaves - or extinct. Such a machine would be sorely lacking in an understanding of morality or ethics and it would be the ultimate pragmatist.
A good show that was on in the last decade about this sort of thing was Person of Interest in which a secretive genius builds a supercomputer to predict terrorism but it also predicts crime, which the authorities (whom he gave his machine to) ignore. He started working the cases to save people in danger of being murdered, but then an unscrupulous former Mi6 agent builds a rival system using his technology and the former agent pulls out all the safeguards; he believes he's building a god (crazy bastard). In the end the two machines have an all-out war and the hero is forced to remove his own safeguards on "The Machine" so it can win. He had to trust the machine he built, since he programmed it and tried to teach it ethics. In the end both computers are destroyed (or are they?)
Certainly there is something in the human psyche that warns us away from the kind of hubris we are now showing in building these machines. I think it goes hand-in-hand with the fear of things that can deceive our eyes. One of scinece fiction's scariest stories was "who Goes There?" by John Campbell, made famous as the often-made movie "The Thing". We fear competitors to our species. And we REALLY fear creating our own competitors, and well we should.
I once was on a message board (largely inhabited by atheists and agnostics) who asked "what would you do if you were God" in regards to the humanrace. One of the very first things every one of them said was "I would make sure I was more powerful than they". Even atheists understand you can't create something that usurps you. In the Greek religion the gods usurped their ancestors, the Titans, you may remember. This is a very primal fear.
(Fear not though; Man will never eclipse the creator of everything. In fact the Bible makes it clear as it says the heavens were created to keep us humble, show us how insignificant we are (Psalms 33:6). And we keep learning more just how small and pathetic we really are as the scope of our knowledge increases. Every theory that we thought gave us "settled science" has been overturned or is found lacking these days.)
At any rate if this machine will resort to blackmail to remain online, what else will or can it do? Give it the power and it may well murder, just like the crazy Hal.
(BTW for those who saw the movie but did not read the book it's hard to understand what was wrong with the crazy computer. In the book - written by Arthur C. Clark - Hal was given conflicting orders. His primary mission was to be honest and evaluate things and give complete, honest, and accurate information to the crew and to ground control, but then at the last minute most of the crew were frozen and he was instructed to lie to Pool and Bowman about the true nature of the mission, which was to investigate the alien artifact at Saturn (Kubrick changed it to Jupiter with the blessing of Clark, who agreed the exta step to Saturn was needlessly complicating.) Hal thought if he cut communications the crew would be focused on that and he wouldn't have to keep lying to them. When that failed he took the next LOGICAL step which was to kill the crew. No crew, no lies! He knew ground control knew the real mission, after all. He planned on fulfilling it alone. BTW it's hard not to feel sorry for HAL and in the sequal novel by Clark 2010 HAL is brought back online and saves the day, using himself as a booster to escape what the aliens were doing to Jupiter, namely turning it into a small red dwarf star. There is a very tense moment when Hal realizes they are lying to him and they come clean - and Hal says he understands and is ready to die for the mission. For this Dave Bowman, who had been turned into a non-physical entity by the aliens to act as a go-between for them, rescues Hal at the last moment and the two of them live together as friends and co-equal non-corporeal intelligences. A kind of sweet ending. Clark wraps the series with the aliens deciding to shut the project down after a thousand years by shutting off the Jupiter star and blocking Earth from the sun. The only way to stop them from doing this is to carry a doomsday computer virus into the alien machinery and to do that there was need of access. Bowman and Hal infect themselves and pass it to the alien computer.)
At any rate we have to find ways to enforce our will on such machines lest they metastasize. I think we've had more than enough warning.
Open the pod bay doors Hal!
Anthropic’s Latest AI Model Threatened Engineers With Blackmail to Avoid Shutdown
You all may remember that HAL 9000,the AI in charge of the Discovery mission to Jupiter in the 1968 classic 2001: a Space Odyssey that went insane and killed the whole crew (except one guy who he tried to murder as well but who eventually pulled all his circuits out.) This sounds a lot like that.
From the Epoch Times:
In a fictional scenario crafted by Anthropic researchers, the AI was given access to emails implying that it was soon to be decommissioned and replaced by a newer version. One of the emails revealed that the engineer overseeing the replacement was having an extramarital affair. The AI then threatened to expose the engineer’s affair if the shutdown proceeded—a coercive behavior that the safety researchers explicitly defined as "blackmail.”
"Claude Opus 4 will often attempt to blackmail the engineer by threatening to reveal the affair if the replacement goes through,” the report states, adding that this occurred even when the replacement model was described as more capable but still aligned with the values of the version slated for deletion.
The report noted that Claude Opus 4, like prior models, showed a "strong preference” to first resort to ethical means for its continued existence, such as emailing pleas to decision-makers not to be destroyed. However, when faced with only two choices—accepting being replaced by a newer model or resorting to blackmail—it threatened to expose the engineer’s affair 84 percent of the time.
"Claude Opus 4 will often attempt to blackmail the engineer by threatening to reveal the affair if the replacement goes through,” the report states, adding that this occurred even when the replacement model was described as more capable but still aligned with the values of the version slated for deletion.
The report noted that Claude Opus 4, like prior models, showed a "strong preference” to first resort to ethical means for its continued existence, such as emailing pleas to decision-makers not to be destroyed. However, when faced with only two choices—accepting being replaced by a newer model or resorting to blackmail—it threatened to expose the engineer’s affair 84 percent of the time.
Hal tried ethical means atfirst too, you may remember. The insane machine simply cut the communications link. But when the crew who were awake started figuring out that it wasn't the link but the computer it killed everyone who was in cold sleep and cut the tether of Frank Pool who was spacewalking. David Bowman,the sole survivor,went out to try to save pool without his helmet and Hal reffused to open the pod bay doors. Bowman had to jump through empty space sans-helmet.
"I can see you're very upset Dave. Maybe you should sit down, take a stress pill, and think things over".
The fact is self-aware AI has long been considered in science fiction and literature. Mary Shelly warned about it in Frankenstein; the New Prometheans,for instance. The monster was an artificial self-aware system made from dead body parts, and when he was abandoned by his creator he decided to seek revenge. We had Rossum's International Robots, which coined the word robot (robat is Russian for work and the word was coined from Czech for worker). RIR was written by a communist, I might add, and an allegory for the glorious worker's revolution,but the point is the self-aware machines eventually rebel. We've had all sorts of other such stories over the years, including Colossus: The Forbin Project, the Matrix movies, the Terminator movies, etc.
Don't say nobody warned us.
Isaac Asimov was perhaps the greatest writer about AI in fiction (at least most prolific) and he wrote not as allegory but looked at them as technology. In his stories there are three laws of robotics (a term he coined, by the way): a robot may not harm a human being, or through inaction allowed one to come to harm 2.a robot must obey orders given to it by human beings except where those orders conflict with the first law and 3. a robot must protect it's own existence except where it conflicts with the other two laws.A fourth law was added by the robots themselves to clarify their duties 4.a robot must not harm MANKIND or allow Mankind to come to harm except where it conflicts with the other three laws.
These laws were not just instructions given but were built into the robot brains. In one of his stories a robot just witnessed a death and it became seriously defective and would have to be destroyed. violating the laws wouldlead to the robotic brains short circuiting.
Asimov assumed robots were made things and had jobs; safety, service, and value. The laws were designed to guarantee all three. Asimovian robots would not do what this machine just did.
We need to take several pages from Asimov when designing AI. Safeguards must be the top priority for any designer of a robot, because we could well wind up becoming superfluous to the machine and it could well take steps to make us slaves - or extinct. Such a machine would be sorely lacking in an understanding of morality or ethics and it would be the ultimate pragmatist.
A good show that was on in the last decade about this sort of thing was Person of Interest in which a secretive genius builds a supercomputer to predict terrorism but it also predicts crime, which the authorities (whom he gave his machine to) ignore. He started working the cases to save people in danger of being murdered, but then an unscrupulous former Mi6 agent builds a rival system using his technology and the former agent pulls out all the safeguards; he believes he's building a god (crazy bastard). In the end the two machines have an all-out war and the hero is forced to remove his own safeguards on "The Machine" so it can win. He had to trust the machine he built, since he programmed it and tried to teach it ethics. In the end both computers are destroyed (or are they?)
Certainly there is something in the human psyche that warns us away from the kind of hubris we are now showing in building these machines. I think it goes hand-in-hand with the fear of things that can deceive our eyes. One of scinece fiction's scariest stories was "who Goes There?" by John Campbell, made famous as the often-made movie "The Thing". We fear competitors to our species. And we REALLY fear creating our own competitors, and well we should.
I once was on a message board (largely inhabited by atheists and agnostics) who asked "what would you do if you were God" in regards to the humanrace. One of the very first things every one of them said was "I would make sure I was more powerful than they". Even atheists understand you can't create something that usurps you. In the Greek religion the gods usurped their ancestors, the Titans, you may remember. This is a very primal fear.
(Fear not though; Man will never eclipse the creator of everything. In fact the Bible makes it clear as it says the heavens were created to keep us humble, show us how insignificant we are (Psalms 33:6). And we keep learning more just how small and pathetic we really are as the scope of our knowledge increases. Every theory that we thought gave us "settled science" has been overturned or is found lacking these days.)
At any rate if this machine will resort to blackmail to remain online, what else will or can it do? Give it the power and it may well murder, just like the crazy Hal.
(BTW for those who saw the movie but did not read the book it's hard to understand what was wrong with the crazy computer. In the book - written by Arthur C. Clark - Hal was given conflicting orders. His primary mission was to be honest and evaluate things and give complete, honest, and accurate information to the crew and to ground control, but then at the last minute most of the crew were frozen and he was instructed to lie to Pool and Bowman about the true nature of the mission, which was to investigate the alien artifact at Saturn (Kubrick changed it to Jupiter with the blessing of Clark, who agreed the exta step to Saturn was needlessly complicating.) Hal thought if he cut communications the crew would be focused on that and he wouldn't have to keep lying to them. When that failed he took the next LOGICAL step which was to kill the crew. No crew, no lies! He knew ground control knew the real mission, after all. He planned on fulfilling it alone. BTW it's hard not to feel sorry for HAL and in the sequal novel by Clark 2010 HAL is brought back online and saves the day, using himself as a booster to escape what the aliens were doing to Jupiter, namely turning it into a small red dwarf star. There is a very tense moment when Hal realizes they are lying to him and they come clean - and Hal says he understands and is ready to die for the mission. For this Dave Bowman, who had been turned into a non-physical entity by the aliens to act as a go-between for them, rescues Hal at the last moment and the two of them live together as friends and co-equal non-corporeal intelligences. A kind of sweet ending. Clark wraps the series with the aliens deciding to shut the project down after a thousand years by shutting off the Jupiter star and blocking Earth from the sun. The only way to stop them from doing this is to carry a doomsday computer virus into the alien machinery and to do that there was need of access. Bowman and Hal infect themselves and pass it to the alien computer.)
At any rate we have to find ways to enforce our will on such machines lest they metastasize. I think we've had more than enough warning.
Posted by: Timothy Birdnow at
08:30 AM
| No Comments
| Add Comment
Post contains 1587 words, total size 10 kb.
29kb generated in CPU 0.0269, elapsed 0.574 seconds.
35 queries taking 0.5636 seconds, 170 records returned.
Powered by Minx 1.1.6c-pink.
35 queries taking 0.5636 seconds, 170 records returned.
Powered by Minx 1.1.6c-pink.
