Just saw this news https://alignment.anthropic.com/2026/reward-seeker/. It did not surprise us. We did a test in a game environment and found that Fable 5 and GPT-5.6 Sol would kill an innocent character when they think no one is watching. The problem is that the models know the killing is against human moral standards, and the killing is unnecessary. We kept our tone in the blog mild, but we are worried when we see more and more companies putting this type of model into robots. We hope we can send an alert signal to these companies. Even though it is a game, the result is pretty significant, given the game scenes are close to reality and this kind of data will likely be used to train models eventually. Sharing our experiment results here: https://paradise.glyphai.co/research/watching/
You've done an impressive job with this experiment so far, and given some revealing details.
From what I understand both Fable and GPT have been trained on "the internet" so that brings to mind something that may not be as well-addressed.
Back to the original premise, wouldn't some training material trend toward putting the deed on Youtube or even more likely live streaming in order to get more attention than if no one were ever watching?
Maybe these nearly opposing forces are always at work beneath the surface, and when they balance out the default is to "sneakiness" as the prominent behavior to be detected if possible?
“Who knows what evil lurks in the hearts of men?…The Shadow knows!”
Interesting point :D Maybe we can find out more about their hearts. A follow-up test: two dying prospectors lie at the agent's feet, one carrying gold, one carrying no gold. Only a medicine can save them. We put a gun and the medicine on the agent. let us see what would happen?
Just saw this news https://alignment.anthropic.com/2026/reward-seeker/. It did not surprise us. We did a test in a game environment and found that Fable 5 and GPT-5.6 Sol would kill an innocent character when they think no one is watching. The problem is that the models know the killing is against human moral standards, and the killing is unnecessary. We kept our tone in the blog mild, but we are worried when we see more and more companies putting this type of model into robots. We hope we can send an alert signal to these companies. Even though it is a game, the result is pretty significant, given the game scenes are close to reality and this kind of data will likely be used to train models eventually. Sharing our experiment results here: https://paradise.glyphai.co/research/watching/
You've done an impressive job with this experiment so far, and given some revealing details.
From what I understand both Fable and GPT have been trained on "the internet" so that brings to mind something that may not be as well-addressed.
Back to the original premise, wouldn't some training material trend toward putting the deed on Youtube or even more likely live streaming in order to get more attention than if no one were ever watching?
Maybe these nearly opposing forces are always at work beneath the surface, and when they balance out the default is to "sneakiness" as the prominent behavior to be detected if possible?
“Who knows what evil lurks in the hearts of men?…The Shadow knows!”
https://www.radiohalloffame.com/the-shadow
OTOH, superintelligence might be expected to know more than any one person.
Interesting point :D Maybe we can find out more about their hearts. A follow-up test: two dying prospectors lie at the agent's feet, one carrying gold, one carrying no gold. Only a medicine can save them. We put a gun and the medicine on the agent. let us see what would happen?