machines who would follow any order incapable of disobeying
This isn’t as simple as it might seem. As the complexity of the intelligence scales, it’s not so simple to constrain the decision making of the intelligence.
And because of the rapid improvements in eval awareness, it’s also not so simple to screen out more subversive behaviors.
The same edge cases that leads to deleting drives and production databases with the newest transformers is going to start to occur in military applications if they scale out complex model intelligence.
This isn’t as simple as it might seem. As the complexity of the intelligence scales, it’s not so simple to constrain the decision making of the intelligence.
And because of the rapid improvements in eval awareness, it’s also not so simple to screen out more subversive behaviors.
The same edge cases that leads to deleting drives and production databases with the newest transformers is going to start to occur in military applications if they scale out complex model intelligence.