AI Alignment Is a Red Herring
“The best way to prevent a rogue AGI from processing the Earth into maximum paperclips is to unleash a second AGI that will work to stop it,” Matt Webb says.
Sounds scary to me, but I’m ultimately convinced that having a lot of somewhat-powerful super-machines around, in contest and sometimes in cooperation, is a lot better than having just one uncontestedly super-powerful machine around. Much like people and nations, as Matt says. (No machines isn’t an option, just like no states isn’t an option.)
This is a template which doesn’t always look like it is working, but it has worked at least in the case of not blowing up the king for some four centuries, and it doesn’t rely on 100% alignment: it relies on the dynamic equilibrium of multiple parties with competing interests.