General Discussion

https://www.cirsd.org/en/horizons/horizons-spring-2025--issue-no-30/why-agi-should-be-the-worlds-top-priority

They make the point about how critical it is to regulate before it (inevitably) gets out of human control because the third stage is a new entity that it more intelligent than the whole of humanity combined. The problem is all the governments are competing to consolidate power around it and not stopping to think about the ethical implications.

I kind of think it may go the way of all new technological improvements and be used mainly for porn and spam advertising. It would be great if it solved all diseases and reversed aging though.

I hadn't twigged that most of Silicon Valley thinks within the next 5 years we will have an autonomous entity that surpasses us. Surely there must be options though about what we do or don't hook up to it. Whenever I try to imagine the more dystopian scenarios I have to remember that at some point you must be able to just unplug or disconnect it.

This doesn't sound promising though:
"With many frontier forms of AI already showing deceptive and self-preserving behavior, and the push toward more autonomous, interacting, self-improving AIs integrated with infrastructures, the impacts and trajectory of AGI can plausibly end up being uncontrollable. If that happens, there may be no way to return to a state of reliable human oversight."