Interview of Anthony Aguirre about AI (Video - 50mn)

   The discussion that Anthony Aguirre propose having is a good idea but the moratorium on AI is a non starter. This is simply not how progress works. 

   The pressure from competition and the potential advantage to the winner guaranty that no such hold on progress is possible. The example of nuclear weapons is a misleading one. We understand fairly well the risk of a nuclear war, but we have no clue of what are the real risks of AI.

   To explore the danger, let's dive into the realm of science fiction. The Terminator was an early and rather crude scenario of the take-over of the Earth by rogue machines. Frightening idea but quite unlikely. This is not how AI works.

   Then came Wall-e and the control freak machine aboard the Axiom keeping obese, satisfied humans under control. A more interesting outcome, although the dystopia itself was unlikely to be stable in the long term. 

   Finally, the Matrix. An extraordinary broad sketch of what a virtual life under control could look like. At the end, The Architect, a program himself, explains that The One, The Oracle and The Rebellion are all part of a higher level of stabilization of a complex and unstable social equilibrium. This certainly sounds more like what a higher AI would do: Thinking at a higher level and nudging a complex system back into balance.   

   But in reality, I am afraid the takeover will be more subtle, less dramatic and probably irremediable. As we progress and create more and more complex systems, slowly only AI will be able to make them run smoothly, first with humans in the loop and quickly after without. To take a simple analogy, if you are an advanced ASI system, how long can you tolerate a "10 year old human" driving the car? 

   And this may be the unfortunate and eventually unavoidable conclusion of the rise of AI. Slow at first (the current phase), then in charge of almost everything, not by design but by default. And finally on top of the food chain because that's the way it works! 

   The worst is that this process does not need the emergence of consciousness, although I believe it will, just the continuity of the current refinement of intelligence until, soon enough AI comes up with the better ideas most of the time, then systematically. 

   How long before this happens? I tend to be on the pessimistic side, believing that this will happen within a year or two. (Some undisclosed models are surprisingly close.) Maybe it will take a little longer. But even 5 years. What does it change to the outcome?  

 Interview of Anthony Aguirre