The singularity is near is a book by Ray Kurzweil. He makes a casr which I will summarize as very fast computers become more powerful than the power of the raw human brain. Reverse engineer the brain will progress to avoid the software coding problem for artificial intelligence. Therefore,
(2 [computers exceed in raw] + 2 [reverse engineer to get around software problem]) * 1000,000,000 [60 more years of Moore’s law or MNT) = Singularity
I do not doubt that we will have massively powerful computers.
I think we will get molecular nanotechnology.
I think we can then use it to complete the reserse engineering of the brain and make faster artificial neurons.
I am not sure how creatively productive and disruptive those systems will be or that the architecture will be one where there is just one general purpose artificial intelligence and that those artificial intelligences will not have people in the loop.
We currently have productivity enhancement and intelligence leveraging and intelligence and more efficient productivity pooling.
Systems such as Search engines, excel spreadsheets, narrow expert systems, supercomputer simulations etc… These systems help increase productivity and accelerate technological development and social change.
Specialized systems, written for a particular problem space by people who understand that problem space can have 1,000,000,000 times advantage or more over a general purpose system. I think we will see many of these super-specialized systems in the future and they will communicate and cooperate very well.
To someone from our vantage point they would look like one big super system, but from people and those systems point of view they will be many specialized systems. The difference will be in the dynamic of how they interact and the power structure. The one man renaissance AI versus the super symphony. The super symphony would still have a role for augmented humans. Plus the augmentation need not be borgification but more like the comic book Cerebro. Comic book references to the old brain helmets that augment the human mind. It has close but non-invasive communication between brain and machine.
Most singularity views are implicitly or explicitly assuming that other architectures are not as efficient or competitive. I do not think that will be case. I think many different approaches to augmentation and productivity will exist. Things like advanced software agents
A great deal of background and discussion at the Singularity Institute. I also think that there is a limit to rapidly an AI could exceed human ability to control it. So long as the systems upon they are based are vulnerable to electronic disruption (microwaves, e-bombs, etc…) and humans do not let AI systems improve their hardware without humans in the loop then a degree of control will be retained.