Since 1975, the number of transistors on a chip doubled every two years. From 1965 to 1975 the number of transistors doubled every year.
“Microprocessors no longer scale at the level of performance they used to — the end of what you would call Moore’s Law,” Huang said. “Semiconductor physics prevents us from taking Dennard scaling any further.”
Huang says new GPU architectures for graphics and AI will continue the increase in performance for GPUs. He predicts 1000X more performance by 2025.