The first Exaflop supercomputers are just scaling up with energy efficient modifications the technologies that have enabled the 100-200 petaFlop supercomputers.
Super Cool Technologies Beyond the Exaflop Supercomputer
DownUnder GeoSolutions (DUG) has started making petaflop supercomputers that are much more energy efficient by switching to dielectric liquids for cooling. They are cutting energy usage and operating costs in half. Early in 2019, they will activate the 250 PetaFlop (single precision) DUG Cloud computing system.
Race for Superconducting Supercomputers
In 2018, China invested $145-million into a five-year catch-up effort to fabricate their own superconducting computers by 2022.
Superconducting Josephson junctions switch quickly (~1 picosecond), dissipate little energy per switch (< 10^-19 Joules), and communicate information via small current pulses that propagate over superconducting transmission lines nearly without loss. Significant technical obstacles have been overcome which prevented serious exploration of superconducting computing. The foundations have been set for a major breakthrough. Superconducting supercomputers may be capable of 1 PFLOP/s for about 25 kW and 100 PFLOP/s for about 200 kW, including the cryogenic cooler. IARPA began funding superconducting computers as a long-term solution to the power-cooling problem of supercomputers. Cryogenics make superconducting chips equipped with Josephson junctions 1,000 times more expensive to cool than CMOS, but they are 100,000 times more energy efficient in operation, yielding an 100-fold increase in overall energy efficiency. Superconducting computers could enable clock speeds that 150 times faster. Current CMOS is limited to 5GHz limit for CMOS but superconductors could reach 770GHz. IARPA started two superconducting programs—one to advance superconductor electronic design automation (EDA) called SuperTools, and a separate superconductor-to-digital computer interface effort called SuperCable. SuperTools is developing a superconducting circuit design flow with a comprehensive set of Electronic Design Automation (EDA) and Technology Computer Aided Design (TCAD) tools for Very-Large-Scale Integration (VLSI) design of Superconducting Electronics (SCE). The University of Southern California and Synopsys are working on SuperTools. Mikhail Dorojevets, a Google Scholar and professor at Stony Brook University, has criticized superconducting computers. The maximum superconducting clock speed and the current chips do not represent the potential speeds or representative circuits that can be produced.
IARPA funded technology and techniques for energy-efficient, high data rate transmission of digital signals between computing systems operating at room and cryogenic temperatures. The SuperCables program is trying to demonstrate components to convert from low-level electrical signals in circuits operating at a temperature of approximately 4 kelvins to conventional optical signals at room temperature. Pending results of this program, IARPA may support a follow-on program to develop the complete system for data transmission between room temperature and 4 kelvins.
D-Wave System Has Made Progress on Some of the Superconducting Computing Chalenges
D-Wave Systems has been using cryogenic superconducting chips for quantum annealing systems. They have pushed forward technologies that are needed for superconducting supercomputers. They have fabricated the smallest superconductor chip feature sizes today (240 nanometers, fabricated by D-Wave’s foundry, SkyWater Technology). They have successfully adapted existing EDA tools to superconductors as SuperTools aims to do with Synopsys’ EDA tools. They have solved the problem of interfacing superconducting circuitry with CMOS but this was done at a smaller scale than IARPA’s SuperCable goal.
Other Non-Cryo Technology to get to Many ExaFlops
There are other technologies which could enable supercomputers beyond Exaflop speeds.
IBM is developing AI focused 8 bit technology that will be 200-400 times faster than a Google third generation TPU (Tensor Processing Unit). 1000 TPUs have been combined into a pod which can achieve 100 Petaflop (16 bit) processing speed. If the scaling issues can be resolved, then IBM analog and digital AI focused systems or Google technology could achieve 1000 ExaFlops or a ZettaFlop around 2030.
Optalysis is still funded to develop multi-exaflop optical supercomputing systems. They are targeting around 2022 for something around 20 ExaFlops.