1.2 Trillion Transistors on a Wafer-Scale AI Chip

The Cerebras Wafer-Scale Engine (WSE) is the largest chip ever built and has one 1.2 trillion transistors. It is the heart of a deep learning system.

It is 56x larger than any other chip. It delivers more compute, more memory, and more communication bandwidth. This enables AI research at previously-impossible speeds and scale.

The Cerebras Wafer Scale Engine 46,225 square millimeters with 1.2 Trillion transistors and 400,000 AI-optimized cores.

By comparison, the largest Graphics Processing Unit is 815 square millimeters and has 21.1 Billion transistors.

Andrew Feldman and the Cerebras team have built the wafer-scale integrated chip. They have successfully solved issues of yield, power delivery, cross-reticle connectivity, packaging, and more. It has a 1,000x performance improvement over what’s currently available. It also contains 3,000 times more high speed, on-chip memory, and has 10,000 times more memory bandwidth.

It has a complex system of water-cooling. It uses an irrigation network to counteract the extreme heat generated by a chip running at 15 kilowatts of power.

The WSE has 18 GB of on-chip memory, all accessible within a single clock cycle, and provides 9 PB/s memory bandwidth. This is 3000x more capacity and 10,000x greater bandwidth than the leading competitor. More cores, more local memory enables fast, flexible computation, at lower latency and with less energy.

AI-optimized cores are connected entirely on silicon by the Swarm fabric in a 2D mesh with 100 Petabits per second of bandwidth. Swarm delivers breakthrough bandwidth and low latency at a fraction of the power draw of traditional techniques used to cluster graphics processing units. Software configures all the cores on the WSE to support the precise communication required for training user-specified models.

There is a 9 page white paper from Cerebras on the chip.

SOURCES- Cerebras
Written By Alvin Wang, Nextbigfuture.com