xAI, is expanding the Colossus supercomputer in Memphis, Tennessee from 100,000 to over 1 million GPUs. This expansion, which has already started, involves a substantial financial commitment, potentially in the tens of billions, and includes collaborations with Nvidia, Dell, and Supermicro for GPU supply and server assembly. xAI’s strategy is to lead in the AI industry by surpassing competitors like Google, OpenAI, and Anthropic. This development not only boosts xAI’s computational power but also positions Memphis as an emerging center for AI technology.
xAI should have 200k H100s and H200s this month.
xAI is buying the first 100,000 B200s in January, 2025 and will have 300,000 B200s by April.
1 Million B200s and Dojo 2 (B200 equivalents) should be ready by the end of 2025.
xAI will shift to Dojo 3 and Nvidia Rubin chips in 2026.
There are rumors the Nvidia Rubin chips could arrive by Q4 of 2025.
Rumor: Nvidia’s Rubin platform could be available 6-months ahead of schedule, media report, as the AI chip giant is already working with supply chain partners in Taiwan on the successor to Blackwell. Nvidia is expected to use TSMC’s 3nm manufacturing process for Rubin, originally…
— Dan Nystedt (@dnystedt) December 4, 2024
Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.
Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.
A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements.
So, by the end of 2026 we will be facing a cluster computation capability of 30m years chip time per calendar year, without hybrid FPGA equivalent considerations that get interesting with scale.
Current backprop methods are hugely (magnitudes) inefficient compared to biology within the thalamus (closest biological equivalent to current networks), which are very regionally focused adaptations.
Should a new methodology for training (e.g. ranvier temporal – kinesin/dynactin release based learnning) be worked out that is more efficient, it would be interesting as to just how quickly a new AI would evolve with the massive compute resource already lined up. Separately the cortex architecture lends to a different compute strategy (more bandwidth than compute) to the thalamus.
Compute clusters may well already be at a point where they are far (magnitudes) more computationally capable than an AGI would require in order to function.
How far will will humanity get to in building compute clusters before things change even more quickly…..