Tomorrow, I will have two new videos that will describe how xAI will use a data center with 80 times the training compute used for GROK 3 in 2026.
I have a far more refined, detailed analysis of the AI singularity and intelligence explosion as it is happening.
The AI is what we are witnessing and living through.
Training compute of frontier AI models was growing by 4-5x per year
Epoch expanded AI model database shows that the compute used to train recent models grew 4-5x yearly from 2010 to May 2024. They find similar growth in frontier models, recent large language models, and models from leading companies.
However, xAI could have a training compute growth pace that is 9-12X times per year and even more.
Within months xAI will have installed 200,000 Nvidia B200 GPUs to go with the 200,000 H100/H200s. This will be 11 times the training compute of the 100K H100s used for the Grok 3 pre-training.
The scaling laws are still holding for the pre-training and this is being enhanced post-training, reinforcement learning and test time training.
There is more than just the energy and hardware. There is growing the data with synthetic data, there is testing, efficiency and much more. There are ways to get 1000X more powerful and productive in the next 1-4 years beyond the raw enery and raw compute.
There is more to understand about the details of what will happen. 80X training compute in 2026 is the lower bound.
This will take many articles and videos and constant analysis to see how reality is matching up with what is possible. But I will identify where to look and how things could be done for maximum benefits.
It is not just which company wins a trillion dollar market by replacing Google search. Those are small potatoes.
— Elon Musk (@elonmusk) February 22, 2025
The expansion to 1.2 gigawatts of power and all of the new B200 chips and Dojo 2 chips will be installed by the end of this year.
Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.
Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.
A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements.
Grok is amazing I asked it about a combination drug treatment for a form of Muscular Dystrophy. It gave a complete analysis and then asked it to create a clinical trial and write a government grant. From what I know about the drugs and similar clinical trials it seemed very accurate. For actual researchers, Grok gives an incredible starting point and saves days of time.
Does anyone know if Grok was trained with scientific journal articles that are behind a paywall?
I tried to make conversation with grok 3 and it’s true that it’s very impressive. You feel like you’re talking to a human being.
In the past I’ve always had a lot of problems with AI and hallucination. When we pushed the AI to its limits, it was always hallucinating. It remains to be seen whether they’ve really managed to overcome this problem. Since they’re still LLMs, I don’t think they can solve the problem for good.
On the other hand, I use AI mainly for translation, and for the moment I find ChatGPT the most impressive. The translated sentences are even better than in the original!
Agree with you , complex scientific problems are out of reach by IA , I try use it for chemistry complex equilibrium and can’t answer yet or even do approximation to simplify … but I gain a lot of time for finding general knowledge in a sec