AI LLM Improvement Rate Slows on Pre-training but Memory and Actions Make AI More Useful

Super Venture Capitalists Bill Gurley and Brad Gerstner analyze the future of AI. The rate of improvement of large language models is slowing for pre-training. However, it is still improving and AI intelligence is improving with post-training and test time training.

OpenAI and Microsoft announced a major AI advancement for 2025, featuring models with infinite memory and expanded context windows.

Infinite Context Windows

Infinite context in AI systems will enable long-term, meaningful relationships with users. AI will remember all of the previous questions. AI will have a better understanding of individual needs, preferences, and behaviors over time. This level of personalization has the potential to transform various sectors, including:

Healthcare: Providing personalized treatment plans based on comprehensive patient history
Education: Tailoring learning experiences to individual student progress and preferences
Customer Service: Offering highly personalized support by recalling past interactions and issues
Research and Development: Analyzing vast amounts of data to identify patterns and insights

3 thoughts on “AI LLM Improvement Rate Slows on Pre-training but Memory and Actions Make AI More Useful”

  1. Why would we ever want robots teaching us? Human consciousness shaped by technology known to be unreliable and quite easy to manipulate. Apparently we are trying to invert the intelligence hierarchy at full speed? When will actual intelligence get some cred back?

    • I look at this as being the foundations of basically every human being getting an accountabilibuddy. Basically, an ai/robot that goes with you through your whole life and looking out for you the whole time.

Comments are closed.