Super Venture Capitalists Bill Gurley and Brad Gerstner analyze the future of AI. The rate of improvement of large language models is slowing for pre-training. However, it is still improving and AI intelligence is improving with post-training and test time training. OpenAI and Microsoft announced a major AI advancement for 2025, featuring models with infinite …
New Linear-complexity Multiplication (L-Mul) algorithm claims it can reduce energy costs by 95% for element-wise tensor multiplications and 80% for dot products in large language models. It maintains or even improving precision compared to 8-bit floating point operations. By replacing complex floating-point multiplication with integer addition Solution in this Paper – Approximates floating-point multiplication using …
Bloomberg reports OpenAI is in talks to raise a $6.5 billion round at over a $150 billion valuation. They are also talking to banks about a $5 billion credit line. The latest funding round, led by Thrive Capital, gives OpenAI a significantly higher valuation than its $86 billion appraisal earlier this year. Apple and Nvidia …
Facebook has ~10x the proprietary language data on database as was used to train the LLaMa models. In images Facebook have 20x more than that. Instagram and Youtube have 2x more that in uploaded video. Tesla’s data capture-ability dwarfs all (at 20x more again) than Youtube. Size matters Facebook has ~10x the proprietary language data …
Anthropic CEO talks about AI scaling laws which is a discussion about how good AI gets with more compute training. Dario Amodei, CEO and Co-founder of Anthropic. Dario talks about the economics of AI development, the comparative advantage of AI companies like Anthropic, AI safety, and his stance on California’s SB 1047 bill. They also …
Chamath has the vision of rebuilding the $1.5 trillion global software stack with many lean 10-20 person companies powered with AI. His vision is to have the team create the product requirements document in extreme detail and feeding it into an AI to build it. This will start working over the next year or so. …
If Large Language Models debate their answers they can reach better answers. A complementary approach to improve language responses where multiple language model instances propose and debate their individual responses and reasoning processes over multiple rounds to arrive at a common final answer. The findings indicate that this approach significantly enhances mathematical and strategic reasoning …
OpenAI’s rumored project Strawberry is focused on enhancing the reasoning capabilities of its AI models. Strawberry is rumored to be based upon the QStar AI advance. QStar was a reasoning advance that supposedly triggered the power struggle inside OpenAI and temporary firing of CEO Sam Altman. Strawberry supposedly enables AI to perform long-horizon tasks (LHT), …
AI large language models have been especially weak on math. There are now several papers from Google Deep Mind, Alibaba and other universities where AI large language models are at Math Olympiad levels and multiple step reasoning even with small models. It's finally here. Q* rings true. Tiny LLMs are as good at math as …
ChatRTX is a demo app that lets you personalize a GPT large language model (LLM) connected to your own content—docs, notes, images, or other data. Leveraging retrieval-augmented generation (RAG), TensorRT-LLM, and RTX acceleration, you can query a custom chatbot to quickly get contextually relevant answers. It all runs locally on your Windows RTX PC or …
x-Tesla AI lead, Andrej Karpathy gave a one hour general-audience introduction to Large Language Models. The core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm. Context: This video is …