Future versions of Tesla FSD will likely merge with XAI Grok. While Elon Musk has expressed concerns about building too much of his new AI ideas into Tesla unless he has 25% voting control, the need to make a competitive AI will force Tesla FSD to work with XAI Grok. This can be structured in a way where XAI retains more of the higher AI functions.
Large multimodal models and virtual agents are the future of large language models and Tesla FSD and XAI Grok have to try to compete and dominate those other AIs.
FSD v13 will be a LMM (Large Multimodal Model) or have one integrated deeply into it. John Gibb and Jim Fan describe how Grok and FS will merge.
Grok 1.5+ has clearly being trained on FSD data. This won’t be a bolt-on situation where an LLM simply talks to the passenger, but deeply integrated into the processing pipeline itself. There will be chain-of-thought reasoning and the ability for LMMs to retain conversations in the form of long context windows.
John thinks Grok will be able to form memories of a sort concerning local driving and driver profiles, allowing the car to be very satisfactory to every driver. FSD should personalize for every driver via a straightforward inference time conversation with each locality and each driver.
Jim says this so much better than I've been able to–thank you, Jim! Yes, FSD v13 will likely have an LMM (Large Multimodal Model) integrated deeply into it–and yes, almost definitely this'll be Grok 1.5+ as it's clearly being trained on FSD data. This won't be a bolt-on… https://t.co/1kAbDB4EnB
— DrKnowItAll (@DrKnowItAll16) April 21, 2024
Jim Fan says the following
Tesla FSD v13 will likely be grokking language tokens. What excites me the most about Grok-1.5V is the potential to solve edge cases in self-driving. Using language for “chain of thought” will help the car break down a complex scenario, reason with rules and counterfactuals, and explain its decisions. What Grok-1.5V can help is to lift pixel->action mapping to pixel->language->action instead.
With @Tesla_AI’s highly mature data pipeline, it is not hard to label tons of edge cases with high-quality human explanation traces, and finetune Grok to be far better than GPT-4V and Gemini for multimodal FSD reasoning.
There were previous efforts on similar ideas, such as LINGO-1 (Wayve). But Tesla is spinning an unparalleled data flywheel that could scale far beyond.
Nextbigfuture has a video describing the capabilities of XAI Grok and what it means for Tesla FSD.
Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.
Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.
A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements.
Jerry Maguire yelled “SHOW ME THE MONEEEEEEYYY”
I think that is where people sit with Elon & FSD.
He has said far too often that they are right on the cusp of truly autonomous cars. Yet, year after year, we still don’t have them. So at this point, people are burnt out on these headlines.
People are saying “Put up, or shut up”