Wolfram Alpha is like a calculator on steroids. It has a lot of science (math, physics and chemistry) formulas and knowledge encoded. Systems like Wolfram Alpha could provide an expert system grounding of hard answers to supplement ChatGPT like systems which are better at communicating and language understanding.
Having improved chat systems orchestrate the answers from many expert systems would mean there would be less errors from ChatGPT making up answers without a proper basis or verification.
Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.
Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.
A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements.
There’s already a couple of interesting if very limited ChatGPT plugins in the Chrome store that are somewhat useful – worth taking a look. Eg. a YouTube transcript summarizer – though usually only part of the transcript can fit into a prompt.
But the future of language model use probably lies in more complex and structured uses, where a LLM is given multiple prompts – e.g. having it generate web searches to collect factual data needed to satisfy user requests. To be economical, LLM requests/responses may need to cost ~1% of present.
Possibly the right approach will be to create compact language models (CLMs) that can run quickly on a personal device and embed those in an assistant app that does a good job of handling most user requests, but occasionally needs to use the full online LLM for part or all of a request.
I remember taking calculus in college. We were training with Wolfram Alpha quite a bit. I remember thinking how one person could invent such a thing. Stephen Wolfram is a true genius. ChatGPT will only increase the power of it by like a million times
ChatGPT and the like should be trained on scientific literature if they aren’t already.
I think GPT4 will bring a lot of surprises in logical and mathematical reasoning by itself, as GPT3 already did, despite its shortcomings.
Seems that educating a LLM using documented computer code, makes the overall model better and more correct overall. My hunch is the mathematical education of GPT3 wasn’t that good compared to its CS training, given the comparative difficulty of getting enough examples of solved and correctly documented mathematical problems (including its markup, which is a bit inconsistent across sources), versus computer code in sites like Stack Overflow, which already use natural and formal languages together, as a Rosetta stone.
Noticing that, they probably increased the number of solved mathematical problems in the training set, resulting in the LLM having a better grasp of the rules involved. They probably even used Mathematica or Wolfram to make and solve those problems and train GPT4.
StackExchange has sister sites on many topics, including mathematics, chemistry, physics, etc. They also use formal symbols together with natural language. Probably a smaller volume of text than Stack Overflow, but still many useful examples.
I have no doubt they used that as well.
My comment was more in the sense of comparative size of the input data. They probably found hundreds of different commented implementations of sorting or other basic algorithms, but probably a couple of readable references for some more esoteric mathematical proofs.
Wikipedia itself has a lot of commented math, and it’s more or less reliable, but it still is less than the computer code out there, which is the bread and butter of those building the Internet.
Also, this might be a matter of just adding more layers and parameters, which seems to be the current general strategy for better training, leveraging the increased abundance of computing resources at their disposal.