Managers are finding AI projects difficult to implement and the results are disappointing.
* Getting and fixing the data to train AI is time-consuming, expensive and the data may not be good enough
What works
* understanding X-ray and MRI images. It is straight forward to correlate particular images with a disease or non-disease result.
* car driving. Again many images and videos are understood from 1 million+ Tesla cars to get to desired results
What seems to be missing
* operating when a lot of data is missing
* translating pattern matching to knowledge graphs and building context and actual understanding or pseudo understanding
* being able to properly generalize learnings
* having a model of the world and reality that sanity checks results
Other critiques say:
A serious challenge is how to develop algorithms that can deal with the combinatorial explosion as researchers address increasingly complex visual tasks in increasingly realistic conditions. Although Deep Nets will surely be one part of the solution, we believe that we will also need complementary approaches involving compositional principles and causal models that capture the underlying structures of the data.
Can all of the problems be solved without the systems failing to significantly improve or getting worse under complexity of trying to overcome problems?
Transferring Learnings to Other Applicable Domains and Continual Learning
Geoffrey Hinton (godfather of deep learning) and Demis Hassabis (Deep Mind) have indicated that AGI is far away. There are many fundamental issues to generalize deep learning and reinforcement learning. There are issues transferring skills between systems and being able to amplify weak reinforcement signals.
A way to solve reinforcement learning’s scalability problem is to amplify the reinforcment signal with hierarchical architecture. This would create a system of sub-goals.
There are limitations with deep learning vision systems.
A typical neural network will forget the last thing it was trained to do. Virtually all neural networks today suffer from catastrophic forgetting.
In 2017, there was a paper called Learning to Continually Learn.
In 2020, Open AI’s Jeff Clune shared ANML (a neuromodulated meta-learning algorithm), which is able to learn 600 sequential tasks with minimal catastrophic forgetting. Clune believes AI like ANML is key to achieving a faster path to the greatest challenge: artificial general intelligence (AGI).
He argued a faster path to AGI can be achieved through improving meta-learning algorithm architectures, the algorithms themselves, and the automatic generation of training environments.
ANML scales catastrophic forgetting reduction to a deep learning model with 6 million parameters.
Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.
Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.
A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements.
Yeah there have been articles written on how AI is “biased”. It’s not biased, based on the available data, it comes to a statistical conclusion. Just because you don’t like the results doesn’t make the AI biased.
A colleague of mine, a protege of John Tukey, cofounded the DoE’s Energy Information Administration and originally called it “Office of Energy Information Validation” because, according to him, “Over 90% of the expense of establishing the EIA was in data cleaning.” He went on to end the first neural network winter with grants to Werbos, Hinton, Rumelhart, MccLelland, etc. when he took over the Systems Development Foundation. This he did, in part, because of the need for better statistical methods to deal with poor data quality, but also (particularly w/re Werbos) the need for dynamical systems identification via recurrent nets. It took me about a decade to convince him that lossless compression was the correct model selection criterion (nowadays being gradually rediscovered as “self-supervised learning”), but by that time he was semi-retired and pretty much forgotten by the AI industry he helped found.
Here’s an excerpt of an email regarding COVID-19 forecasting I sent to a colleague at Oxford, which I Cc’d to him:
“…Sadly, even statisticians usually overlook the fact that the hard job of data cleaning is on a continuum of modeling. This is made even more tragic by the fact that so much work is put into algorithms to clean data, which should provide a clue. Measurement instruments must be modeled if one is to trust the measurements…”
In short, lossless compression can be applied to _raw_ data to subsume data cleaning and imputation of missing data.
The data is a problem because current networks need such an astounding amount of it. Solve the architecture problem and you don’t need that kind of amount of data and ANN becomes much more useful.
You could of course just not have race and gender as an input to the system. Purge the resumes of this information before training the ANN.. But, most likely, the system would home in on qualities such as worked hours, awards, patents a.s.o. and the result would not be “correct”.
On an average, the “correct” answer for Harvard is that asians are less “courageous” and therefore less merited of entry. You have to laugh about that one… Asian americans had figured out the value of extra curricular activities (complete BS, if you ask me) to complement their higher grades, so there was no other recourse than to find their personalities lacking…
They are probably trying to ‘fix’ it.
That prospect scares me the most. How to fix algorithms to perform according to social norms rather than making decisions based on scientific data. Even after they have taken ethnic considerations out of the equation they will continue to pollute the system with weird weightings in order to acheive social justice (from their perspective).
Open source it all.
True. Remember when they trained an ANN to predict who to hire, and it thought that males were a better prospect for hiring? They chucked that ANN out right away…
Not to mention that what is PC today will be NPC next year (sometimes month) and you’ll need to start from scratch.
Even a lot of tech companies have evidence of bad management. Big companies are even worse. I have worked for one of the leading software development companies. The software they used for their own business processes was poor and outdated.
Curve fitting, today’s AI is just a lazy man’s expert system.
Fair point.
Most companies are not high tech companies. Success is maintained via inertia and structure vs innovation or individual competence.
You must have worked on pretty terrible companies then.
In my experience, most high tech companies have reasonably competent people as managers.
“Getting and fixing the data to train AI is time-consuming, expensive …”
If the data used for training is “fixed”, as implied by the above quote, it seems to me that would lessen the chance that the AI would work properly when fed live, un-fixed data. Perhaps that quote is misleading and the data used for training is not all “fixed”. I have no experience in developing AI implementations, so I don’t know how training is done in practice.
Wrong approach to General AI. True Gen AI is creative, intense, and does not develop through rote learning or problem convergence – these are Intern tasks and you need to send your Liberal Arts grad student to this stuff. True AGI needs only to be provided with an open-ended thesis, a set of data sources (not sets), and previous conclusions to compare and ‘bounce’ ideas off of. True AGI is a mis-nomer – it pretends that an AI can take on any field, any idea, any set of problems and then expand out the solutions. Nonsense. You have to bound the start point but unleash the end results – it is the divergence that is the General, not the All-Purpose, any-time use that is proposed. A Ph.D. is only as good as their field, the thinking process and a priori knowledge in that field. Such as it is with AGI, it is but a grad student with inf+ processing; and no substance-abuse, anti-social characteristics, or publish-glory tendencies; it provides thesis-conclusions beyond what your 4-yr Ph.Ds bring. It is ideal for such as pharmacology where the solutions and possibilities are immense compared to the original theses and data sources. It will be both underwhelming and overwhelming. It will not save the world’s problems but simply fill in huge gaps that endless armies of Grad students would have otherwise spent lifetime-equivalents doing. AGI will never be Einstein, only a million-billion Sheldons increasing knowledge’s breadth before they increase knowledge’s forward thrusts.
AI isn’t magic. You have a bunch of problems for which you have poorly defined inputs and outputs to correlate or none at all, and want AI to magically look at whatever random data you have and produce intelligible responses?
Not a chance. First define clearly your problem domain, your valid data input(s) and then train the AI to find the patterns you want.
Alas, that isn’t like running Word and making a memo complaining.
Same for neural nets forgetting. That’s why you train them to recognize something very well and them leave them at a fixed state so it doesn’t forget what it was supposed to know.
Most managers I’ve been around don’t have an original thought in their brain. So it’s not surprising that they are finding it difficult to implement AI. It’s not as much a problem with the technology as it is the people using the technology. Of course AI is going to be expensive to implement at first. It’s the same with most investments. AI is however very different from most other investments in one way: it continually gets better. Once you learn how to do a task with AI, you then don’t forget how to do that task with AI. Once you implement an AI for one task in your business, you can keep developing it for more and more use cases.
Prediction: Filtering the data for PC content will result in unsatisfactory results and impede progress.