Scan and match
1) The system is shown a poor-quality image of a helicopter moving across a screen. It’s read by low-level nodes that each see a 4 x 4-pixel section of the image.
2) The low-level nodes pass the pattern they see up to the next level.
3) Intermediate nodes aggregate input from the low-level nodes to form shapes.
4) The top-level node compares the shapes against a library of objects and selects the best match.
Predict and refine
5) That info is passed back down to the intermediate – level nodes so they can better predict what shape they’ll see next.
6) Data from higher-up nodes allows the bottom nodes to clean up the image by ignoring pixels that don’t match the expected pattern (indicated above by an X). This entire process repeats until the image is crisp.
Dileep George built an original demonstration program, a basic representation of the process used in the human visual cortex. Most modeling programs are linear; they process data and make calculations in one direction. But George designed multiple, parallel layers of nodes — each representing thousands of neurons in cortical columns and each a small program with its own ability to process information, remember patterns, and make predictions.
George and Hawkins called the new technology hierarchical temporal memory, or HTM. An HTM consists of a pyramid of nodes, each encoded with a set of statistical formulas. The whole HTM is pointed at a data set, and the nodes create representations of the world the data describes — whether a series of pictures or the temperature fluctuations of a river. The temporal label reflects the fact that in order to learn, an HTM has to be fed information with a time component — say, pictures moving across a screen or temperatures rising and falling over a week. Just as with the brain, the easiest way for an HTM to learn to identify an object is by recognizing that its elements — the four legs of a dog, the lines of a letter in the alphabet — are consistently found in similar arrangements. Other than that, an HTM is agnostic; it can form a model of just about any set of data it’s exposed to. And, just as your cortex can combine sound with vision to confirm that you are seeing a dog instead of a fox, HTMs can also be hooked together. Most important, Hawkins says, an HTM can do what humans start doing from birth but that computers never have: not just learn, but generalize.
An HTM trained to identify helicopters from picture can be fed images it has never seen before, images of highly distorted helicopters oriented in various directions. To human eyes, each was still easily recognizable. Computers, however, haven’t traditionally been able to handle such deviations from what they’ve been programmed to detect, which is why spambots are foiled by strings of fuzzy letters that humans easily type in. George clicked on a picture, and after a few seconds the program spit out the correct identification: helicopter. It also cleaned up the image, just as our visual cortex does when it turns the messy data arriving from our retinas into clear images in our mind. The HTM even seems to handle optical illusions much like the human cortex. When George showed his HTM a capital A without its central horizontal line, the software filled in the missing information, just as our brains would.
Numenta is being applied to help monitor the sensors for air traffic control and il platforms. There are seeing good results with speed improvements over traditional approaches and correct identification of high risk situations. There is no degradation with more information as is the case with some AI systems.