Along the way—progressing through Phase 0, Phase 1, Phase 2, and Phase 3—we have journeyed from neuroscience to supercomputing, to a new computer architecture, to a new programming language, to algorithms, applications, and now to a new chip—TrueNorth.
Fabricated in Samsung’s 28nm process, with 5.4 billion transistors, TrueNorth is IBM’s largest chip to date in transistor count. While simulating complex recurrent neural networks, TrueNorth consumes less than 100mW of power and has a power density of 20mW / cm2
Deep learning efforts today are run on standard computer hardware using convolutional neural networks. Indeed the approach has proven powerful by pioneers such as Google and Microsoft. In contrast neuromorphic computing, whose spiking neuron architecture more closely mimics human brain function, has generated less enthusiasm in the deep learning community. Now, work by IBM using its TrueNorth chip as a test case may bring deep learning to neuromorphic architectures.
Writing in the Proceedings of the National Academy of Science (PNAS) in August (Convolutional networks for fast, energy-efficient neuromorphic computing), researchers from IBM Research report, “[We] demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that approach state-of-the-art classification accuracy across eight standard datasets encompassing vision and speech, perform inference while preserving the hardware’s underlying energy-efficiency and high throughput.”
The impact could be significant as neuromorphic hardware and software technology have been rapidly advancing on several fronts. IBM researchers ran the datasets at between 1,200 and 2,600 frames/s and using between 25 and 275 mW (effectively >6,000 frames/s per watt). They report their approach allowed networks to be specified and trained using backpropagation with the same ease-of-use as contemporary deep learning. Basically, the new approach allows the algorithmic power of deep learning to be merged with the efficiency of neuromorphic processors.
“The new milestone provides a palpable proof of concept that the efficiency of brain-inspired computing can be merged with the effectiveness of deep learning, paving the path towards a new generation of chips and algorithms with even greater efficiency and effectiveness,” said Dharmendra Modha, chief scientist for brain-inspired computing at IBM Research-Almaden, in an interesting article by Jeremy Hsu on the IBM work posted this week on the IEEE Spectrum (IBM’s Brain-Inspired Chip Tested for Deep Learning.)
PNAS - Convolutional networks for fast, energy-efficient neuromorphic computing
Abstract - Convolutional networks for fast, energy-efficient neuromorphic computing
Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented energy-efficiency through a new chip architecture based on spiking neurons, low precision synapses, and a scalable communication network. Here, we demonstrate that neuromorphic computing, despite its novel architectural primitives, can implement deep convolution networks that
(i) approach state-of-the-art classification accuracy across eight standard datasets encompassing vision and speech,
(ii) perform inference while preserving the hardware’s underlying energy-efficiency and high throughput, running on the aforementioned datasets at between 1,200 and 2,600 frames/s and using between 25 and 275 mW (effectively over 6,000 frames/s per Watt), and
(iii) can be specified and trained using backpropagation with the same ease-of-use as contemporary deep learning. This approach allows the algorithmic power of deep learning to be merged with the efficiency of neuromorphic processors, bringing the promise of embedded, intelligent, brain-inspired computing one step closer.
A) Two layers of a convolutional network. Colors (green, purple, blue, orange) designate neurons (individual boxes) belonging to the same group (partitioning the feature dimension) at the same location (partitioning the spatial dimensions). (B) A TrueNorth chip (shown far right socketed in IBM’s NS1e board) comprises 4,096 cores, each with 256 inputs, 256 neurons, and a 256 ×× 256 synaptic array. Convolutional network neurons for one group at one topographic location are implemented using neurons on the same TrueNorth core (TrueNorth neuron colors correspond to convolutional network neuron colors in A), with their corresponding filter support region implemented using the core’s inputs, and filter weights implemented using the core’s synaptic array.
SOURCES- IBM, PNAS, HPC wire