A Synapse is a Memristor and Memcapacitors have no Resistance : New Era in AI and Electronics

Neuron cell architecture. (a) Despite having only four pins (discs), cell circuitry (b) can compute analog dot products of large numbers of input signals and synaptic weights, using summing amplifiers. Cell processing implements shunting dynamics. (c) Dendrites (horizontal nanowires) collect inputs from other neurons; axons (vertical nanowires) carry information to other neurons

New Scientist reports that Memristors could revolutionize artificial intelligence work. Other information and the pictures in this article are from a Scientific Discovery through Advanced computing article on a plan for a Memristor chip and a reference to an Arxiv article.

Williams and Snider have teamed up with Gail Carpenter and Stephen Grossberg at Boston University, who are pioneers in reducing neural behaviours to systems of differential equations, to create hybrid transitor-memristor chips designed to reproduce some of the brain’s thought processes.

Di Ventra and his colleague Yuriy Pershin have gone further and built a memristive synapse that they claim behaves like the real thing [19 page pdf paper Experimental demonstration of associative memory with memristive neural networks

Leon Chua, who came up with the theory of memristors in 1971, has been busy extending his theory of fundamental circuit elements, asking what happens if you combine the properties of memristors with those of capacitors and inductors to produce compound devices called memcapacitors and meminductors, and then what happens if you combine those devices, and so on.

“Memcapacitors may be even more useful than memristors,” says Chua, “because they don’t have any resistance.” In theory at least, a memcapacitor could store data without dissipating any energy at all. Mighty handy – whatever you want to do with them. Williams agrees. In fact, his team is already on the case, producing a first prototype memcapacitor earlier this year, a result that he aims to publish soon. “We haven’t characterised it yet,” he says. With so many fundamental breakthroughs to work on, he says, it’s hard to decide what to do next.

Might a new era in artificial intelligence be at hand?

The Defense Advanced Research Projects Agency certainly thinks so. DARPA is a US Department of Defense outfit with a strong record in backing high-risk, high-pay-off projects – things like the internet. In April last year, it announced the Systems of Neuromorphic Adaptive Plastic Scalable Electronics Program, SyNAPSE for short, to create “electronic neuromorphic machine technology that is scalable to biological levels”.

Williams’s team from Hewlett-Packard is heavily involved. Late last year, in an obscure US Department of Energy publication called SciDAC Review, his colleague Greg Snider set out how a memristor-based chip might be wired up to test more complex models of synapses. He points out that in the human cortex synapses are packed at a density of about 10^10 per square centimetre, whereas today’s microprocessors only manage densities 10 times less. “That is one important reason intelligent machines are not yet walking around on the street,” he says.

Snider’s dream is of a field he calls “cortical computing” that harnesses the possibilities of memristors to mimic how the brain’s neurons interact.

The cortical computing using memristor plan is described here at Scientific Discovery through Advanced computing.

connectivity is achieved by interleaving the neurons of different cortical layers in silicon while stacking with multiple levels of imprinted nanowires that interconnect them. Positive and negative feedback is rampant; in fact, it is necessary to implement cortical algorithms. Simulations show the architecture is very tolerant of device variation and defective components.

The basic idea is to emulate a laminar structure of the cortex by interleaving layers in CMOS. Neurons (gray boxes) are implemented in conventional CMOS; axons and dendrites (blue) in multiple layers of nanowires imprinted on top of the silicon; and synapses (yellow) in memristive (dynamical) junctions formed between selected adjacent layers of imprinted nanowires. CMOS neurons connect to the nanowires through metallic pads (black disks) on the top surface of the silicon. Nano vias (blue cylinders) allow neurons to connect to nanowires at several levels. Neurons in different cortical layers are represented by different shades of gray. Interconnections between and within cortical layers are accomplished with multiple levels of imprinted nanowires. Nanowires are rotated slightly relative to neuron edges to allow long-distance connections. Synaptic nanodevices are created wherever orthogonal nanowires, separated by memristive material, cross each other.

Although silicon neurons cannot be stacked as they are in a biological brain cortex, the same connectivity is achieved by interleaving the neurons of different cortical layers in silicon while stacking with multiple levels of imprinted nanowires that interconnect them.

Nano/CMOS architecture for laminar, cortical circuits (left panel). Neurons are implemented in CMOS (gray), axons and dendrites in nanowires (blue). Synapses are implemented at the junctions of crossing wires separated by memristive material (yellow). Top view (right panel) shows how slight rotation of nanowires allows neurons to communicate via synapses to a neighborhood of other neurons. The small size of memristive nanodevices allows for a large ratio of synapses to neurons, necessary for neuromorphic computation; densities greater than 10^10 devices/cm2 have already been achieved.

The potential applications of neuromorphic computing are stunning: intelligent adaptive control, pattern recognition, decision making, and intelligent-user interfaces with “common-sense” robotics. Because neuromorphic and digital computation have largely non-overlapping applications, future multi-core processors can be envisioned containing support for two cores. Digital cores would be used for number crunching and other conventional applications, and neuromorphic cores would be used for reasoning and adapting to a changing and uncertain world. Undoubtedly, cortical computing will require a series of many small, tentative steps and experiments, if it can be achieved at all in solid-state devices. However, if successful, the market for such intelligent, adaptive systems would be staggering