Artificial Intelligence  

Jeff Hawkins and his team at Numenta have developed a theory of how the brain learns and understands sequences of patterns that may be an essential component for creating intelligent machines.

Researchers at Numenta Inc. have published a new theory that represents a breakthrough in understanding how networks of neurons in the neocortex learn sequences. A paper, authored by Numenta co-founder Jeff Hawkins and VP of Research Subutai Ahmad, “Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex,” has been published in the Frontiers in Neural Circuits Journal, a publication devoted to research in neural circuits, serving the worldwide neuroscience community.

"This study is a key milestone on the path to achieving that long-sought goal of creating truly intelligent machines that simulate human cerebral cortex and forebrain system operations."

“This study is a key milestone on the path to achieving that long-sought goal of creating truly intelligent machines that simulate human cerebral cortex and forebrain system operations,” commented Michael Merzenich, PhD, Professor Emeritus UCSF,Chief Scientific Officer for Posit Science.

Related articles

• Robots For Everybody 
• Artificial Intelligence Research Given New Funding Kickstart
• Gary Marcus Urges People To Think About The Threat of Artificial Intelligence 

The Numenta paper introduces two advances 

First, it provides an explanation of why neurons in the neocortex have thousands of synapses, and why the synapses are segregated onto different parts of the cell, called dendrites. The authors propose that the majority of these synapses are used to learn transitions of patterns, a feature missing from most artificial neural networks.

Second, the authors show that neurons with these properties, arranged in layers and columns - a structure observed throughout the neocortex - form a powerful sequence memory. This suggests the new sequence memory algorithm could be a unifying principle for understanding how the neocortex works. Through simulations, the authors show the new sequence memory exhibits a number of important properties such as the ability to learn complex sequences, continuous unsupervised learning, and extremely high fault tolerance.

Implications for Artificial Intelligence and Neuroscience

“Our paper makes contributions in both neuroscience and machine learning,” Hawkins noted. “From a neuroscience perspective, it offers a computational model of pyramidal neurons, explaining how a neuron can effectively use thousands of synapses and computationally active dendrites to learn sequences. From a machine learning and computer science perspective, it introduces a new sequence memory algorithm that we believe will be important in building intelligent machines.”

“This research extends the work Jeff first outlined in his 2004 book On Intelligence and encompasses many years of research we have undertaken here at Numenta,” said Ahmad, “It explains the neuroscience behind our HTM (Hierarchical Temporal Memory) technology and makes several detailed predictions that can be experimentally verified. The software we have created proves that the theory actually works in real world applications.”

Numenta’s primary goal is to reverse engineer the neocortex, to understand the detailed biology underlying intelligence. The Numenta team also believes this is the quickest route to creating machine intelligence. As a result of this approach, the neuron and network models described in the new paper are strikingly different than the neuron and network models being used in today’s deep learning and other artificial neural networks. Functionally, the new theory addresses several of the biggest challenges confronting deep learning today, such as the lack of continuous and unsupervised learning.

SOURCE  Business Wire

By 33rd Square

Embed

 Neuroscience  

Scientists have carefully reverse engineered and created a 3D model of a synapse. The resulting model will serve as a reference source for neuroscientists of all specializations in the future, and will support future research.

Synapses are the contacts between nerve cells that allow the flow of information that makes our brains work. However, the molecular architecture of these highly complex structures has been largely  unknown until now.

"This 3D model of a synapse opens a new world for neuroscientists."

Now, a research team from Göttingen, led by Prof. Silvio O. Rizzoli from the DFG Research Center and Cluster of Excellence Nanoscale Microscopy and Molecular Physiology of the Brain (CNMPB) of the University Medical Center Göttingen, managed to determine the copy numbers and positions of all important building blocks of a synapse for the first time. This allowed them to reconstruct the first scientifically accurate 3D model of a synapse.

This effort has been made possible by a collaboration of specialists in electron microscopy, super-resolution light microscopy (STED), mass spectrometry, and quantitative biochemistry from the UMG, the Max Planck Institute for Biophysical Chemistry, Göttingen, and the Leibniz Institute for Molecular Pharmacology in Berlin.

The results have been published in the journal Science in an article titled, "Composition of isolated synaptic boutons reveals the amounts of vesicle trafficking proteins". Highlighting the impact of this work, the presented model has been selected as the cover of the respective issue of the Science journal.

The reverse engineering output, shown above displays a synapse in cross section. The small spheres are synaptic vesicles. The model shows 60 different proteins.

“This 3D model of a synapse opens a new world for neuroscientists,” says Rizzoli, senior author of the publication. Particularly the abundance and distribution of the building blocks have long been terra incognita, an undiscovered land. The model presented by Rizzoli and his team now shows several hundreds of thousands of individual proteins in correct copy numbers and at their exact localisation within the nerve cell.

“The new model shows, for the first time, that widely different numbers of proteins are needed for the different processes occurring in the synapse,” says Dr. Benjamin G. Wilhelm, first author of the publication. The new findings reveal: proteins involved in the release of messenger substances (neurotransmitters) from so called synaptic vesicles are present in up to 26,000 copies per synapse. Proteins involved in the opposite process, the recycling of synaptic vesicles, on the other hand, are present in only 1,000-4,000 copies per synapse.

Related articles Drug Found That Reverses Loss of Brain Connections in Alzheimer's Disease  Phosphatidylserine Shown to Slow Familial Dysautonomia and Parkinson's Diseases  Theodore Berger On Neuro Engineering

These details help to solve a long-lasting controversy in neuroscience: how many synaptic vesicles within the synapse can be used simultaneously? Apparently, more than enough proteins are present to ensure vesicle release, but the proteins for vesicle recycling are sufficient for only 7-11% of all vesicles in the synapse. This means that the majority of vesicles in the synapse cannot be used simultaneously.

The most important insight the new model reveals, is however that the copy numbers of proteins involved in the same process scale to an astonishingly high degree. The building blocks of the cell are tightly coordinated to fit together in number, comparable to a highly efficient machinery. This is a very surprising finding and it remains entirely unclear how the cell manages to coordinate the copy numbers of proteins involved in the same process so closely.

The new model will serve as a reference source for neuroscientists of all specializations in the future, and will support future research, since the copy number of proteins can be an important indicator for their relevance. But the research team led by Rizzoli does not plan to stop there: “Our ultimate goal is to reconstruct an entire nerve cell”. Combined with functional studies on the interaction of individual proteins this would allow to simulate cellular function in the future – the creation of a “virtual cell”.

An impressive video animation, below has been created from the obtained data to visualize the structure and protein distribution of a synapse.




SOURCE  Nanowerk

By 33rd Square

Embed

 Artificial Intelligence  

A new brain-inspired synaptic transistor looks toward highly efficient and fast parallel computing and unleash true machine learning.

It is well known that the best supercomputers are staggeringly inefficient, power-hungry machines.

Meanwhile, our brains have upwards of 86 billion neurons, connected by synapses that not only complete myriad logic circuits; they continuously adapt to stimuli, strengthening some connections while weakening others. We call that process learning, and it enables the kind of rapid, highly efficient computational processes.

Materials scientists at the Harvard School of Engineering and Applied Sciences (SEAS) have now created a new type of transistor that mimics the behavior of a synapse. The novel device simultaneously modulates the flow of information in a circuit and physically adapts to changing signals.

Exploiting unusual properties in modern materials, the synaptic transistor could mark the beginning of a new kind of artificial intelligence: one embedded not in smart algorithms but in the very architecture of a computer. The findings appear in Nature Communications.

“There’s extraordinary interest in building energy-efficient electronics these days,” says principal investigator Shriram Ramanathan, associate professor of materials science at Harvard SEAS. “Historically, people have been focused on speed, but with speed comes the penalty of power dissipation. With electronics becoming more and more powerful and ubiquitous, you could have a huge impact by cutting down the amount of energy they consume.”

The human mind, for all its phenomenal computing power, runs on roughly 20 Watts of energy (less than a household light bulb), so it offers a natural model for engineers.

Related articles Andrea Morello Explains Moore's Law and Quantum Computers  Silicon Photonics Breakthrough Promises To Extend Moore's Law  Researchers Develop Design to Seamlessly Integrate Circuits onto Graphene

“The transistor we’ve demonstrated is really an analog to the synapse in our brains,” says co-lead author Jian Shi, a postdoctoral fellow at SEAS. “Each time a neuron initiates an action and another neuron reacts, the synapse between them increases the strength of its connection. And the faster the neurons spike each time, the stronger the synaptic connection. Essentially, it memorizes the action between the neurons.”

In principle, a system integrating millions of tiny synaptic transistors and neuron terminals could take parallel computing into a new era of ultra-efficient high performance.

While calcium ions and receptors effect a change in a biological synapse, the artificial version achieves the same plasticity with oxygen ions. When a voltage is applied, these ions slip in and out of the crystal lattice of a very thin (80-nanometer) film of samarium nickelate, which acts as the synapse channel between two platinum "axon" and "dendrite" terminals. The varying concentration of ions in the nickelate raises or lowers its conductance—that is, its ability to carry information on an electrical current—and, just as in a natural synapse, the strength of the connection depends on the time delay in the electrical signal.

The device consists of the nickelate semiconductor sandwiched between two platinum electrodes and adjacent to a small pocket of ionic liquid. An external circuit multiplexer converts the time delay into a magnitude of voltage which it applies to the ionic liquid, creating an electric field that either drives ions into the nickelate or removes them. The entire device, just a few hundred microns long, is embedded in a silicon chip.

The synaptic transistor offers several immediate advantages over traditional silicon transistors. For a start, it is not restricted to the binary system of ones and zeros.

“This system changes its conductance in an analog way, continuously, as the composition of the material changes,” explains Shi. “It would be rather challenging to use CMOS, the traditional circuit technology, to imitate a synapse, because real biological synapses have a practically unlimited number of possible states—not just ‘on’ or ‘off.’”

The synaptic transistor offers another advantage: non-volatile memory, which means even when power is interrupted, the device remembers its state.

Additionally, the new transistor is inherently energy efficient. The nickelate belongs to an unusual class of materials, called correlated electron systems, that can undergo an insulator-metal transition. At a certain temperature—or, in this case, when exposed to an external field—the conductance of the material suddenly changes.

“We exploit the extreme sensitivity of this material,” says Ramanathan. “A very small excitation allows you to get a large signal, so the input energy required to drive this switching is potentially very small. That could translate into a large boost for energy efficiency.”

The nickelate system is also well positioned for seamless integration into existing silicon-based systems.

“In this paper, we demonstrate high-temperature operation, but the beauty of this type of a device is that the 'learning' behavior is more or less temperature insensitive, and that’s a big advantage,” says Ramanathan. “We can operate this anywhere from about room temperature up to at least 160 degrees Celsius.”

For now, the limitations relate to the challenges of synthesizing a relatively unexplored material system, and to the size of the device, which affects its speed.

“In our proof-of-concept device, the time constant is really set by our experimental geometry,” says Ramanathan. “In other words, to really make a super-fast device, all you’d have to do is confine the liquid and position the gate electrode closer to it.”

In fact, Ramanathan and his research team are already planning, with microfluidics experts at SEAS, to investigate the possibilities and limits for this “ultimate fluidic transistor.”

He also has a seed grant from the National Academy of Sciences to explore the integration of synaptic transistors into bioinspired circuits, with L. Mahadevan, Lola England de Valpine Professor of Applied Mathematics, professor of organismic and evolutionary biology, and professor of physics.

“In the SEAS setting it’s very exciting; we’re able to collaborate easily with people from very diverse interests,” Ramanathan says.

For the materials scientist, as much curiosity derives from exploring the capabilities of correlated oxides (like the nickelate used in this study) as from the possible applications.

“You have to build new instrumentation to be able to synthesize these new materials, but once you’re able to do that, you really have a completely new material system whose properties are virtually unexplored,” Ramanathan says. “It’s very exciting to have such materials to work with, where very little is known about them and you have an opportunity to build knowledge from scratch.”

“This kind of proof-of-concept demonstration carries that work into the ‘applied’ world,” he adds, “where you can really translate these exotic electronic properties into compelling, state-of-the-art devices.”


SOURCE  Harvard

By 33rd Square

Subscribe to 33rd Square

 Neuroscience  

A new technique that allows scientists to measure the electrical activity in the communication junctions of the nervous systems has been developed by a researcher at Queen Mary University of London.

The junctions in the central nervous systems that enable the information to flow between neurons, known as synapses, are around 100 times smaller than the width of a human hair (one micrometer and less) and as such are difficult to target let alone measure.

Now, a new technique that allows scientists to measure the electrical activity in individual synapses of the nervous systems has been developed by a researcher at Queen Mary University of London.

By applying a high-resolution scanning probe microscopy that allows three-dimensional visualisation of the structures, the team were able to measure and record the flow of current in small synaptic terminals for the first time.

Related articles Breakthrough Technique Gives Unprecedented View Inside The Brain  Drug Found That Reverses Loss of Brain Connections in Alzheimer's Disease  Three New Studies Push the Field of Connectomics

“We replaced the conventional low-resolution optical system with a high-resolution microscope based on a nanopipette,” said Dr Pavel Novak, a bioengineering specialist from Queen Mary’s School of Engineering and Materials Science.

“The nanopipette hovers above the surface of the sample and scans the structure to reveal its three-dimensional topography. The same nanopipette then attaches to the surface at selected locations on the structure to record electrical activity. By repeating the same procedure for different locations of the neuronal network we can obtain a three-dimensional map of its electrical properties and activity.”

The research, published in Neuron, opens a new window into the neuronal activity at nanometre scale, and may contribute to the wider effort of understanding the function of the brain represented by the Brain Activity Map Project (BRAIN initiative), which aims to map the function of each individual neuron in the human brain.


SOURCE  Queen Mary University of London

By 33rd Square

Subscribe to 33rd Square