Artificial Intelligence  

Jeff Hawkins and his team at Numenta have developed a theory of how the brain learns and understands sequences of patterns that may be an essential component for creating intelligent machines.

Researchers at Numenta Inc. have published a new theory that represents a breakthrough in understanding how networks of neurons in the neocortex learn sequences. A paper, authored by Numenta co-founder Jeff Hawkins and VP of Research Subutai Ahmad, “Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex,” has been published in the Frontiers in Neural Circuits Journal, a publication devoted to research in neural circuits, serving the worldwide neuroscience community.

"This study is a key milestone on the path to achieving that long-sought goal of creating truly intelligent machines that simulate human cerebral cortex and forebrain system operations."

“This study is a key milestone on the path to achieving that long-sought goal of creating truly intelligent machines that simulate human cerebral cortex and forebrain system operations,” commented Michael Merzenich, PhD, Professor Emeritus UCSF,Chief Scientific Officer for Posit Science.

Related articles

• Robots For Everybody 
• Artificial Intelligence Research Given New Funding Kickstart
• Gary Marcus Urges People To Think About The Threat of Artificial Intelligence 

The Numenta paper introduces two advances 

First, it provides an explanation of why neurons in the neocortex have thousands of synapses, and why the synapses are segregated onto different parts of the cell, called dendrites. The authors propose that the majority of these synapses are used to learn transitions of patterns, a feature missing from most artificial neural networks.

Second, the authors show that neurons with these properties, arranged in layers and columns - a structure observed throughout the neocortex - form a powerful sequence memory. This suggests the new sequence memory algorithm could be a unifying principle for understanding how the neocortex works. Through simulations, the authors show the new sequence memory exhibits a number of important properties such as the ability to learn complex sequences, continuous unsupervised learning, and extremely high fault tolerance.

Implications for Artificial Intelligence and Neuroscience

“Our paper makes contributions in both neuroscience and machine learning,” Hawkins noted. “From a neuroscience perspective, it offers a computational model of pyramidal neurons, explaining how a neuron can effectively use thousands of synapses and computationally active dendrites to learn sequences. From a machine learning and computer science perspective, it introduces a new sequence memory algorithm that we believe will be important in building intelligent machines.”

“This research extends the work Jeff first outlined in his 2004 book On Intelligence and encompasses many years of research we have undertaken here at Numenta,” said Ahmad, “It explains the neuroscience behind our HTM (Hierarchical Temporal Memory) technology and makes several detailed predictions that can be experimentally verified. The software we have created proves that the theory actually works in real world applications.”

Numenta’s primary goal is to reverse engineer the neocortex, to understand the detailed biology underlying intelligence. The Numenta team also believes this is the quickest route to creating machine intelligence. As a result of this approach, the neuron and network models described in the new paper are strikingly different than the neuron and network models being used in today’s deep learning and other artificial neural networks. Functionally, the new theory addresses several of the biggest challenges confronting deep learning today, such as the lack of continuous and unsupervised learning.

SOURCE  Business Wire

By 33rd Square

Embed

Artificial Intelligence  

Numenta founder Jeff Hawkins spoke recently at the 2016 Neuro-Inspired Computational Elements Workshop at UC Berkeley about the biological nature of intelligence and what it might mean for the development of artificial intelligence.

Palm Computing and Numenta founder Jeff Hawkins spoke recently at the 2016 Neuro-Inspired Computational Elements Workshop at UC Berkeley about the biological nature of intelligence and what it might mean for the development of artificial intelligence. The video is embedded below.

Related articles

• Jeff Hawkins Talks Building Brains 
• The Clever World of Artificial Intelligence 
• Ben Goertzel On Aspects of Artificial General Intelligence 

Hawkins is the author of On Intelligence, a book he wrote before founding Numenta, an artificial intelligence company that applies his biologically-inspired theories to algorithm creation. Hawkins actually prefers that his company not be biologically inspired, but biologically constrained. The book describes hierarchical temporal memory (HTM), an unsupervised to semi-supervised online machine learning model that models some of the structural and algorithmic properties of the neocortex.

HTM is a method for discovering and inferring the high-level causes of observed input patterns and sequences, thus building an increasingly complex model of the world.

In his talk, Hawkins rapidly covers how learning takes place in the brain though communication and feedback in the neocortical columns (pictured above), and at the synapses. "It is an extremely robust system in every single way," he states. "Neurons, synapses, dendrites -- this is an important property if you are ever going to build these things in hardware."

"It’s that learning process that makes intelligence," Hawkins told Wired last year. "We continue to learn for our entire life. Intelligence isn’t something you reach. It’s a continuous process you go through."

Hawkins then presents 'Hawkins' List' of how the biological parts of intelligence can be organized into their base functional elements:

Hawkins’ List of the Functional Components of Intelligence

1) Networks of neurons that learn and recall sequences (required)

- Continuous learning, not batch (required) - Many simultaneous predictions (required)
- Robust (required)
 HTM: active dendrites, synaptogenesis, no spikes

2) Regions that use sequence memory for: 

- sensory inference (required)
- sensory-motor inference (required)
- motor generation (required)

3) Hierarchy of regions (required) 

- number of regions (parameter)
- size of regions (parameter)
- connectivity graph (parameter)

4) Embodiment (required) 

- sensors (parameter)
- built-in behaviors (parameter)
- emotions/motivations (parameter)
- episodic/spatial memory (parameter)

"If you're going to build an intelligent system, it's going to have to have networks of neurons that learn and recall sequences."

"If you're going to build an intelligent system, it's going to have to have networks of neurons that learn and recall sequences," states Hawkins. "That is its fundamental 'tissue' or premise that is has to use. All inference, almost all inference in auditory, visual and somatosensory is inference of sequences. All motor behaviour is playing back sequences. This is not something to be added to a system it is a core fundamental principle. "

Using his theory and applications of it at Numenta, Hawkins shared his two aspirational goals of brain-inspired computing. First, he would like to see a machine that acts like a super mathematician. "You could literally build a hierarchy that's designed where part of the behaviours are mathematical behaviours."

Hawkins points to the need for a super smart Robonaut

He also looks forward to our future in space. "We need to make super engineer scientist robots to do this stuff," he says. "I know this sounds crazy, I know, but I'm not joking." The machines will be needed to do a lot of the early heavy lifting on off planet colonies before human beings will ever be ready to go Hawkins projects.

Hawkins doesn't think these goals will happen in his lifetime, but expects they will come to pass soon and his theories will be part of their creation.

SOURCE  UC Berkeley

By 33rd Square

Embed

Artificial Intelligence  

In the 1990s, Jeff Hawkins became both rich and famous when he invented the Palm Pilot—a device that in no small way ushered in a whole new era of mobile computing. These days, though, he’s on a far more ambitious mission. His goal: to build a machine that can think and reason on its own by mimicking the workings of the human brain. Here, Hawkins discusses the both risks and rewards of artificial intelligence.

Related articles Russian Scientists Claim Artificial Intelligence Breakthrough  Jeff Hawkins on the Path to Machine Intelligence  10 Signs You Should Invest In Artificial Intelligence

"There are things we should be scared of."

SOURCE  University of California Television

By 33rd Square

Embed

 Artificial Intelligence  

Numenta founder Jeff Hawkins recently  told a Silicon Valley conference that he believes his company will successfully produce the first computers which can think and reason on their own,"in less than five years."

Jeff Hawkins, the man behind the first Palm Pilot and the brain-inspired artificial intelligence company Numenta told a Silicon Valley conference recently that he believes his company will successfully produce the first computers which can think and reason on their own. "We're going to finish this off in less than five years." he stated.

"Once we understand those principles of the neocortex, we can modify them -- we don’t need to be true to evolutionary biology. We still have so much to learn about the basics of how biology works. Progress is incremental but also exponential. We’re going to finish this off in less than five years, I believe."

Hawkins delivered his prediction at the SVForum iHuman Conference in San Mateo, California.  He has been building up Numenta for the past nine years, following in-depth neuroscience research which he detailed in his book, On Intelligence,

Widely regarded as one of the country's foremost experts in the biological structure of the human brain, Hawkins has been working on machine learning technology that would give computers the ability to gather large amounts of data and use it to become even smarter as time goes on.

"I'm trying to reverse engineer the neocortex," explained Hawkins, referring to the area of the brain where spatial reasoning and conscious thought reside.

"Intelligence shouldn’t be measured by any particular task. What characterizes intelligence is extreme flexibility… building a flexible learning system. [Some AI is] focused on being human-like; our work here is not being human-like at all. It’s about understanding the general principles of intelligence that we can apply to all kinds of problems," said Hawkins.

Related articles Stuart Armstrong and Nick Hawes Discuss the Future of Artificial Intelligence  Jeff Hawkins On Open Intelligence  Ben Goertzel On Aspects of Artificial General Intelligence

Hawkins believes reverse engineering the neocortex is the fastest way to intelligent machines. Neuroscience has shown that language and touch work on the same principles, and Hawkins predicts that a machine’s abilities will unfold in a similar way once scientists are able to tap inherent potential.

“Once we understand those principles of the neocortex, we can modify them -- we don’t need to be true to evolutionary biology,” Hawkins said. “We still have so much to learn about the basics of how biology works. Progress is incremental but also exponential. We’re going to finish this off in less than five years, I believe.”

The implications of Hawkins' work and the potential new technology offered by Numenta are significant. Today's computers are largely programmed by humans in much the same way that they were 30 years ago. If machines could learn on their own without the vast amounts of time and effort to program them, then it would dramatically revolutionize how technology is created today.

Earlier this year, Numenta announced the first commercial application of its research when it revealed that their Grok IT for Analytics platform was now running on Amazon Web Services. Grok, which is named after the term for "understanding" in Robert Heinlein's 1961 science fiction novel Stranger in a Strange Land, is being used by Amazon to detect pattern changes in the operation of their servers.

"We can go beyond what biology has done," said Hawkins. "I believe the quickest way to get to intelligent machines is the path we're taking."

There is also growing concern that legitimate machine self-learning could unleash a dangerous world where computers without human control would replicate the good and the bad. Hawkins addressed this directly last week at the iHuman Conference when he said "a lot of smart people think this is really dangerous, but I think they are wrong."

Hawkins also said that people should be more worried about the ability for humans to self-replicate viruses on computer systems today. "I'm far more worried about people in some lab making a new virus than machine intelligence," said Numenta's co-founder.

The talk below is not the exact talk Hawkins gave at iHuman, but was produced very near the same time.



SOURCE  EE Times

By 33rd Square

Embed

 Artificial Intelligence  

At this year's Exponential Finance conference, Singularity University's Neil Jacobstein spent time reviewing current major developments in artificial intelligence, and what they might mean for the future.

Singularity University's Co-Chair on artificial intelligence Neil Jacobstein discusses how businesses, financial institutions and individuals are utilizing AI to address credit risk analysis, investment decisions, sentiment analysis, high frequency trading and avoiding corporate/market flash crashes in the video above.

The video was recorded at the Exponential Finance conference earlier this summer.  The meeting, hosted by Singularity University in partnership with CNBC, is an intensive two-day conference that brought together top experts to inform financial services leaders how technologies—such as artificial intelligence, quantum computing, crowdfunding, digital currencies and robotics—are impacting business.

"AI is more than just a lever, it's a fulcrum that can increase the leverage of all the other exponential technologies."

During the talk, Jacobstein also provides a sweeping overview of AI—from 50 years of successful AI applications to augmenting humans with AI—and what this means for the world at large.

"AI is more than just a powerful set of problem-solvers," states Jacobstein. "AI is more than just a lever, it's a fulcrum that can increase the leverage of all the other exponential technologies."

Related articles Bruce Sterling Says The Singularity Has No Business Model  Energy Efficient Brain Simulator Outperforms Supercomputers  Will Artificial Intelligence Serve Us or Replace Us?

According to Jacobstein, AI is currently based around three main areas of development: statistical and deep machine learning; task and domain specific knowledge engineering; and biologically inspired computing architectures. Most of the key applications created in the last 15 years or so use all three of these techniques.

He goes on to cover some of the major commercial and academic developments in AI, including Jeff Hawkins' Grok (Numenta), Facebook's hiring of Yann LeCun, Geoff Hinton's deep learning work, Kaggle, IBM's SyNAPSE and Watson work, Siri, Google Now and more.

"Eventually we will build an artificial neocortex that is much larger than the ones we have today," Jacobstein reports. "The human neocortex today is about the size of a large dinner napkin.  One can imagine building a neocortex not constrained by the cage of the skull, with the surface area the size of this room, or Manhattan, or New York, or the U.S. or even the planet."

Is this excessive, he asks.  "No, it's not excessive, because we are going to have billions of sensors around the world, millions of sensors around our corporations, thousands, perhaps millions around our bodies and we're going to get real-time interpretation and prediction from those sites using AI."

Somewhat unnerving is that Jacobstein concludes his talk with the preview clip of David the android from the movie Prometheus, suggesting this is a realization of AI in the future.

Jacobstein suggests that the changes coming from AI, will be radical and revolutionary. "The key to harnessing that well of revolution is learning with computers, learning to race with computers, not just against them."


SOURCE  Exponential Finance

By 33rd Square

Embed

 Artificial Intelligence  

As we track developments in artificial intelligence there is undoubtedly a greater awareness around the current capabilities of the technology and the path forward.  It is now widely agreed that we are in an unprecedented era of large-scale development in the field.

In the history of artificial intelligence research, there have been some especially dark periods when the deliverables did not match the hype. The popular name for these periods like two major spans 1974–80 and 1987–93 is "AI winter".  In the last few years though significant progress, enthusiasm and funding has led to a warmer environment for AI, or a new spring.

For investor Aydin Senkut, "The reality is, there are a very limited number of AI and machine learning experts in the world, which is one reason why it's been getting so much attention."

Although most experts agree that artificial general intelligence (AGI) is still far off on the horizon, the current development push may accelerate its arrival.

At the recent IARPA-sponsored conference, exploring the future of research into Machine Intelligence from Cortical Networks (MICrONS), experts were asked to propose MICrONS-related projects that aim to "create a new generation of machine learning algorithms derived from high-fidelity representations of cortical microcircuits to achieve human-like performance on complex information processing tasks."

"The intelligence community puts money into it, the government puts money into it. The environment is pretty warm right now, it's a new spring."

According to The Register's Jack Clark, conference attendees were especially guarded in how transformational the artificial intelligence community is progressing.

In spite of this, the conference looked to get answers and direction for the lofty research areas involving creating a new generation of machine learning algorithms derived from high-fidelity representations of cortical microcircuits to achieve human-like performance on complex information processing tasks.

To achieve this goal, the conference looked to:

• Propose an algorithmic framework for information processing that is consistent with existing neuroscience data, but that cannot be fully realized without additional specific knowledge about the data representations, computations, and network architectures employed by the brain;
• Collect and analyze high-resolution data on the structure and function of cortical microcircuits believed to embody the cortical computing primitives underlying key components of the proposed framework;
• Generate computational neural models of cortical microcircuits informed and constrained by this data and by the existing neuroscience literature to elucidate the nature of the cortical computing primitives; and
• Implement novel machine learning algorithms that use mathematical abstractions of the identified cortical computing primitives as their basis of operation.

Using data and techniques derived from neuroscience and connectomics, the direction from IARPA seems to be to duplicate the brain's wiring to improve machine learning.  Far from working in isolation, today's AI researchers are looking to biology for inspiration, especially at the only known actual intelligence in the universe: our brain. MICrONS conference attendees included (presumably) Jeff Hawkins from Numenta, who is well known for his theory of intelligence derived from the study of the neocortex.

Related articles Is It Humanity's Fate To Be The 'Mitochondria' of Super-Intelligent AIs?  The New Era of Cognitive Computing  Gary Marcus Urges People To Think About The Threat of Artificial Intelligence

Hawkins' former colleague Dileep George and his company, Vicarious have, in the meantime, been expanding their funding for their mostly under-wraps work in artificial intelligence. Their list of investors includes Elon Musk, Peter Thiel, Mark Zuckerberg, Jeff Bezos, Ashton Kutcher, and Dustin Moskowitz.

Some of the initial buzz around Vicarious involves their system's cracking of CAPTCHA tests.  "We should be careful not to underestimate the significance of Vicarious crossing this milestone," said Facebook co-founder and board member Moskovitz. "This is an exciting time for artificial intelligence research, and they are at the forefront of building the first truly intelligent machines."

London-based DeepMind is working on similar projects as Vicarious, like advanced image recognition, though they too are quite secretive about what exactly they're up to. The company was recently acquired by Google.  Google is gobbling up AI and robotics companies as well as researchers such as Geoff Hinton.

The other big corporate names in Silicon Valley are also hungry for AI expertise.  Facebook recently started up an AI endeavor headed by Yann LeCun, and Amazon is known to be heavily researching the area.  Microsoft has progressed well beyond the days of "Clippy," with Cortana about to be widely released and Microsoft Research publishing impressive results.

IBM, long the leader in cutting-edge AI has spun off Watson into a billion dollar company, with new partnerships for the cognitive computing engine springing up all the time.  The company has even partnered with long-time rival Apple to develop applications that will use IBM's technology.

The AI research community is now used to taking a long view on things, despite the hype around the technological Singularity. Time will tell how productive and revolutionary this new AI spring has been, but the effort is large

"The intelligence community puts money into it, the government puts money into it," said one attendee at MIRCrONS. "The environment is pretty warm right now, it's a new spring."


SOURCES  The Register, Business Insider

By 33rd Square

Embed

 Artificial Intelligence  

Jeff Hawkins, founder of Numenta recently gave a talk highlighting his approach to machine intelligence—an approach that is highly dependent on an understanding of the human neocortex.

Understanding how the brain works and building machines that work on the same principles is one of the greatest quests of our time. In a recent talk (video below) by Jeff Hawkins, founder of Palm and now, Numenta describes recent advances in neocortical theory, including why the brain uses sparse distributed representations and how the brain makes predictions from high velocity sensory data streams.

"[Neural networks] may be the right thing to do, but it's not the way brains work and it's not the principles of intelligence, and it's not going to lead to a system that can explore the world or systems that can have behavior."

The talk features a demonstration of Hawkins' product called GROK, that uses a detailed model of neocortical memory to act on machine generated data and how developers can contribute to the development of intelligent machines via the NuPIC open source project.

"I'm surprised by how few people believe they need to understand how the brain works to build intelligent machines," Hawkins says. "I'm disappointed by this."

"I want to bring about intelligent machines, machine intelligence, accelerated greatly from where it was going to happen and I don't want to be consumed – I want to come out at the other end as a normal person with my sanity," Hawkins told The Register recently. "My mission, the mission of Numenta, is to be a catalyst for machine intelligence."

While major firms like Google and Facebook, and small companies like Vicarious, are striding over well-worn paths, Hawkins following the theories he laid out in his book, On Intelligence, believes he is taking a new, and better approach.

Related articles Dendrites Found To Process Information In the Brain  Artificial Intelligence Research Given New Funding Kickstart  Jeff Hawkins Talks Building Brains

This approach is also the source of the greatest criticisms of Hawkins' work. With so little actually confirmed about how brain works, he is essentially working from a theory; not evidence.

"No one knows how the cortex works, so there is no way to know if Jeff is on the right track or not," Dr Terry Sejnowski, the laboratory head of the Computational Neurobiology Laboratory at the Salk Institute for Biological Studies in San Diego, California, told The Register. "To the extent that [Hawkins] incorporates new data into his models he may have a shot, and there will be a flood of data coming from the BRAIN Initiative that was announced by Obama last April."

"I know of no other cortical theories or models that incorporate any of the following: active dendrites, differences between proximal and distal dendrites, synapse growth and decay, potential synapses, dendrite growth, depolarization as a mode of prediction, mini-columns, multiple types of inhibition and their corresponding inhibitory neurons, etcetera. The new temporal pooling mechanism we are working on requires metabotropic receptors in the locations they are, and are not, found. Again, I don't know of any theories that have been reduced to practice that incorporate any, let alone all of these concepts," he wrote in a post to the discussion mailing list for Numenta's open-source NuPic.

Numenta's approach also relies on time. Its Cortical Learning Algorithm is essentially an engine for processing streams of information, classifying them, learning to spot differences, and using time-based patterns to make predictions about the future.



SOURCE  The Register, GOTO Conferences

By 33rd Square

Embed

 Artificial Intelligence  

Recently, Jeff Hawkins briefly outlined the breakthrough he and his team at Numenta have claimed in the field of machine intelligence. They have applied theory into practice, with the product, Grok.

J

eff Hawkins' company, Numenta is working on machine intelligence. Their approach is to first understand, in detail, how the neocortex works and then build systems that work on those principles.

In the past few years they claim that they have made good progress on both the theory and implementing the theory in software. The resulting technology is a type of memory system based on sparse representations and distributed sequence memory. Like the brain, systems built on this technology need to be trained on streaming sensory data. The system learns the spatial and temporal structure in sensory data.

Related articles Michio Kaku on the Intelligence Revolution  Chris Eliasmith Discusses SPAUN's Artificial Intelligence  Jeff Hawkins Talks Building Brains

Numenta has applied this technology to a commercial product called Grok. Grok is fed data from sensors, it discovers the structure in the data and uses this to make predictions and detect anomalies. Grok is being applied in diverse industries such as energy pricing, product forecasting, predictive maintenance, and finance.

According to Hawkins, Grok represents a new approach to machine learning and he believes it will be the fundamental component of all machine intelligence.

Hawkins is an engineer, serial entrepreneur, scientist, inventor, and author. He was a founder of two mobile computing companies, Palm and Handspring, and was the architect of many computing products such as the PalmPilot and Treo smartphone. Throughout his life Hawkins has also had a deep interest in neuroscience and theories of the neocortex. In 2002 he founded the Redwood Neuroscience Institute, a scientific institute focused on understanding how the neocortex processes information. The institute is now located at U.C. Berkeley.

In 2004 we wrote the book On Intelligence, which describes progress on understanding the neocortex. In 2005 he co-founded Numenta, a startup company building a technology based on neocortical theory. It is his hope that Numenta will play a catalytic role in the emerging field of machine intelligence.

SOURCE  OSCON 2013

By 33rd Square

Subscribe to 33rd Square

 Artificial Intelligence  

Neuroscience and artificial intelligence expert, Jeff Hawkins, recently gave a Google Tech Talk, where he discussed recent advances in understanding the neocortex and how we are applying them to model millions of high velocity data streams.

According to neuroscience and artificial intelligence expert, Jeff Hawkins, the neocortex works on principles that are fundamentally different than traditional computers.

In this Tech Talk from Google, Hawkins discusses recent advances in understanding the neocortex and how we are applying them to model millions of high velocity data streams.

The talk starts with a description of sparse distributed representations, which are the fundamental units of information in brains. He then discuss how these representations are learned and how the brain processes them to build predictive models from sensory data.

Hawkins' company, Numenta has built a product called Grok that emulates these capabilities of the neocortex.

Grok is being used to understand high velocity machine generated data in many different domains. I will give a brief introduction to Grok and speculate on the future of machine intelligence.

At the end of the talk, Hawkins takes questions, including one from Google's new head of engineering, Ray Kurzweil.



SOURCE  Google Tech Talks

By 33rd Square

Subscribe to 33rd Square