Deep Learning: Intelligence from Big Data

 Artificial Intelligence  

This month, the Stanford Graduate School of Business held a sold out session on deep learning moderated by Steve Jurvetson.

A machine learning approach inspired by the human brain, deep learning is taking many industries by storm. Empowered by the latest generation of commodity computing, deep learning begins to derive significant value from Big Data. It has already radically improved the computer’s ability to recognize speech and identify objects in images, two fundamental hallmarks of human intelligence.

In deep learning, each layer of representation builds upon the previous ones, so that, as the system goes deeper, the representation becomes clearer.

"Now we’re getting into a world where we can take measurements of the physical world, like pixels in a picture, and turn them into symbols that we can sort."

Industry giants such as Google, Facebook, and Baidu have acquired most of the dominant players in this space to improve their product offerings. At the same time, startup entrepreneurs are creating a new paradigm, Intelligence as a Service, by providing APIs that democratize access to deep learning algorithms.

Recently, Moderator Steve Jurvetson, Partner, DFJ Ventures discussed the trends in artificial intelligence with panelists Adam Berenzweig, Co-founder and CTO, Clarifai; Naveen Rao, Co-founder and CEO, Nervana Systems; Elliot Turner, Founder and CEO, AlchemyAPI; and Ilya Sutskever, Research Scientist, Google Brain to learn more about this exciting new technology and be introduced to some of the new application domains, the business models, and the key players in this emerging field.

The discussion took place at the Stanford Graduate School of Business this month.

Jurvetson explains why deep learning has had so much impact on AI in the last few years. First, there’s a lot more data around because of the Internet, there’s metadata such as tags and translations, and there’s even services such as Amazon’s Mechanical Turk, which allows for cheap labeling or tagging. Jurvetson focuses on new developments in artificial intelligence including Big Data, and algoritmic advances in using unlabeled data, unsupervised training and successive layers of learning that are leading to many new advances in the field.

Jurvetson, like others states that neural networks have been around for a long time, but the new factor is that they can now process Big Data thanks to the progress of Moore's Law.  In the case of Google Brain, for instance, 1 billion synapses were used to detect cats from YouTube videos.

A fan of Ray Kurzeil, Jurvetson, when showing Kurzweil's Moore's Law plot states that it may the most important graph ever plotted, not just in technology and business. He also mentions how quantum computing may have considerable impact on deep learning as well.  As a concluding image, Jurvetson shows a picture of a black swan drinking from a fire hose to represent what is coming in the field of artificial intelligence.

Berenzweig, a former engineer at Google for 10 years, made the case that deep learning is “adding a new primary sense to computing” in the form of useful computer vision. Deep learning is forming that bridge between the physical world and the world of computing, according to Berenzweig. “Now we’re getting into a world where we can take measurements of the physical world, like pixels in a picture, and turn them into symbols that we can sort,” he says.

Related articles Ben Goertzel On Aspects of Artificial General Intelligence  AI System Breaks CATCHA Turing Test  The Rise of Artificial Intelligence

Berenzweig does a demo of the ClarifAI system, which you can try too at: http://clarifai.com/  For ClarifAI, the ImageNet database is incredibly useful for scoring their image recognition system. The convolutional neural network used by ClarifAI can recognize 10,000 objects now.

Deep learning is “that missing link between computing and what the brain does,” according to Nervana Systems' Rao. It i more than just fast computation now, “We can start building new hardware that takes computer processing in a whole new direction,” more like the brain does. Also, this now means that the business case is opened up for deep learning. "We really need to scale these things up," says Rao.

What excites Sutskever, a colleague of Geoffrey Hinton, is that neural networks and deep learning, actually works.  "It may seem that we have these programs that seem to do very complicated things, that the programs must be very complicated themselves, but that is not the case," he says.

Turner, states his company’s mission is to “democratize deep learning.” The company is working in many industries from advertising to financial services to business intelligence, helping companies apply it to their businesses.  They feature demos on text/language and image recognition on their website.

Turner is excited by the transition in machine learning. "Despite having the word 'machine' in the name, machine learning historically had a lot of human involvement.  It relied on the innate cleverness of individuals in the process known as feature engineering to translate raw data into something that traditional shallow learning algorithms could effectively deal with." These processes are now becoming weakly supervised or even unsupervised.

These breakthroughs are also allowing the smaller teams and companies to compete with the larger organizations.

Jurvetson asks the panel if the work is a path leading to AI (we can assume he is talking about AGI based on his description).  Turner replies that a lot of different problems are now being solved with human data - what is important to people (pictures of houses, cats, dogs, etc.), and these will be a factor in how neural networks evolve.

Sutskever, says that it is in progress on learning principles that is important.  "Whenever we make conceptual progress on learning principles, we will make very huge practical progress very rapidly."


SOURCE  VLAB

By 33rd Square

Embed