DeepMind Teaches Artificial Intelligence to Read

 Artificial Intelligence  

DeepMind researchers found a novel way to train their neural networks—have them teach themselves how to read, and they found that the way news stories are written online could provide a handy shortcut for the process.

Teaching machines to read natural language documents is still a challenge in the field of artificial intelligence. Reading systems can be tested on their ability to answer questions posed on the contents of documents that they have seen, but until now large scale training and test datasets have been missing for this type of evaluation.

Researchers at Google's DeepMind have now developed a new methodology that resolves this bottleneck and provides large scale supervised reading comprehension data. This work may lead to new deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure.

The research has been published online.

Related articles Ben Goertzel On Aspects of Artificial General Intelligence  The Rise of Artificial Intelligence  The Clever World of Artificial Intelligence

The researchers taught their neural networks to learn is by feeding them huge data sets of annotated examples.

Karl Moritz Hermann at DeepMind in London and fellow researchers found that the special way Daily Mail and CNN write online news articles allows them to be used to train artificial intelligence. The format of the articles suggested a way of creating an annotated database: take the news articles as the texts and the bullet point summaries as the annotation. Plus the sheer volume of articles available online creates an ideal database that computers can use to learn and then answer related about.

Put another way, DeepMind is using Daily Mail and CNN articles to teach computers to read.

"This allows us to develop a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure."

The database created is huge, made up of 110,000 articles from CNN and 218,000 articles from the Daily Mail websites.

For the training, the researchers give the following example of a type of problem known as a Cloze query, that machine learning algorithms are often used to solve. Here, the goal is to identify X in these modified headlines from the Daily Mail: a) The hi-tech bra that helps you beat breast X; b) Could Saccharin help beat X ?; c) Can fish oils help fight prostate X ?

"This allows us to develop a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure," state the researchers.

Hermann and his team showed that a simple type of data mining algorithm called an ngram search could easily find the answer by looking for words that appear most often next to all these phrases. The answer, of course, is the word “cancer.”

The system is limited however, because of the database source.  The researchers admit this is just one instantiation of a very general idea, but it can be developed further. "The incorporation of world knowledge and multi-document queries will also require the development of attention and embedding mechanisms whose complexity to query does not scale linearly with the data set size," they add.


SOURCE  MIT Technology Review

By 33rd Square

Embed