Deep learning has made large inroads in the world of computer vision, and many other recognition tasks, in recent years. Microsoft has just announced that it is bringing the technology to its HoloLens system, integrating a deep neural network into the system's holographic processor.
Many of the most difficult recognition and computer vision problems have seen major gains in recent years. Now, Microsoft hopes to embed this technology into the latest version of their HoloLens augmented reality computer system.
"I work on HoloLens, and in HoloLens, we’re in the business of making untethered mixed reality devices, writes Microsoft's Marc Pollefeys, Director of Science for HoloLens. "We put the battery on your head, in addition to the compute, the sensors, and the display. Any compute we want to run locally for low-latency, which you need for things like hand-tracking, has to run off the same battery that powers everything else. So what do you do?"
"You create custom silicon to do it."
"Mixed reality and artificial intelligence represent the future of computing."
"Mixed reality and artificial intelligence represent the future of computing," Pollefeys writes.HoloLens contains a custom multiprocessor called the Holographic Processing Unit, or HPU. It is responsible for processing the information coming from all of the on-board sensors, including Microsoft’s custom time-of-flight depth sensor, head-tracking cameras, the inertial measurement unit (IMU), and the infrared camera.
Related articles
According to Microsoft, the HPU is part of what makes HoloLens the world’s first–and still only–fully self-contained holographic computer.Recently, Harry Shum, executive vice president of the company's Artificial Intelligence and Research Group, announced in a keynote speech at the Conference on Computer Vision and Pattern Recognition (CVPR), that the second version of the HPU, currently under development, will incorporate an AI coprocessor to natively and flexibly implement deep neural networks (DNNs).
The chip supports a wide variety of layer types, that will be fully programmable. Shum demonstrated an early spin of the second version of the HPU running live code implementing hand segmentation at the conference.
The AI coprocessor is designed to work in the next version of HoloLens, running continuously, off the HoloLens battery.
According to Pollefeys, this is the kind of technology that needs to be developed to bring about "mixed reality devices that are themselves intelligent."
0 comments:
Post a Comment