Virtual Reality  

A biomedical engineering student from Portugal has used Leap Motion controller and Oculus Rift Headset to create an app in which you can interact with a 3D reconstruction of a human brain and its network of connectivity graphs.

Filipe Rodrigues, a biomedical engineering student from Lisbon, Portugal has created a way to examine brain connectivity in virtual reality as part of his research.  The system uses a Leap Motion Controller and Oculus Rift headset.

Related articles Real World Experience Now Can Be Captured in Virtual Reality  Tactile Virtual Reality System Created  Leap Motion Straps Their Sensor To A VR Headset and the Implications Will Blow Your Mind

“In the beginning, I wanted to develop a haptic glove that I could use with Leap Motion in a virtual reality scenario, allowing me to feel the stuff I touched.” He explained, “For example, when I interrupt brain connectivity, I want to feel the vibrating on my fingers. That was the initial goal for my thesis. I’m still working on it, but I got too involved with the software part of it. I’m loving working with Leap Motion so I’ve postponed the glove component of the project a bit.”

"It’s a good way to visualize complex concepts and try to make them as appealing as possible. Leap Motion is a great tool for this."

The experiment uses slices of MRI scans and Multimodal Brain Connectivity Analysis to reconstruct a 3D model of the brain. Using tractography images, the user is able to see how regions of the brain are connected to each other. Based on this information, a matrix can be drawn of all the connected regions.

“Turning a boring 2D [MRI] visualization into a 3D interactive one is a great way of appealing to people who are maybe not from this discipline,” Rodriques told Leap Motion, “It’s a good way to visualize complex concepts and try to make them as appealing as possible. Leap Motion is a great tool for this.”

The goal is that for every interruption your hand makes on the graph, information would pop up about exactly what you’re interacting with. He’d also like to expand the interaction design to make the project as accessible as possible for the general user.

SOURCE  Leap Motion

By 33rd Square

Embed

 Interfaces  

Working with ultrasound, researchers have developed a new method of haptic feedback. The approach lets users feel invisible holograms in mid-air.

You wont have to imagine what it would be like to touch a priceless artifact in a musuem, or a character from a video game, or your in-progress CAD file, or an organ scanned from your own body in the future.  Researchers have developed a prototype method for creating three-dimensional haptic shapes in mid-air using focused ultrasound. The potential applications for the technology are nearly limitless, and will help define how we interact with digital media in a few years.

Unlike another method that used puffs of air, the approach applies the principles of acoustic radiation force, whereby the non-linear effects of sound produce forces on the skin which are strong enough to generate tactile sensations.

This mid-air haptic feedback eliminates the need for any attachment of actuators or contact with physical devices.

The user perceives a discernible haptic shape when the corresponding acoustic interference pattern is generated above a precisely controlled two-dimensional phased array of ultrasound transducers.

"Touchable holograms, immersive virtual reality that you can feel and complex touchable controls in free space, are all possible ways of using this system."

The research published in ACM Transactions on Graphics, was led by Dr Ben Long and colleagues Professor Sriram Subramanian, Sue Ann Seah and Tom Carter from the University of Bristol’s Department of Computer Science, could change the way 3D shapes are used.  The new technology could enable surgeons to explore a CT scan by enabling them to feel a disease, such as a tumor, or bones from an X-, using haptic feedback.

The method uses ultrasound, which is focused onto hands above the device and that can be felt. By focusing complex patterns of ultrasound, the air disturbances can be felt as floating 3D shapes.

To demonstrate the effect of the system visually, the researchers have demonstrated the ultrasound patterns by directing the device at a thin layer of oil so that the depressions in the surface can be seen as spots when lit by a lamp.

Related articles New Fluid Simulation System Promises To Bring Realistic Water To Video Games  Virtual Reality: Past and Present  Computer Program Learns To Play Classic Nintendo Games

The system generates an invisible 3D shape that can be added to 3D displays to create something that can be seen and felt. The research team have also shown that users can match a picture of a 3D shape to the shape created by the system.

Long, said, “Touchable holograms, immersive virtual reality that you can feel and complex touchable controls in free space, are all possible ways of using this system.

“In the future, people could feel holograms of objects that would not otherwise be touchable, such as feeling the differences between materials in a CT scan or understanding the shapes of artefacts in a museum.”


SOURCE  University of Bristol

By 33rd Square

Embed

 Interfaces  

Researchers have developed a new app enabling users to operate their smartphone with gestures. This development expands the range of potential interactions with such devices.

It does seem slightly odd at first: you hold the phone in one hand, and move the other in the air above its built-in camera making gestures that resemble sign language.

Sometimes you move your index finger to the left, sometimes to the right. You can spread out your fingers, or imitate a pair of pliers or the firing of a pistol. These gestures are not, however, intended for communicating with deaf people; they are for controlling your smartphone.

"While touch input works well for many scenarios, we demonstrate numerous interaction tasks such as mode switches, application and task management, menu selection and certain types of navigation, where such input can be either complemented or better served by in-air gestures."

By mimicking the firing of a pistol, for example, a user can switch to another browser tab, change the map’s view from satellite to standard, or shoot down enemy planes in a game. Spreading out your fingers magnifies a section of a map or scrolls the page of a book forwards.

All this gesturing wizardry is made possible by a new type of algorithm developed by Jie Song, a Master’s student in the working group headed by by Otmar Hilliges, Professor of Computer Science at ETH Zurich. The researchers presented the app to an audience of industry professionals at the UIST Symposium in Honolulu, Hawaii.

The program uses the smartphone’s built-in camera to register its environment. It does not evaluate depth or color. The information it does register – the shape of the gesture, the parts of the hand – is reduced to a simple outline that is classified according to stored gestures. The program then executes the command associated with the gesture it observes. The program also recognizes the hand’s distance from the camera and warns the user when the hand is either too close or too far away.  It works like making the smartphone a Kinect Sensor.

Related articles Use Gesture Interfaces Throughout Your Home Using Only a WiFi Signal   OMRON May Bring Hand Gesture Recognition To Mobile Devices   Google Facial Recognition Patent Filing Looks Ready For YouTube And Project Glass

“Many movement-recognition programs need plenty of processor and memory power”, explains Hilliges, adding that their new algorithm uses a far smaller portion of computer memory and is thus ideal for smartphones. He believes the application is the first of its kind that can run on a smartphone. The app’s minimal processing footprint means it could also run on smart watches or in augmented-reality glasses.

"Our goal is not to replace the touchscreen as primary input device, but rather to augment and enrich the existing interaction vocabulary using gestures," claim the researchers. "While touch input works well for many scenarios, we demonstrate numerous interaction tasks such as mode switches, application and task management, menu selection and certain types of navigation, where such input can be either complemented or better served by in-air gestures."

The program currently recognizes six different gestures and executes their corresponding commands. Although the researchers have tested 16 outlines, this is not the app’s theoretical limit. What matters is that gestures generate unambiguous outlines. Gestures that resemble others are not suitable for this application. “To expand its functionality, we’re going to add further classification schemes to the program,” says the researcher.

He is convinced that this new way of operating smartphones greatly increases the range of interactivity. The researcher’s objective is to keep the gestures as simple as possible, so that users can operate their smartphone effortlessly.

But will smartphone users want to adapt to this new style of interaction? Hilliges is confident they will. Gesture control will not replace touchscreen control, but supplement it. “People got used to operating computer games with their movements.” Touchscreens, Hilliges reminds us, also required a very long adjustment period before making a big impact in consumers’ lives. He is therefore certain that this application – or at least parts of it – will find its way onto the market.




SOURCE  ETH Zurich

By 33rd Square

Embed

 Interfaces  

Two years after buying Perceptive Pixel, Microsoft is ramping up the mass-production of large-screen multi-touch displays.

Stephen Elop, Microsoft's Executive Vice President of Devices, told attendees of Microsoft's Australian Partner Conference recently that Microsoft was getting ready to "mass produce" large format multi-touch displays.

The announcement comes two years after the acquisition of Jeff Han's Perceptive Pixel Inc. (PPI) by Microsoft.

PPI's large-screen touch displays and interactive whiteboards originally sold in the $80,000 range, but Microsoft has exploring ways to make the displays more affordable.

Microsoft has also been focusing  a lot on large format multi-touch as it builds out the Lync Room System with multiple vendors.

When Microsoft bought PPI, the team was folded into Microsoft's Office division because of the synergies between the Office team and PPI around improving teleconferencing technology.

Last year, the Perceptive Pixel team was moved under the Microsoft Devices team. Under Elop's direction, the group is now focused on making the touch-and pen-enabled interface on these devices more useful and more in-line with the "One Windows" work happening across the company.

The work also build on Microsoft's first Surface devices, which were large format interactive multi-touch tables that have faded out of the public consciousness a bit.

Related articles Ultrahaptic Prototype Lets You Feel Gesture Controls In Mid-Air  Ten Best Multi-Touch Display Systems  Craig Mundie on How Tomorrow's Technologies Will Shape Our World

Microsoft's intention is to enable developers to write applications that shine on these large-screen devices, while not requiring them to deviate from the increasingly unified cross-device development platform Microsoft is building.

Rumors around Microsoft's acquisition of other large format interactive display companies, like SMART Technologies seem to have faded, as the industry continues to struggle to define a niche and killer app for the devices.

Elop didn't reveal any new details about pricing or distribution during his remarks at the Partner Conference, but the fact he mentioned PPI leads many to suspect that news on the devices is imminent. Alex Wilhelm at TechCrunch hints the announcement may be part of the upcoming release of Windows 9.


SOURCE  ZDNet

By 33rd Square

Embed

 Interfaces  

inFORM is a Dynamic Shape Display that can render 3D content physically, so users can interact with digital information in a tangible way. inFORM can also interact with the physical world around it, for example moving objects on the table’s surface. Remote participants in a video conference can be displayed physically, allowing for a strong sense of presence and the ability to interact physically at a distance.

At the MIT Media Lab, the Tangible Media Group believes the future of computing is tactile. Unveiled recently, the inFORM is MIT's new scrying pool for imagining the interfaces of tomorrow.

Akin to a large scale form of nano cloudlets, the inFORM is a surface that three-dimensionally changes shape, allowing users to not only interact with digital content iphysically, but even hold hands with a person hundreds of miles away. And that's only the beginning.

Created by Daniel Leithinger and Sean Follmer and overseen by Professor Hiroshi Ishii, the technology behind the inFORM isn't that hard to understand. It's basically a fancy Pinscreen, one of those executive desk toys that allows you to create a rough 3D model of an object by pressing it into a bed of flattened pins.

Related articles MYO Brings Mobility To Gesture Control Interfaces  MIT's Media Lab Director Joi Ito Thinks Singularity's View Is Wrong  Microsoft Kinect Research Points To Future Natural Interface Applications

With inFORM, each of those "pins" is connected to a motor controlled by a nearby laptop, which can not only move the pins to render digital content physically, but can also register real-life objects interacting with its surface thanks to the sensors of a hacked Microsoft Kinect.

"Right now, the things designers can create with graphics are more powerful and flexible than in hardware," Leithinger told Co.Design. "The result is our gadgets have been consumed by the screen and become indistinguishable black rectangles with barely any physical controls. That's why BlackBerry is dying."

Moving beyond the simulated affordances of tablets and today's smartphones is what the researchers are working towards. Follmer says, "As humans, we have evolved to interact physically with our environments, but in the 21st century, we're missing out on all of this tactile sensation that is meant to guide us, limit us, and make us feel more connected," he says. "In the transition to purely digital interfaces, something profound has been lost."

"Ten years ago, we had people at Media Lab working on gestural interactions, and now they're everywhere, from the Microsoft Kinect to the Nintendo Wiimote," says Follmer. "Whatever it ends up looking like, the UI of the future won't be made of just pixels, but time and form as well. And that future is only five or ten years away. It's time for designers to start thinking about what that means now."

Imagine empty rooms with inFORM as the floor.  Such a space could conceivably be morphed into countless configurations, and combined with augmented reality and virtual reality could create an immersive physical and digital environment.

SOURCE  Fast Company

By 33rd Square

Subscribe to 33rd Square

 Interfaces  

The U.S. Patent and Trademark Office has published an Apple patent for a method of generating and manipulating a 3D object on a computing device, with the process controlled by special gestures made above a touchscreen's surface. Combined with a hologram display, the system could work much like Tony Stark's does in the Iron Man movies.

Apple's U.S. Patent No. 8,514,221, titled "Working with 3D objects," describes a graphical interface that allows a user to "generate and manipulate 3D objects using 3D gesture inputs," Apple Insider reports.   More specifically, the interface can be a computer assisted design (CAD) application running on a computer with a touch-sensitive surface, such as an iPad.

The patent refers to a device that can detect the location of fingers with a combination of capacitive touch sensors and proximity sensors embedded in the display. These two components can be separate, or the capacitive sensors themselves can act as proximity sensors by measuring the capacitance of a nearby finger.

Does this mean that proximity sensing will be a new feature on the next iPad release?

Related articles Use Gesture Interfaces Throughout Your Home Using Only a WiFi Signal  Microsoft Kinect Research Points To Future Natural Interface Applications  MYO Brings Mobility To Gesture Control Interfaces

The patent describes how 3D objects can be extruded from 2D images with the proposed system. Generating the 3D renderings, and manipulating 2D objects with conventional touchscreen gestures, such as pinching. Instead of continuing with established iOS gestures, the invention deviates by introducing a third axis of control.

Further, modifications can be performed with "pinch and pull" or "pinch and push" gestures, which extrude or push in a portion of a 3D object, respectively. In another embodiment, objects can be rotated around the z-axis with a twisting motion.

One interesting application described in the document is dubbed "sculpting" mode. This embodiment treats the 3D object as if it were made of clay, like the interface used in ZBrush software. Depending on finger movement, a user can make indentations, stretch, or squeeze the shape so that one area is made smaller and another larger. A "pinch-twist-and-pull" gesture can also be used to break off a piece of the digital clay.

In another use scenario, textures, colors and other surface attributes can be selected and tuned by hovering over a screen's surface. These gestures are largely made perpendicular to the device's display, with selected attributes corresponding to height away from the screen. For example, an object can be made brighter when a finger hovering over a "Brightness" UI region is moved farther away from the display. Slide bars, buttons, and other assets may act as alternative controls.

Finally, the patent makes mention of using 3D or stereoscopic glasses to make the experience even more engaging. Also noted are further UI implementations such as iconography, graphical assets and general usability considerations.

Proximity sensors similar to that described in Apple's patent is just now making its way to the mass market in the Leap Motion Controller and Kinect sensor.

Robbie Tilton has put together a demonstration video showing a Leap Motion controlling a Three.js rendered 3D hologram that gives a preliminary example of the possibilities of 3D interfaces. In the demo, a globe is projected on a 4-sided prism, and while it’s not quite as good as what Tony Stark has in Iron Man, it’s still very interesting.




SOURCE  Apple Insider

By 33rd Square

Subscribe to 33rd Square