WE’VE evolved over millions of years to sense the world around us. When we encounter something, someone or some place, we use our five natural senses to perceive information about it. That information helps us make decisions and choose the right actions to take. But arguably the most useful information that can help us make the right decision is not naturally perceivable with our five senses, namely the data, information and knowledge that humankind has accumulated about everything and which is increasingly all available online.

Although the miniaturisation of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no link between our digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen.

SixthSense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. SixthSense frees information from its confines by seamlessly integrating it with reality, and thus making the entire world our computer.

The SixthSense prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant-like mobile wearable device.

Both the projector and the camera are connected to the mobile computing device in the user’s pocket.

The projector projects visual information enabling surfaces such as walls and physical objects around us to be used as interfaces; while the camera recognises and tracks a user’s hand gestures and physical objects using computer-vision-based techniques.

The software program processes the video stream data captured by the camera and tracks the locations of the coloured markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer vision techniques.

The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.

The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by multi-touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements.

The drawing application lets the user draw on any surface by tracking the fingertip movements of the user’s index finger. SixthSense also recognises a user’s freehand gestures (postures). For example, the SixthSense system implements a gestural camera that takes photos of the scene the user is looking at by detecting the “framing” gesture. The user can stop by any surface or wall and flick through the photos he or she has taken.

SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognises those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing an “@” symbol lets the user check his or her e-mail.

The SixthSense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the user’s­ wrist projects an analogue watch.

This is only the beginning of what SixthSense technology is capable of, and it’s relatively cheap too. The current prototype system costs about $350 (roughly R2 800) to build.

— www.pranavmistry.com

IT’S not often easy to notice how and when new technology is integrated into society and, more specifically, into our daily lives. “The future” is not something that happens over- night, which is a good thing, because if it did, our heads would probably explode. Gradual change and implementation is good, as it allows our brains to adjust to technological changes.

The bulk of today’s technology was inconceivable 100 years ago, just as the future 100 years from now is. But predictions are still possible. Think about those “futuristic” sliding doors that many gawked at in the Star Trek films; well, we’ve got those now!

The exciting development called SixthSense technology may enter into modern society as early as the next decade. The capabilities of this technology seem so advanced that you really have to see it to believe it, and fortunately you can.

There is a video on YouTube called “Ted sixth sense technology” that provides a great overview and demonstration of this amazing product. I highly recommend searching for it and giving it a watch; although, if you currently had a SixthSense device, you would be able to watch it on this very page …

SixthSense frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer.

An online video of the latest news bulletin relating to the front page story of a newspaper is displayed on its surface.

An Amazon rating appears on a book. The user is also able to call up reviews and reader comments relating to the particular book.

If no surface is available, your hand can be used. Here a hand is used to make a phone call.