Lenovo just released the first mobile phone that can see the world in 3D. What does this mean? Is it merely another gimmick? Or the next evolution in ubiquitous mobile technologies that could transform healthcare? The Klick Labs team can't wait to tell you...
Back in 2013, Apple acquired a company called PrimeSense. This is the outfit responsible for the Kinect technology people may be familiar with from XBox. It was Microsoft’s answer to the wildly popular and disruptive Wii from Nintendo, and it allows games to be played without a controller.
The technology works by blasting out a grid of infra-red dots, invisible to the human eye. These dots are then cast on anything the unit is pointed at, and sensitive stereoscopic cameras pick up how the grid is acting on a 3D surface that it’s hitting. This data is then combined with traditional camera colour and light data, to produce a high quality live 3D image.
Klick Labs has experimented with both the Xbox Kinect developer kit for Windows, as well as the Leap Motion. We use Leap Motion to allow deeper interaction with our virtual reality experiences, pairing it with Oculus Rift to allow the user to literally reach out and touch virtual environments.
The result is compelling as a novel interaction device. But what’s about to happen when these technologies get integrated into a mobile phone is something much more revolutionary.
As we know, the confluence of a myriad of technologies is what makes smart phones so revolutionary.
Steve Jobs famous 2007 demo of the first iPhone summed that up. as he repeated the now iconic phrase: “An iPod, a Phone, a revolutionary internet communications device!” It was the combination of the existing technologies, with the ease of use of a highly responsive capacitive touch screen that made the iPhone so revolutionary.
So what happens when you put 3D depth sensing and structural and environmental awareness on a mobile handset? You combine that with GPS technology, highly accurate motion sensing data, and Internet connectivity to unleash a completely new breed of experiences.
Imagine for a moment that you’re having trouble navigating your way through a labyrinthine hospital. With this kind of technology, you would be able to hold up your mobile phone and have the directions to your next appointment or critical procedure highlighted for you on the floor. Green arrows or footprints could guide you along your way, all the while dynamically avoiding obstacles that lie in your path. GPS taken to the next level! The ramifications for sight assistance and accessibility are exciting.
Or imagine you can hold up your mobile phone, and much in the same way you can now capture beautiful panoramic images, you could turn yourself around once and easily capture a complex 3D model of a room you’re in. What could you then do with a 3D model like this? Dragging and dropping that new IKEA couch into your living room to see if it’s a good fit comes to mind. Or in a clinical setting: testing to see if the Da Vinci robotic surgery assistant, or the new MRI machine would fit in your clinic as it’s currently configured. If your phone can see your surroundings in 3D, then you’re suddenly empowered to layer an unlimited digital library of objects and contexts onto it!
Google has been experimenting with this technology for a few years too, in the form of their Project Tango. Project Tango demos of large, clunky and obtrusive mobile prototypes have been wowing the tech crowd for a while now. But this recent announcement from Lenovo marks the very first time it will be available commercially, and in the hands of the consumer.
The potential for medical innovations is limitless, with accurate wound measurements being able to be recorded, or precise skin condition monitoring done at home by the patient themselves. For medical device manufacturers, it’s easy to imagine that a reliable digital handshake could take place between a mobile phone and a new medical device, such as a blood glucose meter or hearing aid. The need for QR codes disappears when you have a mobile phone that has such sophisticated 3D vision.
It’s a very exciting time for Digital Innovation, as the mobile technologies continue to push to bigger and more astounding heights. We suspect something like this may be announced from Apple in an upcoming version of their flagship handset. Perhaps they will finally integrate the technology they acquired 3 years ago from PrimeSense.
Whether they do or not, we don’t have to wait for them. We’ve got all the tools we need to build the next enhanced digital experiences, right now!
If you’re a digital explorer and curious about what’s on the horizon, and how we can use it to improve patient or HCP experiences for your customers, feel free to reach out to us here at Klick Labs to get the conversation started! We’re excited about the bright future, and you should be, too.