Skip to main content

Apple buys PrimeSense: is 3D gesture support in the Mac or iPad's future?

Apple has acquired PrimeSense, a leading developer of 3D sensors that's best known for their role for creating the technology that powers the Kinect interface for Microsoft's Xbox 360 video game console. Does this mean that Macs or iPhones will be getting their own Kinects? Unlikely. But PrimeSense's acquisition may give us some insight to the future of gestures on the Mac and iOS platforms.

First of all, let's understand what Apple just bought for what's speculated to be up to about $360 million.

PrimeSense is a semiconductor company. They've developed 3D sensor technology; then they put that technology on silicon, so it can be incorporated into other products. PrimeSense in particular is a fabless semiconductor maker - that is, they outsource the actual manufacturing of their chip designs to semiconductor foundries, much like Apple does with the A7 processor used in the new iPads and iPhone 5S.

PrimeSense's technology enables computers to look at a scene in three dimensions. What makes it unique is PrimeSense's patented "Light Coding" technology, which uses a combination of near-infrared sensing and off-the-shelf image sensors like you what you find in a digital camera. PrimeSense's technology can differentiate the dimensions of a room from the location of a desk, and the movements of people within the environment.

You can see why this is perfect fodder for a video game system - if you've never used Kinect, the games made to work with it treat your body like a giant controller. Dance games track your movements to make sure you're in time with the song and matching the steps, for example.

While video games are the most broadly-used market for PrimeSense's products, they're not the only one. PrimeSense's products also have applications in healthcare: doctors and nurses can use it for patient monitoring, making sure that a patient isn't in distress. Touchless displays allow for data input without ever touching a physical surface like a keyboard or glass screen - more hygenic. The same use makes PrimeSense a great option for the classroom and boardroom, too, where teachers or managers might be using whiteboards.

PrimeSense has already sold millions of these systems to Microsoft, but their next generation technology is very promising in a different direction, including an embedded reference design created for use in tablets, TVs and PCs. They've already shown it working in an Android tablet. It's this new technology, and whatever else PrimeSense has cooking, that has me the most interested in what Apple might do.

Touch-driven interfaces like iOS, Android and Windows Phone 8 have their practical limits in terms of usability. Hybrid PC/laptops haven't exactly set the world on fire - why dirty up your laptop screen with finger smudges?

Imagine, then, Mac laptops that allow you to gesture in the space above your keyboard instead of using the trackpad. Data visualization, 3D modeling, and other tasks we use our computers for could be very different. So could the essential desktop user interface metaphor. If it's a virtual desktop.

If this all sounds too "Minority Report" to be like real life, you're wrong. You can already do this on a Mac or PC using the $79.99 Leap Motion Controller. More than 100 apps have already been adapted to or written to work with the Leap Motion Controller already.

This is a three inch long box that sits in front of your keyboard. It'd be even better if it was built in to the bezel of your MacBook's screen. Or in the keyboard of the iMac or Mac mini you're using.

Not to mention the inevitable flood of apps that would follow, through the respective platforms' App Stores.

But the killer app, as far as I'm concerned, is the application for this technology in the Apple TV. Again, going back to how the Kinect works with Microsoft game consoles, navigating the menus of your home entertainment system using touchless technology is certainly a lot better than fumbling for the tiny remote control that ships with the thing.

Gesture and movement-based games on the Mac aren't a new thing. iSight cameras have been available for the Mac for a decade now. Shortly after they were introduced a company called Strange Flavour published a game called ToySight that presaged what Xbox 360 users would be doing with their Kinect years later - you could play games, interacting with objects on your Mac's screen using a mirror image of yourself.

Those first iSight cameras were bulky, barrel-shaped devices that you'd affix to the top of your Mac's screen using sticky tape or clamps. Now iSight cameras are standard issue on most Apple devices (the Mac mini and the forthcoming Mac Pro are the two lone holdouts, since they don't have built-in screens), and they're so tiny and integrated that they're easy to miss unless you know what to look for. PrimeSense's next generation technology could also be integrated unobtrusively.

iSight cameras could become that much smarter through the integration of PrimeSense technology - the Mac or iOS device could have a better understanding of what you looking like and how you're moving in your environment. So there are practical applications for this that have nothing to do with games or modeling software - PrimeSense technology could just make Mac and iOS device use better, too.

If Apple's past acquisitions are any indication, it'll be some time before we see any practical applications of PrimeSense's technology hit the Mac or iOS platforms. In the interim, we can fantasize and speculate a bit about what our favorite applications might be.

So fire away in the comments - I'd like to hear how you imagine PrimeSense tech might work on Apple products in the future.

  • I'm imagining this in terms of motion control / virtual interface desktop computing application such as replacing complex keyboard commands - like opening / closing a hand like an iOS pinch or moving a hand from left to right to move from app to app instead of Command + Tab or (my personal favorite) giving the camera "the finger" force quits a misbehaving app.
    My other favorite use for this would be a supporting UI for an iWatch. Of course that would be more localized with finger gestures, but...
    *(I'd also like to be able to see what I'm typing, but te iMore app won't scroll my input field - my typing is hidden behind the keyboard. ) Sent from the iMore App
  • Oh, I almost forgot... The major missing cool-factor about sci-fi virtual/motion interface is the holographic feedback! Gotta have that! Sent from the iMore App
  • I always found the Kinect to be pretty crappy, but I am told the new Kinect is better. There is a reason why Microsoft dumped this technology.
  • Yeah, because they bought PrimeSense's competitor. PrimeSense was already working on higher resolution technology before this acquisition ever happened.
  • just curious on who they bought and why was their tech so much farther ahead than PrimeSense
  • They acquired both 3DV and Canesta. As far as being "farther ahead," PrimeSense hasn't been sitting still - the Kinect is based off a specific chip design PrimeSense created several years ago called Carmine, which it's been continuing to refine - but the company has also created what it calls the smallest 3D sensor on the market, which they've shown in a Nexus tablet as proof of concept.
  • thanks for that. the 3d sensor in a tablet seems pretty interesting and is something I could see Apple doing for an iPad. That said, I still find it telling that Microsoft was using Primesense technology and decided that it would be better to go in a different direction.
  • "In a different direction" meaning one that they own, in this case. Not to put too fine a point on it, but that's exactly what Apple is doing as well. ;)
  • I'm really excited to see what they do with this technology. There are so many possibilities!!! There is iSight camera integration and iTV. Then if a Apple TV App Store was ever to launch with this technology the app possibilities are endless. This technology is even more exciting than the fingerprint sensor. Sent from the iMore App
  • This acquisition just shows that Apple isn't being lazy innovating like what others are spouting. And this opens up a whole lot of potential that Apple can tap into (from simple commands for changing channels on the Apple TV to a more popular use for a Kinect-like gaming using iPhones as controllers, and the maybe to a more complex technology like a 3D-input device) and I believe it would be Apple, like with what they're doing with fingerprint-sensors, will be the pioneer to bring it into mainstream use. Sent from the iMore App
  • "This acquisition just shows that Apple isn't being lazy innovating" You do realize that purchasing innovation isn't really innovating, right?
  • Sure hope so. This is great for Apple! Sent from the iMore App
  • As long as it's better than the Leap Motion device. It has to make sense, not feel awkward. Sent from the iMore App
  • 3D gesture support for future Apple products sounds pretty intuitive and cool. I believe that there will be a lot of uses of this in all areas of life. Sent from the iMore App
  • I see this much more as a defensive move by Apple than a promise of new applications. I remember all the promises of liquid metal that in the end turned into a little silly tray ejector. And frankly, talking about dirty up your laptop screen with finger smudges in a tablet world is a lame argument.
  • This would be great! I don't think that they would have it (at first at least) for smaller devices like the iOS devices but more for the Mac and the Apple TV. I hope it would be for Apple TV first, using that silver remote to type is a MAJOR PAIN!! Even having to use the Remote app can sometimes be annoying. Gestures for my tv is always a plus!
    Freaky, a little, but a plus. Sent from the iMore App
  • I'd really like to seen this used in the next iteration of the Apple TV. I can see them doing what xbox one is doing except better and without the $500 price tag. Sent from the iMore App
  • To me this feels reminiscent of their acquisition of authentec and it feels so good! When I first heard about apple buying a fingerprint reader company I was left salivating at all the possibilities the new technology could bring to apple devices. I knew, however, it would be years before true integration and then a slow rollout of features for the new hardware (where we currently are). I think this situation will be nearly identical but I am definitely drooling about the possibilities 3D sensor tech can bring to the Apple TV, Mac lineup and even smaller touch devices like the phones and a watch. I think this could also play into their batman belt of security features. Couple solid facial recognition and a fingerprint reader and suddenly your devices become that much harder to brute force into and could possibly lead to iCloud profiles that follow YOU not your device. Now, when apple buys or "invents" holographic feedback to compliment this, don't even get me started. Sent from the iMore App
  • I think this tech might be a great addition to iBeacons. They would then be able to sense the actual room/mall/store etc. as well as track the people travelling inside. It could do things like give you a map of the mall you're in with your location. It could allow you to search for the store you want to get to and the ibeacons could lead the way. The tech is interesting and being able to manipulate the interfaces would be great but depending on how well it works and how natural it feels. If you're sitting there with your hands hanging in the air then it might not be fun after awhile.
  • I though the laser keyboard, hovering mouse, and even the Leap Motion device. All these devices have to get smarter before they can be accepted into the mainstream. Kinect is pretty damn close, but still clumsy, at least on the 360 (haven't tried the One). Hopefully, Apple will put them through its paces more than they did with Siri. Sent from the iMore App