A newly published Apple patent describes a mobile AR device that can detect objects and overlay them with information. The system would contain at least one wearable camera, as well as a screen and UI.

The invention is particularly focused on doing so in a power-efficient way, with possible usage scenarios including guided tours. When walking around a museum, the device would scan interesting objects and switch briefly to a high-power mode to display interesting information and play audio.

Fast detection and description of objects would likely come from a depth detection system. This could be similar to the iPhone 7 Plus’ dual camera, or a separate wearable device. Apple suggests the technology could have a use in future smartphones.

Representing Virtual Information in a Real Environment

Apple’s second patent describes a different AR technology. The invention would label points of interest in augmented reality whilst also paying attention to occlusion perception. One example is the overlay of information about nearby destinations like a shopping mall.

Apple patent


Using depth sensing technology, geometry models and positioning data, Apple would overlay the information in a way that makes sense. The text would account for the user’s viewpoint and wouldn’t get in the way of important objects. It would display on either transparent device or a smartphone screen. Once more, a depth-sensing technology is used to determine the user’s current environment, creating a depth map that virtual elements can be displayed on.

At this point, it’s unclear if Apple is planning to bring this technology to market. However, recent rumors suggest the company is working on some kind of AR device. The patents were filed by AR specialist Metaio before Apple acquired it in 2015, but that doesn’t stop the company implementing such technology in later iPhones.

You can read the patents for yourself here.