Apple Advances Presence technology that ID's Individuals without passwords & understands both Voice & Gesture Commands – Patently Apple
Back in 2009 Patently Apple posted a patent application report titled “Apple Introduces Presence Detection for Notebooks, Desktops & TV.” The invention covered employing a variety of radiation emissions to determine when you enter the hardware’s field of detection and, in response your entrance, activate your hardware. The image below is from the 2009 patent.
Then came Google advancing the technology and patenting a radar system (01, 02, 03) called ‘Soli’ that they introduced with Pixel 4. It also recognizing hand gestures.
Apple’s new iPad Pro now uses a similar, perhaps superior technology called LiDAR with the iPad Pro’s new cameras. Yet for Apple’s latest patent published today by the US Patent & Trademark Office titled “Electronic Device with Co-Located Independent Radar Transceivers,” Apple has chosen to emphasize the use of radar transceivers that could acknowledge presence with greater accuracy along with being able to accept both voice and hand gestures to control devices in an environment/a room such as a MacBook, iMac, HomePod, Apple TV and other IoT devices.
Apple’s patent FIG. 1 below is a block diagram illustrating an example of an electronic device performing radar measurements.
At a given time, at least a subset of the independent radar transceivers #112 may transmit radar signals #114 and perform the radar measurements on the returned or the reflected radar signals to determine the location of one or more objects, such as an object #116 (e.g., an individual, furniture, a wall or boundary, etc.) in an environment #100 (such as a room) that includes electronic device #110. For example, the transmitted radar signals 114 may include pulsed radar signals and/or continuous-wave radar signals.
Depending on the device being accessed and controlled, there could be 4 or 8 fields of view. In FIG 9 above we see an example of a device that contains 8 fields of view that can be used to identify an individual in a room.
An individual (adult or child) can be identified at least in part on the radar measurements by determining a vital sign of the individual and/or a medical condition of the individual.
In some embodiments, the electronic device may determine based at least in part on the radar measurements: one or more dimensions of environment 100, an acoustic property of object 116 or environment 100 (such as absorption or a reverberation time), and/or an orientation of electronic device 110 in environment 100 shown in patent FIG. 1.
In these ways, the measurement techniques may allow and electronic device to determine the location of and individual (#116 of FIG. 1), including when they are moving and/or are static. Moreover, by identifying the individual, the measurement techniques may facilitate an improved user interface for controlling the electronic device (#110 of FIG. 1).
For example, based at least in part on their identity, the individual may be automatically authorized to control an electronic device.
Furthermore, stored user preferences and/or historical actions of the identified individual may be accessed, so that electronic device may automatically provide a user experience that is customized to the individual.
Additionally, based at least in part on the determined location, the electronic device may directionally listen for spoken commands from the individual or may be able to determine when the individual is providing a command for electronic device (e.g., by detecting a gesture or using the measured radar signals).
This may allow the individual to control the particular electronic device without using keywords, such as a wake word to activate electronic device.
Instead, the electronic device using Apple’s radar system may provide an ever-present sensing capability. In some embodiments, the determined vital sign(s) and/or the medical condition(s) of the individual facilitates an additional service or application.
Alternatively, or additionally, the determined dimension(s), acoustic property and/or orientation may allow sound (such as a sound field) output by speakers #122 (of FIG. 1) in or associated with the electronic device to be tailored or customized to conditions in the environment. These capabilities may improve the user experience when using the electronic device.
In some embodiments, the electronic device #110 may be an iPhone (smartphone), MacBook (a laptop), an iPad (a tablet), a HomePod (a wireless future wireless speaker), iMac (desktop computer) or an IoT device.
Lastly, Apple notes that “The presence estimation function may define the location (such as the range) of a non-static object,” like an individual. In order to make the presence estimation function smooth, time and range averaging may be performed.
Apple’s patent application 20200088870 that was published today by the U.S. Patent Office was filed back in Q3 2018. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
Jouya Jaddian: Senior Engineering Manager. Experience in radio/light detection and ranging
Vaneet Pathak: Manager, Wireless Design
Mikheil Tsiklauri: Senior Algorithm Scientist – Apple Special Projects – Developing Radar based systems for localization and mapping.