Mobile App Uses Sonar to Track Mid-Air Finger Movements

The mobile world has come a long way in a short span. These days, our smartphones are

connected to the cloud, among other mobile devices, such as smartwatches. Phone in pocket, you can answer an incoming call and feel like a secret agent as you talk covertly into your wrist.

Some of these smartwatches, however, can be as small as 39 mm by 33 mm. The diminutive sizes could pose a problem for the human finger.

Univ. of Washington researchers are looking to change that though with their FingerIO app, which translates nearby finger movements to a smartwatch or smartphone via sonar.

The app is the subject of a paper slated to be presented in May at the Association for Computing Machinery’s CHI 2016 conference, which will be held in San Jose, Calif.

FingerIO is inspired by another app developed by the Univ. of Washington called ApneaApp. Once launched, ApneaApp utilizes the smartphone’s speaker and microphone to send out sonar signals and listen to their reflections. Based on the changes in the reflected waves, which are caused by changes in breathing patterns, the app can discern whether a subject is having a sleep apnea event.

“After this work we realized the same idea might be useful in the context of human computer interaction, especially for wearable devices where the small screen and buttons make interaction difficult,” Vikram Iyer told R&D Magazine.    

The result: an app capable of translating gestures on a nearby surface to a mobile device. It even works when the phone is lodged in a pocket, or a smartwatch is hidden under a sleeve.

“Most smartphones and smartwatches already have speakers and microphones built in” and “sound waves don’t require a line of sight between the finger and the device,” said Iyer.

Using the hardware already in place, the researchers successfully demonstrated finger tracking with an average accuracy of 8 mm for the smartphone and 12 mm for smartwatches. Smartwatch users, additionally, can affix volume, or flip through a menu by simply flicking their finger near the device.

“The processing overhead is much lower than radar solutions that use much higher frequency radio signals,” added Iyer.

In the experiment with a Samsung Galaxy S4, the researchers used sound waves within 18-20 kHz. The type of signal used is called Orthogonal Frequency Division Multiplexing, which is typically used in wireless communication, according to the Univ. of Washington.  

“We’re in the process of improving our research prototype to move towards a marketable product,” Iyer concluded. “Bringing the technology to market wouldn’t require physical changes to phones as the microphones and speakers are already available, so a product version would likely run within a specific application or as a background process. We’re still in the process of exploring options for commercializing the technology.”

Source: http://www.rdmag.com/