As a sort of followup to yesterday’s post, here’s another surprising thing a camera can do:
First, the microphone detects that a person is entering a PIN. On many apps, the device will vibrate each time a number is tapped. That vibration creates a sound that is picked up by the microphone, which lets the malware know that a “touch event” is happening — in this case it is the entering of a secret PIN.
Then the camera takes over. The camera isn’t looking for reflections in your eyes or triangulating what numbers you’re looking at while typing in the code. The researchers use the camera to detect the orientation of the phone and determine where the user’s finger is on the screen. On-screen keypads typically display number in a standard order, so if the program can tell where a finger is tapping on the screen based on how the person is holding it, it can deduce what number is there.
This was presented last week at an international cybersecurity conference in Germany. Fortunately, nobody is exploiting this method yet. The Cambridge researchers were just showing that it’s possible. (Here’s a PDF of their presentation).