App Blog

iPhone X facial recognition

iPhone X's Facial Recognition - How does it work, is it safe and what does it mean for app development?
November 18, 2017

With the iPhone X Apple has taken facial recognition to the next level. We take a look at Face ID - what it is and what it means...


What is it and how does it work?

Face ID replaces the fingerprint reader in all aspects, so it’s not just used to unlock the phone. It also works for Apple Pay and identifies you in a multitude of apps that require it. It uses specialized hardware which Apple calls the “TrueDepth camera”. Depsite the name, this is actually a set of cameras, ambient light sensors, dot projector and an infrared laser, all located on the iPhone X’s controversial notch.

All this is necessary because the facial recognition technology goes far beyond recording your face to recognize you. Otherwise it would be enough to put a photo in front of the phone to deceive it, or conversely it wouldn’t recognize you if you wore a hat or put on some sunglasses.

The first time you use Face ID you need to perform a 3D scan, which basically consists of looking at the camera and turning your face side to side. The infrared illuminator allows Face ID to do this scan in the dark.

An algorithm of artificial intelligence, harnessing the power of the iPhone X's new A11 Bionic processor, generates a 3D image of the face that registers forms, volumes, skin markings and other biometric data. The phone's neural processing  learns to know your face and identifies you even if you change your hairstyle, wear glasses or a hat, or let your beard grow.
Components of Apples TrueDepth camera

Is it safe?

One of the reasons that Apple offered to justify Face ID is that the face is much safer than fingerprints as a method of identification. With Touch ID, a fingerprint is wrongly identified 1 out of every 50,000 times. With Face ID, the rate drops to 1 in 1,000,000 times. This is why:

For a start, the depth map of your face is stored and encrypted in a secure enclave of the iPhone X hardware, so it’s never sent over the internet or processed in the cloud.

Nor will Face ID activate randomly, such as when you look at the iPhone with your eyes closed or sideways. You must "pay attention", according to Apple, that is, look at the camera with your eyes open to activate Face ID.

The 3D projection of 30,000 points prevents deception of Face ID with a photo. In fact, Apple has even tried 3D masks created by the best Hollywood specialists, and nobody has been able to cheat Face ID.  In spite of everything, Phill Schiller recognized that problems can arise with people who share many genetic similarities. In a joking tone, he recommended "keep the phone away from your evil twin".

New applications

More exciting than its security potential is facial recognition’s use in new and exciting applications. So far, two broad types have appeared: animojis and augmented reality.

Animoji monkeyAnimojis: The iPhone X uses facial recognition to turn your head movements into the movements of animated emojis. The phone records your muscle movements to clone your expressions in – so far - 12 different animojis: an alien, a hen, a little pig and of course the famous smiling poo... and the result is spectacular. You can use these emojis in the Messages application – and already a trend of “Animoji Kareoke” (lip syncing songs as your favorite animoji) has emerged.

Augmented reality on iPhone XAugmented reality: The spatial scanning provided by the TrueDepth camera, coupled with the processing power of the A11 Bionic processor, has allowed Apple to bring top tier augmented reality to the iPhone. In the keynote, Tim Cook unveiled the first example of its use - an exclusive game called The Machines, which recreates a battle scenario on an empty table.

So too glasses brand Warby Parker is cleverly using the iPhone’s camera to scan a user’s face in order to determine the styles of glasses that would best suit him or her. While the company had previously provided a feature allowing the user to see how different sunglasses would appear on their face, this latest improvement takes the user experience to the next level.

And of course, someone has already developed a fart app which makes your breath appear as a fart.

It’s not hard to see that augmented reality presents exciting possibilities for developers and businesses. Businesses could show potential customers their house painted or with a new couch, their car with tinted windows or what they would look like in new clothes. Indeed, the iPhone X’s augmented reality promises an exciting time for app development and we’ll stay right at the forefront.