What is Face ID, how does it work, can the police use it against you, will glasses and beards trip it up, and everything else you need to know about Apple's new biometric sensor for iPhone X!

Face ID is Apple's new facial identity scanner. It replaces Touch ID, Apple's fingerprint identity scanner, on the next generation iPhones X. But change is scary and the new is also always the unknown. That's why we're already seeing a lot of stress and sensationalism — just like we did with Touch ID.

Well, let's clear that up.

September 15. 2017: Apple comments on Face ID

Apple has sent iMore the following comment on Face ID:

Our teams have been developing the technologies behind Face ID for several years, and our users' privacy has been a priority since the very beginning.

Face ID provides intuitive and secure authentication enabled by the TrueDepth camera system and the A11 Bionic chip, which uses advanced technologies to accurately map and match the geometry of a user's face. Face ID data never leaves the device, is encrypted and protected by the Secure Enclave.

We've tested Face ID on people from many countries, cultures, races and ethnicities, using over one billion images to train our neural networks and defend against spoofing.

We're confident that our customers will love using the feature and find it an easy and natural way to unlock their iPhone X. We will offer more details on Face ID as we near the product's availability.

Face forced

One of the biggest areas of fear, uncertainty, and doubt surrounding a potential Face ID facial identity scanner is that it will make it easier to law enforcement and government agencies to gain access to our devices.

(That a significant segment of humanity is more concerned about illegal search and seizure by law enforcement agencies than the criminals they're meant to protect us from should embarrass and appall governments around the world, but that's a different editorial for a different day.)

"Easier" is tough to parse, though. A few years ago, when Touch ID was introduced, we saw similar concerns: That someone could wait for you to fall asleep, or incapacitate or restrain you, and then simply touch your finger to the sensor to unlock your phone. if they didn't know which finger(s) (or, humorously, other body part) you'd used for for Touch ID, there was a chance they could trigger Passcode lockdown after repeated failed attempts, then simply torture you for the answer and/or passcode, but the tendency towards convenience made the thumb the likely digit in almost all cases.

What happens in the real world can always be different than what is later ruled admissible or inadmissible in the courts, and illegal search and seized didn't begin or end with Touch ID and won't begin or end with Face ID. Likewise for individuals on the criminal side who want access.

Face ID, by default, requires you to be looking at iPhone X to unlock. So if you close your eyes and look away, someone would have to force you to open your eyes again for Face ID to unlock. Just like they'd have to force your thumb onto Touch ID to unlock. It would also make it very difficult to have Face ID unlock if you're sleeping, which is currently much easier with Touch ID.

For people who require security, a strong password is available. For those who want convenience, biometrics. Don't want to risk your finger or face being used against you, don't use your finger or face for unlock. (Or disable it by quintuple-clicking the Side button before going places or entering situations you don't trust.)

In a perfect world, Apple would let those who want even more security to require both password and biometrics — and a trusted object like Apple Watch as well. Hopefully, Apple is working on that.

Hollywood tricks

Another source of concern has been the use of photos, videos, makeup, masks, and evil twins — perhaps even and up to plastic surgery — to gain access through the Face ID system.

Part of the reason for this is too much Hollywood. The other part is poor implementations of face scanning to date, similar to how we had poor implementations of fingerprint scanning before Touch ID.

Yet Hollywood is who Apple turned to to help train the Face ID system. Photos were never a major concern because of how Apple scans faces in 3D space. But, Apple had practical effects artists create makeup and masks to try to fool Face ID, then used that makeup and those masks to train the neural network on the Apple A11 Bionic chip to prevent against just that type of attack.

Video spoofing attempts, which can be much harder to detect than static, single-dimensional photos, was similarly trained against.

For evil twins, Touch ID always had a 1/50,000 chance a random stranger's fingerprint pattern would match yours enough to gain access. For Face ID, that chance drops to 1/1,000,000. Still, with twins, the chances are much higher. There are ways to look at the data pulled into the system to detect against twins, triplets, etc. based on differences that have developed over time. But we'll have to wait and see how that works out in the real world.

Lights, cameras, "selfie"

People have also been concerned about the utility of Face ID at night or in the dark, at angles like when the phone is flat on a table, and when in use, like for authorizing Apple Pay.

For no or low light, Apple is using a "flood illuminator" to make sure your face remains scannable to infrared. Then the "dot projector" hits you with 30,000 points for identification purposes. The camera system then captures the data, converts it into math, and pushes it through the Neural Network block on the A11 Bionic chip to see if what was captured matches what was registered. If it does, you're authenticated.

The camera system has a fairly wide field of vision so, depending on the exact angles, you can tap the screen to wake iPhone X, and be scanned by Face ID, under a variety of conditions.

You can also disable "attention", which requires you to be looking directly at iPhone X for Face ID to unlock, but for most people leaving it on is better for security.

For Apple Pay, the process is largely unchanged. Double-clicking the Side button replaces double clicking the Home button to invoke Apple Pay, glancing at Face ID replaces holding on Touch ID, and the tap on the NFC payment terminal remains the same. (If you typically invoke Apple Pay buy tapping the NFC terminal first, you may have to pull back to authorize with Face ID, depending on the exact positioning.)

Glasses, hats, scarves, beards, and more

Because Face ID uses the Neural Network block in Apple's A11 Bionic system-on-a-chip, it continually learns and adapts to how your face, facial hair, hairstyle, and facial coverings vary and change over time.

With the "dot projector" making up to 30,000 points available, there's a lot of data for Face ID to work with. Apple also built and tested the system to make sure it works under the widest varieties of conditions possible.

That includes if you have your glasses on or off (though sun glasses might obscure "attention" mode, which is on by default). If you change your hair style or color. If you grow a mustache and/or beard and then shave them off. If you're wearing religious or climate-based facial coverings (though full-on Canadian-style ski masks and goggles might obscure too many points of identification to be useful — I look forward to testing that!)

There may be cases where too much changes at once and Face ID fails to recognize you. If/when that happens, the system will simply kick you back to Passcode, just like Touch ID does when the moisture level (or wetness) of your finger was different enough to prevent a scan.

Accessibility on-board

For people who have low or no vision, Face ID will guide you through the setup and authentication process, including cueing you on positioning.

You can also disable "attention" so, even if you're not looking directly at iPhone X, it will still unlock when enough of your face is within the system's field of vision.

Face ID limitations

All that said, there are some limitations with Face ID that could affect people. The biggest is that, unlike Touch ID which allows for the registration of up to 5 fingers, Face ID currently only allows for one face. That means no sharing easy access with family members, friends, or colleagues.

Also, Face ID does need to have your face within view of the camera system to unlock. Touch ID can work regardless of the camera system orientation. That means, in some situations, you will have to adjust your position for Face ID to work.

Biometric bias

It's important to remember that biometrics are a convenience. Before Touch ID, many people wouldn't even go to the trouble of using a 4-digit passcode to lock their iPhone. Now, thanks to the convenience, many of them do. It's nowhere nearly as secure as using a long, strong, pseudorandom password for every unlock, but most people won't ever be willing to go through that process daily, much less hundreds of times a day. So, biometrics.

No doubt we'll go through all the same stress and sensationalism with Face ID we went through when Apple introduced Touch ID back in 2013.

We'll see people with CSI-level resources making dummy heads to try to fool it — and it's quite possible some will succeed in the same way they succeeded with Touch ID. We'll see headlines about how your face can now betray you, and goofy examples of facial contortions recommended to avoid forced scanning. We'll see... everything we typically see whenever Apple introduces any new feature.

And then we'll use it, forget the fuss, and move on. Just like we did with Touch ID. Just like we do every year.