Scan your face to pay, scan your face to enter the subway, scan your face to punching… the era of traveling the world by just scanning your face is approaching. With the help of facial recognition, our lives become more flexible. How to verify the function, security, and recognition rate of facial recognition technology turn out to be the key topic in software testing. In this article, we will talk about facial recognition testing combining with a practical example.

So, if you want to know:

1. What is facial recognition?

2. What are use cases for face recognition?

3. How to test the facial recognition software?

Let’s get started.

What is facial recognition?

Facial recognition technology is a kind of biometric recognition technology based on facial features. It uses a camera to collect images or video containing human faces and automatically detects and tracks human faces. Facial recognition technology is divided into three parts: face detection, face tracking, and face comparison. We can detect the face of a living body in a complex background and scene through facial recognition and perform dynamic target tracking on the detected face in real-time. And compare the sampled facial image with the inventory face image sequentially and find the best match.

What are use cases for face recognition?

There are many use cases for facial recognition technology, such as the security management of enterprises and residences, identification and management of electronic passports and ID cards through biometric recognition, the government sectors use facial recognition to trace criminals, the bank ATM and computer login, etc. In general, face recognition is mostly used as identity verification for users which is usually divided into 1:1 face recognition and 1:n face recognition, etc. For example, 1:1 face recognition is mainly used to determine whether the person in the two photos is the same person. The model will give a score with the rule that the higher the score, the greater the possibility of being the same person. In reality, a threshold value M will set. If the score value is greater than the M, then the result is positive. In the same way, 1:n face recognition means that one face performs n-time 1:1 face recognition in the photo library, and finally matches the photo with the highest score in the face library.

How to test the facial recognition software?

1. The test standard

Take the facial recognition transfer as an example, we will test the connectivity, message format, invalid parameter verification, and return code, etc. Besides, facial recognition uses biometric technology and artificial intelligence algorithms and models. The transfer speed and result will be affected by the light, the background, the hairstyle, glasses, and other decorations. Although the model is built and trained perfectly, 100% accuracy is currently not guaranteed. Therefore, the standard divers in different sectors, the face recognition accuracy rate and pass criteria are determined by each domain. The general face recognition accuracy rate of the industry is greater than or equal to 99%. When the interface of the face recognition model can be successfully invoked, and the accuracy rate of recognition meets the standard requirements, it is deemed to have passed the test.

2. Test process

The following figure shows the main test process of a facial recognition model. Firstly, we need to prepare a certain amount of test data of two parts: one is a set of test materials, such as a pair of face photos, the other one is to tag label for the materials according to the model to be tested, that is, the matchup of face photos. After the data preparation is done, we can start testing.

A message requesting the service of the model is constructed for each test material and sent to the “face recognition service”, and compares the returned result with the label. After all the test data are been tested, calculate the accuracy rate, recall rate, misrecognition rate, and other indicators based on the result set to evaluate the tested model.

The focus of different models and use case divers, and the indicators are also different. In order to improve test efficiency, batch automatic testing is usually used by writing test scripts.

The test data is very important because, on the one hand, the number of data will influence the result. Too little data is prone to contingency. The result of the test sample has a greater impact on the model evaluation index. On the other hand, the test data should be “real”, fitting the real situation including the positive and negative examples. In order to be close to the use cases, you can prepare a photo collection(A) of daily life and a photo collection(B) of identification photos. When choosing the daily life photo, the ages, hats wearing, glasses-wearing, etc should be considered. And then tag the label of the two collections.

Considerations for facial recognition data

Light and dark

Hairstyle

Whether to wear glasses?

Whether to wear a hat?

Whether the face is blocked?

Is the face facing the camera?

Background-color

Whether there are other faces in the background?

Face synthesis

Other

Randomly select one photo from collection A and collection B, and send a request message for 1:1 facial recognition. A total of N*N request messages will be sent, of which two photos of N requests are the same person, which is called a positive example. And the photos in the remaining N*(N-1) requests are different people, which is called negative examples. Use the “N*N message” to request facial recognition, and compare the returned result with the tags. “misrecognition” refers that the 1:1 facial recognition model returns a high score, exceeding the threshold M, for two different person’s photos, that is, the negative examples is taken as the same person. The misrecognition rate is the percentage of misrecognized negative examples in total negative examples.

“Pass” refers to the 1:1 facial recognition model return back a high score for two photos of the same person, that is, the positive example is judged to be the same person. “Pass” means that the 1:1 face recognition model for two pictures of the same person returns a higher score, that is, the positive example is judged to be the same person; the “pass rate” refers to the percentage of the passed positive examples in the total positive examples. We hope that the misrecognition rate is 0%, and the pass rate is 100%. While the misrecognition rate and the pass rate are inseparable from the threshold M. if the threshold M is set low, the extreme case is set to 0, then the photos of different persons can be easily judged as the same person. The higher the threshold M is set, the less the possibility of misrecognition and passing, if the extreme case is set to 100, although the photos of different persons can be effectively recognized, the photos of the same person could not be recognized either, which influence the user experience. Therefore, there is a trade-off between the two when setting the threshold M. Since the threshold M has not been determined during the test, and considering 1:1 facial recognition is used for identity verification, which requires lower error-tolerant, the misrecognition rate can be confirmed first, and then the corresponding threshold M, finally the pass rate. The pass rate can be used as the evaluation index of the model. The calculation of test evaluation indicators is usually realized through automation, and test scripts are written to realize the whole process of constructing request messages, simulating sending, receiving and analyzing, recording results, and calculating evaluation indicators.

The above is the specific test method of face recognition technology, but actually, for different use cases and model characteristics, it is necessary to consider the test data and the selection of model evaluation indicators in combination with actual conditions.