This post is simply an informative piece and has no political bearing.
With the implementation of the SafeCity Project in Mauritius comes the adoption of Facial Recognition Technology (FRT), and as per a local newspaper, FRT implementation is undergoing tests. Discussion with other people on FRT has made me realised that while people are adamant that FRT is pervasive and dangerous, some don’t understand the basics behind FRT. Without going in-depth into FRT and its privacy concerns, let’s see what it is.
What is FRT?
We’ve seen FRT in sci-fi movies where protagonists and otherwise use a picture of a suspect where the facial details of the latter is analysed. Facial details such as the size of pupils, eye colour, eyebrow length, facial contours and nose length are cross-checked against a database containing predefined details of different persons until a match is found.
This is not far from how FRT works. While EFF (Electronic Frontier Foundation) defines Face Recognition (FR) as “..a method of identifying or verifying the identity of an individual using their face..”, FRT is about using computer vision algorithms or techniques to identify a person or persons either from images or from videos.
Typical FR algorithms have 4 basic phases:
- Face detection
- Image normalisation
- Feature extraction
- Recognition or classification of the extracted features to identify the individual
In the first phase, processing power is utilised to find out which pixels of an image correspond to a face or the spatial and geometric distribution of facial features that make up a face, and that part is cropped. Once the face detection phase is done, the image is standardised to account for illumination, shadows and size. Finally, the cropped and standardised image is converted to a mathematical and digital template which is compared against biometric templates stored in a database. When a match is found, it is generally scored to determine its accuracy.
This is how typical FRT operate. There are many variations as there are currently open-source tools or vendor tools and libraries that are available to develop and implement FRT.
FRT applications around the world
Personally, I use FRT to unlock my mobile phone and laptop. It’s convenient and fast as my mobile’s tempered glass does not work well with the on-screen fingerprint sensor. And one direct consequence of using Windows Hello authentication on my laptop was actually forgetting my password after a while. In other places, FRT is being used to unlock cars or mobile phones, in targeted advertising and security and safety purposes. For this blog post, I’ll focus mostly on some countries using FRT for safety and security.
In the US, Clearview AI augmented law enforcement agencies with FRT. Clearview AI scrapped images from social media sites, which were in turned use to identify an individual. This company had created a database containing more than 3 billion images. After a while, it was known that this tool was made available to private companies as well.
In China, FRT is used mainly for domestic surveillance and to arrest criminals. In some places, FRT would also enable identifying jaywalkers and they would instantly be fined via instant messaging. Another use of FRT was to identify a minority of people and their ethnicity.
In UK, the South Wales Police make use of AFR which is an applied FRT on real-time images obtained from a camera attached on top of a van. The captured images are compared to a custody database of around 500,000 people on a watchlist.
Now, in Mauritius we are using FRT:
Cette technologie sera utilisé pour suivre le mouvement des criminels récidivistes, les personnes recherchées et celles portées disparues.” Les images de ces catégories de personnes seront téléchargées sur la base de données du système par des policiers formés et autorisés.Excerpt retrieved from Safe City : test en cours pour la reconnaissance faciale
The system automatically generates alarms when detecting suspicious personnel. Facial recognition accuracy exceeds 95 percent, and license plate recognition accuracy exceeds 99 percent.Excerpt retrieved from Building a Safe Mauritius, the Inspiration for Heaven
These two quoted texts show that FRT promises an enhanced public safety and security. While this is great for an urban or smart society, we need to also understand issues that may or may not come with adopting FRT.
About using FRT for safety and security
CCTV cameras are generally used to identify a suspect in case of theft or criminal activity after days or weeks. Now, when adding FRT to that equation, the matter of identifying a suspect can be done in a matter of hours. This is a great benefit towards public safety and security but there are concerns of adopting FRT in a democratic society.
Using FRT requires understanding the social aspects and possible influence on the public and that deployment of FRT often comes without transparency, guidelines and policies. Before reflecting on whether FRT impedes the right to privacy, it is a scary thought that in the wrong hands, FRT can be misused for authoritarian needs.
FRT can be viewed as a surveillance tool as it stores information pertaining to recognised individuals digitally and for a set amount of time. These digitised information can reveal the activities, location and routine of unsuspected individuals and thus, revealing their sensitive information.
While some individuals might think of wearing masks to mislead FRT, a Japanese company claims it can identify people wearing masks. Or even buy a pair of special glasses or change their fashion style to avoid being recognised or identified, there is also Gait Recognition techniques. Gait Recognition is basically recognising people from the way they walk and it is quite accurate.
Coming back to FRT, FRT in the US led to a wrongful arrest in 2020, and this shows the danger of a skewed dataset an FR algorithm was trained on. While the algorithm is said to exceed 95 percent accuracy, it would be an interesting idea to see how it fares on a Mauritian-faces dataset given that Mauritius is a multi-ethnic society.
FRT adoption should lead to questions regarding its impact on privacy and racial equity, resulting in research and publications on the FRT usage in Mauritius. Elsewhere, there are whitepapers, recommendations and policy guidelines on FRT as well as its accuracy and performance.
Existing Recommendations and Policies
Currently, one can train an FR algorithm on a dataset such as Labeled Faces in the Wild with an accuracy of over 90 percent. However, it is important to understand that even if FRT performs well in one context, it may not reach the same performance in another diverse context.
A report from Stanford University discusses the challenge of FRT performance as well as recommendations. Two recommendations that stand out are the transparency of imagery and documentation.
Vendors should provide users and third-party evaluators meaningful access to testing imagery so that they can conduct independent validation of in-domain performance.
FRT vendors and developers should ensure their models are created in a way that is as transparent as possible, capable of being validated by the user, and well documented.Excerpt from “Domain Shift and Emerging Questions in Facial Recognition Technology“
While adoption of FRT in Mauritius is slowly taking shape, the ethical and societal impacts should not be disregarded. Proper research is a must as well as vendor disclosure on their FRT testing and benchmarking.
FRT market is expected to double by 2026. Outside of enhanced public safety and security, FRT can be used in other ways and contexts. Personally, I think we might see Pay-by-Face where one’s face replaces their bank cards, in the next 10 years. Or retail shops able to adopt FRT for targeted advertising or an automatic ordering of menu items whenever you walk in a restaurant.
To conclude, I should really adopt a skincare routine now.