The U.S. Department of Homeland Security tested technology that can recognize masked faces

The U.S. Department of Homeland Security (DHS) is testing facial recognition technology that would allow it to identify people wearing masks with what it says is high accuracy. That’s according to a press release published this week, which reveals that the DHS’ Science and Technology Directorate (S&T) conducted a pilot as part of its third annual Biometric Technology Rally at a facility in Maryland.

While a number of companies claim to have developed technologies that can identify mask wearers, like the cloth masks designed to minimize the spread of viruses, the DHS proposes applying it to screening processes at airports and other ports of entry. For example, U.S. Customs and Border Protection’s (CBP) Simplified Arrival program, which recently expanded to airports in Las Vegas, San Francisco, and Los Angeles, uses facial recognition to verify the identity of airline travelers arriving in the U.S.

According to the press release, S&T’s in-person event included 10 days of human testing with 60 facial recognition configurations, six camera-based face and iris recording systems, 10 matching algorithms, and 582 “diverse” test volunteers representing 60 countries. Each system was evaluated based on its ability to “reliably” take images of each volunteer with and without masks and minimize processing time.

With masks, the DHS claims the median accuracy of all systems was 77% identification rate and the best-performing system correctly identified people 96% of the time. That’s roughly in line with findings from the U.S. National Institute for Standards and Technology earlier this year, which found that the best performers of over 150 commercial facial recognition algorithms had a false match rate of about 5% when high-coverage masks were worn. But the DHS concedes that performance varied widely, from 4% accurately identified for the worst-performing algorithm. Moreover, on average, cameras couldn’t capture photos for 14% of the masked volunteers and couldn’t find a face for 1%. (One commercial iris recognition system S&T benchmarked failed to capture photos 33% of the time.)

The DHS initially raised concerns that face masks meant to protect against the spread of the novel coronavirus might interfere with facial recognition technology, according to a report from The Intercept. “[This technology] could reduce the need for people to remove masks at airports or ports of entry … thereby protecting both the public and frontline workers during the COVID-19 era,” the DHS said in the press release this week.

As early as 2016, CBP, the largest federal law enforcement agency of the DHS, began laying the groundwork for the program of which Simplified Arrival is a part: the $ 1 billion Biometric Entry-Exit Program. Through partnerships with airlines like Delta and JetBlue, CBP has access to manifests that it uses to build facial recognition databases incorporating photos from entry inspections, U.S. visas, and other U.S. Department of Homeland Security corpora. Camera kiosks at airports capture live photos and compare them with photos in the database, attempting to identify matches. When there’s no existing photo available for matching, the system compares the live photos to photos from physical IDs including passports and travel documents.

As the CBP explains, with Simplified Arrival, travelers on international flights pause for photos at primary inspection points after debarking from the plane. If the alternative photo-matching process fails, they undergo the traditional inspection process.

CBP says that to date, more than 57 million travelers have participated in the Biometric Entry-Exit Program (up from 23 million as of March 2020) and that over 300 imposters have been prevented from illegally entering the U.S. since September 2018. But Simplified Arrival and other pilots under the umbrella of the Biometric Entry-Exit Program are inconsistent, opaque, and potentially discriminatory.

While travelers can opt out of Simplified Arrival by notifying CBP officers at inspection points, a Government Accountability Office audit found that CBP resources regarding the Biometric Entry-Exit Program provide limited information and aren’t always complete. At least one CBP call center information operator the GAO reached in November 2019 wasn’t aware of which locations had deployed the technology, and some airport gate signage is outdated, missing, or obscured.

It’s also unclear the extent to which CBP’s facial recognition might exhibit bias against certain demographic groups. In a CBP test conducted from May to June 2019, the agency found that 0.0092% of passengers leaving the U.S. were incorrectly identified, a fraction that could translate to a total in the millions. (CBP inspects an estimated over 2 million international travelers every day.) More damningly, photos of departing passengers were successfully captured only 80% of the time due to camera outages, incorrectly configured systems, and other confounders. The match failure rate in one airport was 25%.

Despite the controversial nature of the CBP’s ongoing efforts, the U.S. Transportation Security Administration recently announced that it, too, would begin piloting checkpoints at airports that rely on facial scans to match ID photos. The White House has mandated that facial recognition technology be in use at the 20 busiest U.S. airports for “100 percent of all international passengers” entering and exiting the country by 2021.

CBP recently proposed a rule that would authorize the collection of facial images — which can be stored for up to 75 years — from any noncitizen entering the country. A coalition of civil rights groups led by the American Civil Liberties Union filed an objection alongside The National Immigration Law Center, Fight for the Future, the Electronic Frontier Foundation, and twelve others.