NSW Police Stop Using Ineffective and Racially Biased Facial Recognition Identification System

New South Wales police minister Yasmin Catley revealed on 8 April 2025 that the New South Wales Police Force recently ended its use of the PhotoTrac Suspect Identification System, which involved facial recognition software produced by German company Cognitec, and while neither the politician nor the law enforcement agency provided reason for discontinuance, the program is notorious for its ineffectiveness generally, and for producing false identification matches when it comes to racial minorities in particular.

The question regarding the program had been taken on notice by the minister during a 12 March NSW budget estimates hearing, which was in response to a series of queries primarily from NSW Greens MLC Abigail Boyd, who’s been tracking developments relating to facial recognition technology use by law enforcement, along with its increasing reliance on article intelligence (AI).

Technologies, like facial recognition, are increasingly being incorporated into policing systems, as they’re designed and marketed to cut down on the time it takes to achieve tasks. But while this shift towards reliance on machines and software to achieve certain work may expedite the completion of tasks and cut down on human error, these high-tech methods are prone to make their own mistakes.

Facial recognition technology has long been known to produce racially biased outcomes, as it’s more likely to misidentify people of colour as suspects, and these false results disproportionately apply to women as well.

The simple explanation for these misidentifications is that these technologies are usually being developed by white men, who then train their systems on majority white male samples, and hence, it’s more accurate in its matching of these types of people.

The obvious issue with the increasing police application of such technologies is that while it might free up officers to concentrate on other tasks, these software packages are generating leads in respect of innocent individuals based on their physical similarities to an offender, and in this manner, these systems are also reproducing the same racial prejudices local policing cultures already harbour.

Inherited racial bias

Boyd has been on the warpath when it comes the ever-increasing reliance of the NSW Police Force on emerging technologies to generate leads, especially as while this utilisation of this technology is rapidly growing, its major deficiencies are too increasingly being recognised, and yet the NSW police has continued to apply it.

Since February, PhotoTrac has been removed as part of the NSW suspect identification system (SIS), which is an initiative that can match CCTV images with pre-existing mugshots of prior offenders already in the possession of NSW police. Although, while state law enforcement has now dropped Cognitec’s long critiqued PhotoTrac, it is continuing to use other facial recognition software.

During 23 February 2024 budget estimates, Boyd raised the fact that the US-based National Institute of Standards and Technology had tested 189 facial recognition systems from 99 developers, including Cognitec, and the organisation had found that these systems misidentify people of colour 10 to 100 times more often than they misidentify Caucasians.

Further, the US institute found in August last year that the latest PhotoTrac algorithm was continuing to misidentify West African people almost seven times more often than Europeans. However, when it came to the NSW police at that same point in time, our state’s agency was continuing to employ the 2011 Cognitec algorithm, which was the original version of the technology our state purchased.

So, it was against this backdrop that Boyd asked NSW police deputy commissioner David Hudson during February 2024 estimates, whether he and his police force were confident that the system they were using at that time was not exhibiting or generating racially prejudice results.

“Yes, we are,” the deputy commissioner insisted. “We don’t use facial recognition or facial matching services as the only evidence that we will use to charge someone. We use it as an investigative tool to give us an indication of matching, say, a CCTV photograph with our offender photographs.”

The various budget estimates exchanges between Boyd and Catley, during her tenure as police minister, have revealed that over the 12 months of 2022/23, PhotoTrac was used to generate 411 leads, over 2023/24, the number of leads came to 561, while from July 2024 through to February this year, the now no longer used program had turned up 371 leads.

When further pressed by the NSW Greens MLC, Hudson doubled down on his assertion that no racial bias had been identified over the fifteen years of the system’s use, despite it suddenly being dropped of late.

Yet, Boyd retorted that the system was obviously misidentifying people, and even if they were subjected to further police scrutiny, once someone has been identified as a suspect, even the innocent are more likely to then be found guilty.

Saving time at the expense of the innocent

Boyd also put a series of questions to minister Catley during 12 March estimates, which when written answers were delivered on 8 April, further revealed that NSW state law enforcement has been experimenting with multiple software packages that are applying emerging technologies to real life incidents and crimes, which are undoubtably resulting in at least a modicum of questionable results.

Catley outlined that in terms of safeguards against these technologies the AI and Emerging Biometrics Ethics Board was established in August 2022, and its remit has been expanding ever since. The board is made up of the various commanders of policing units and it also includes two independent experts to provide extra input, as to the appropriate and ethical use of this tech.

The NSW Police Force’s AI and Emerging Biometrics Ethics Principles were established in 2023, and comprise of the ethical use of the technologies, while any AI projects proposed by NSW police must undergo an official assessment. The ethical framework also provides oversight to such programs, but it doesn’t require compliance. And NSW police is unaware of any breaches of this system.

In answering a question about PhotoTrac, Catley admitted it had still been utilising a 2011 developed algorithm until it was shut down the month prior. And in terms of standard operating procedures, there is one set in respect of the NSW police Facial Recognition Unit and its Real Time Intelligence Centre.

Artificial intelligence leading the police

In its 2018 Face Off report, British civil liberties group Big Brother Watch identified the prominent racial bias involved in such technologies, as police forces in that country were promptly incorporating facial recognition into their systems, and it identified at that point in time that the technology was misidentifying innocent people as suspects in crimes in a staggering 95 percent of cases.

During the 2020 rise in the Black Lives Matter movement in the United States, Microsoft announced that it would no longer be selling or developing facial recognition systems for law enforcement agencies, due to the well documented evidence that the technology is flawed, especially in terms of producing racially prejudiced outcomes.

Minister Catley further set out in questions on notice last month that despite Cognitec having updated its PhotoTrac algorithm on numerous occasions since its launch, the NSW police had not purchased any of the improved versions and simply carried on using the 2011 algorithm of a rapidly developing technology carrying serious real-life consequences for civilians it was identifying.

And while it’s reasonable to assume that the scrutiny critics like Boyd have been subjecting the NSW authorities to in terms of a convenient work solution for police, simultaneously producing rather inconvenient results for wrongly identified and charged civilians, there remains the question as to whether a more urgent development caused police to suddenly drop PhotoTrac like a led balloon.