Technology Rules: The Advent of New Technologies in the Justice System (Justice and Home Affairs Committee Report)

Proposing regulation often raises hackles, but it is another way of requiring standards to be met. Standards are a good thing—in themselves and because something known to meet agreed standards is more likely to be trusted. For example, standards can ensure, to the greatest possible extent, that conscious and unconscious bias—such as racial bias in stop and search tools—is not baked in. That is to the benefit of the producer as well as others. In other words, standards support innovation.

Procurements deserve a lot of attention. A police officer procuring a product can be vulnerable to an overenthusiastic sales pitch—we heard some horror stories—or a one-sided contract. I would have loved to see a form of contract, for instance, about the ownership of data, both input and output. Does the commercial producer of the programme own it? It is a big question, which makes one wonder about data inadequacy, but I will not go there this afternoon. We were not able to get hold of a form of contract: commercial confidentiality gets in the way.

National standards would include requirements in respect of reliability, accuracy and performance in the context of their use, evaluation, validity, suitability and relevance. It is very worrying if standards are regarded as a threat.

We heard a lot about the independence of police and crime commissioners, and that PCCs and chiefs ensure compliance with human rights. I heard that as overdefensive. Of course each force should pick products to suit its local needs; there are 43 forces applying the same law. By analogy, the BSI kitemark is in common use for many products in other sectors—in other words, certification. The police could have a choice among certified products. That would not preclude them picking products to suit their own local priorities. Operationally, this would not mean that the police do not have to assess both the necessity and proportionality of each deployment.

This is all part of governance. The point was made more than once, including by government: “You can always go to court to sort things out”, but the courts’ role is to apply the law, and nothing goes to court unless someone takes it there. That needs determination, emotional energy and money. By definition, the judgment will not be a comprehensive assessment nor a systematic evaluation.

In a similar vein, the Minister said to us that Parliament is the national ethics body—to be fair, I think that was a throwaway line—but I doubt that we are qualified for that. However, Parliament has a role in establishing a national body: independent, on a statutory basis and with a budget. We think there should be a single national body. Our report lists 30 relevant bodies and programmes. That makes for very complicated governance.

There can never be a completely one-stop shop, but that does not mean that simplification is not needed. It is not surprising that there is confusion as to where to find guidance. The committee recommends a body where all relevant legislation, regulation and guidance are collated, drawing together high-level principles and practice. Primary legislation should be for general principles, with detailed regulation setting minimum standards—not so prescriptive as to stifle innovation, but recognising the need for the safe and ethical use of technologies. We recommend the use of statutory instruments, despite the procedural drawbacks with which your Lordships are familiar, as a vehicle for regulations and a basis for guidance, with scope for non-statutory guidelines.

To assess necessity and proportionality, we need transparency. A duty of candour is associated more with the health service, but we urge the Government to consider what level of candour would be appropriate to require of police forces regarding their use of new technologies.

We also recommend mandatory participation in the Government’s algorithmic transparency standard—currently, it is voluntary—and that its scope be extended to all advanced algorithms used in the application of law which has implications for individuals. This would in effect produce a register, under the aegis of the central body. I understand that the Information Commissioner’s Office and Thames Valley Police, and no doubt more, are involved with the standard, and there is clear wish to link compliance with it to processes to improve technology and to enable police to exchange information about what works and what does not. There is a wish too to link it to independent oversight.

Ensuring the ethical use of any tool is fundamental. That has to be integral to the use of the tool, as we have seen with live facial recognition and the London gangs matrix, whose review apparently led to the removal of the names of some 1,000 young black men. The West Midlands Police are leaders with their ethics committee, both in having it and in how it is used—I have been very impressed by what I have heard and seen of its operation. There are similar bodies in a few, but only a few, other forces. If we get the standards right, the tools will be better trusted, by the citizen and the police themselves. That will free up police resources.

Current legislation provides that a person shall not be subject to

“a decision based solely on automated processing, including profiling, which … significantly affects him.”

The then Home Secretary assured us that decisions about humans would always be taken by humans—a human in the loop—but clicking a button on a screen is not enough when one starts from the mindset that “the computer is always right”. We agreed with the witness who said that the better way is that the machine is in the loop of human decision-making.

Does the human understand what it and he are doing? “Explainability” is essential; I had not come across that term before, but it seems to be used a lot in the sector. It is essential for the user, the citizen affected and everyone else. If the police officer does not understand the technology, how can he know if he—or it—has made a mistake? A critical approach in the best sense is needed.

The Sunday Times recently reported on new AI which will detect sex pests and thugs on trains who intend to assault rail passengers. It said:

“When a woman is sitting on her own in a carriage with empty seats, it could also assess whether she feels threatened when a man comes to sit down next to her or whether she welcomes his presence.”

There is no hint there might be some fallibility in all this. With all of this, noble Lords will not be surprised that we identified a lot of training needs.

We received the Home Office response to our report in the summer. I wrote on behalf of the committee to the then Home Secretary that we were “disheartened”—the best term I felt I could use courteously—by the

“reaction to what we hoped would be understood as constructive conclusions and recommendations. These are very much in line with the recommendations of other recently published work”.

Indeed, a workshop discussing the report last week at the Alan Turing Institute bore this out. The response read to us as more satisfied with the current position than was consonant with the evidence we had used. I will not quote from the Government’s response as I am optimistic that the Minister today will be able to indicate an understanding of our conclusions and an enthusiasm to progress our recommendations. I beg to move.

“the inexorable rise in blockchain technologies”,

which will

“immutably record every event or transaction in our lives.”

He also predicted that a

“truly integrated online digital justice system to resolve civil family and tribunals disputes”

would be in place in England and Wales by the mid-2020s at the latest. It is quite a thought.

It is very easy to be seduced by the technologies themselves, but I would like to pull focus to questions of transparency, governance and accountability. We are told that much accountability within the system now rests with police and crime commissioners. My own dealings with such a commissioner give me no reassurance at all—quite the opposite, in fact. I do not believe that PCCs can provide adequate or even meaningful accountability, especially where fast-moving technology is concerned. They lack the necessary expertise and, looking at some of the turnouts in PCC elections, they lack the authority too.

With both the criminal and civil justice systems so overstretched and behindhand, it is all the more tempting to succumb to the allure of the glittering baubles of high tech, AI, algorithms and all the rest, with the promise they appear to offer of a faster, slicker set of outcomes. If we are also persuaded that those outcomes are also more just and fairer, with human fallibility stripped out, the Lorelei cry may prove irresistible. Yet, again and again during the course of our inquiry, we heard from experts how algorithms, however sophisticated, can be “gamed”. If this is true, I wonder whether algorithms can ever truly be fit for purpose within a justice system.

It all takes us inevitably back to the old, uneasy, irreconcilable tension between the supposedly sacrosanct principle of operational independence versus the ultimate need for accountability to prevent a police force or chief going rogue, which, as I have witnessed myself, does indeed happen from time to time, although fortunately rarely. I am becoming increasingly troubled by what we call “fairness metrics”. We hear much talk of using AI, not simply to deliver the status quo more effectively and efficiently, but actively to make society “fairer”—a subjective and loaded term, if ever I heard one—by rectifying perceived social, economic and other inequalities. If that initiative acquires significant momentum, we as parliamentarians must surely be profoundly concerned about what is being factored in.

I see a clear analogy here with the development of automation and AI in the automotive sector. We were told six or seven years ago that driverless cars would be on our roads by 2021. The reality is, they are still not here. Safe implementation is a vital consideration, as is the need for an appropriate legislative and regulatory framework both pre and post placement and, ideally, through testing in a sandbox environment to ensure the veracity and reliability of algorithms.

Rushing the implementation of automation and AI would be damaging enough in the context of automated vehicles, but getting it wrong risks pushing back mass-market adoption of technologies designed to improve productivity and mobility. A similar mistake is surely inconceivable and wholly unacceptable in the context of the criminal and civil justice systems. Who is keeping a close eye on all this? Is it Ministers?

I am sad that the noble Lord, Lord Blunkett, is not here. To quote from the evidence that we received from the Minister, when I asked at question 107,

“Will you be keeping a careful eye on this?”

The Minister responded,

“That is a very good question which I will have to think about … We have some brakes and levers that we can pull”.

At that point, the noble Lord, Lord Blunkett, said,

“There are ways and means, I promise you.”

At the end of the day, that is what this debate is all about. Who is keeping a careful eye? Is it officials? If it is, from which of the plethora of departments and public bodies that are active in this field will they emerge?

We come back to accountability. Who has practical, day-to-day responsibility for the legal, ethical and active use of advanced technologies of this kind? Who has day-to-day decision-making powers, and where is the practical transparency and ultimate accountability? The reality is always that ultimate responsibility must rest with Ministers and Parliament. The Executive takes the decisions and faces the scrutiny of the legislature in either or both of our Houses of Parliament. The question then is how to make that work quickly, effectively and reliably.

It is perhaps inevitable that a report of this kind raises more profound questions than it would ever be capable of answering, especially when addressing so complex and controversial a topic. I was worried at the time of publication that we would not succeed in our aim of moving Ministers to share our concerns. The trials and tribulations within the Government in recent months have not served to calm my fears. Now we appear—I stress, appear—to be in a period of much-needed stability again. I hope we catch the eyes and ears of Ministers and make a difference, for in the field of radical innovation, just as in the field of criminal and civil justice, prevention of an undesirable outcome is invariably preferable to cure.

In endorsing this report, the committee unanimously decided that now is the time to start acting. It cannot be right that 43 constabularies are doing their own thing, most in isolation from each other, evaluating as they go along, at best—if they do it at all. However, that evaluation is not open to public scrutiny—it does not provide a route through to the point that the noble Lord, Lord Hunt, made: who is accountable? Parliament has to be accountable, and how do we discharge those responsibilities without the information in the first place? Each constabulary develops the use of the technologies to its local policing objectives and does different tasks to different levels of complexity. I am not making a case for a national police force, and neither was the committee, but it would be helpful to those constabularies to be provided with clarity from government on the basic principles that they should be observing, as this fits within the wider justice system.

Some are in no doubt—it may be the case; I do not have a crystal ball—that advanced technologies have a huge potential in assisting the police in delivering priorities and a policing system that commands confidence, trust and respect, improving efficiency, productivity and problem-solving. That is the sales pitch to the constabularies. But—and it is a very big but—these technologies have challenging and significant downsides with regard to civil liberties and human rights, as the noble Baroness, Lady Hamwee, pointed out. If not addressed, they will undermine the same confidence, trust and respect, and, if inadvertently and wrongly used, they will undermine the concept of fairness in our justice system. We should be under no illusion that if these technologies are allowed to mushroom in the police service without clear, consistent, understandable standards and protections, we will build up significant problems.

We urgently need consideration at national level of the trade-offs in using these new technologies: human rights versus interference with those rights, while ensuring that the interventions are necessary and proportionate. When we asked where the balance was, who was accountable and who was watching, answer came there none. On their effectiveness, we asked: are the public safer with these enhanced technologies and do these technologies make a difference? Again, in the absence of evaluation, answer came there none.

Transparency is a crucial principle because, increasingly, citizens want transparency about how their personal information is used and shared. Many benefits flow from transparency, including identifying problems early on and, crucially, improving the public’s trust in data-driven technologies. With that trust comes a pathway to developing appropriate technologies in supporting priorities. It happens elsewhere, so the committee suggested a central organisation or regulating body. NICE does it for the health service with regard to the efficacy of drugs. Why can it not be done in the justice system? The embryology authority balances what is possible medically with what is acceptable ethically. Why can we not use similar models?

Does the Minister think that police forces should satisfy themselves in advance of using new technologies, through independent verification, that the software program does not have an unacceptable level of bias? How can we be confident that historic cultural bias is not built into the system? Does the technology actually work and do what we want?

There are steps forward that could be taken—I know it is going to be very difficult—to deliver two central propositions recommended in this report. First, will the Minister agree to bring together the 43 constabularies, either by requiring or facilitating them, to share their knowledge and experience in this area so that we can begin the painful process of getting on the right side of this development? Secondly, will he consider appointing an expert panel of academics and practitioners to advise him on how to make progress on having the correct balance for a regulatory authority that protects us and our civil liberties, but enables the police and the justice system to do their job effectively?

Despite what paragraph 4 of the response says, neither House of Parliament has ever adequately considered or rigorously scrutinised automated facial recognition technology. We remain in the precarious position of police forces dictating the debate, taking it firmly out of the hands of elected parliamentarians and instead—as with the recent College of Policing guidance—marking their own homework. A range of studies have shown that facial recognition technology disproportionately misidentifies women and BAME people, meaning that people from those groups are more likely to be wrongly stopped and questioned by police, and to have their images retained as the result of a false match.

The response urges us to be more positive about the use of new technology, but the UK is now the most camera-surveilled country in the Western world. London remains the third most surveilled city in the world, with 73 surveillance cameras for every 1,000 people. The last Surveillance Camera Commissioner did a survey, shortly before stepping down, and found that there are over 6,000 systems and 80,000 cameras in operation in England and Wales across 183 local authorities. The ubiquity of surveillance cameras, which can be retrofitted with facial recognition software and fed into police databases, means that there is already an apparatus in place for large-scale intrusive surveillance, which could easily be augmented by the widespread adoption of facial recognition technology. Indeed, many surveillance cameras in the UK already have advanced capabilities such as biometric identification, behavioural analysis, anomaly detection, item/clothing recognition, vehicle recognition and profiling.

The breadth of public concern around this issue is growing clearer by the day. Many cities in the US have banned the use of facial recognition, while the European Parliament has called for a ban on the police use of facial recognition technology in public places and predictive policing. In 2020 Microsoft, IBM and Amazon announced that they would cease selling facial recognition technology to US law enforcement bodies.

Public trust is crucial. Sadly, the new Data Protection and Digital Information Bill does not help. As the Surveillance Camera Commissioner said last year, in a blog about the consultation leading up to it:

“This consultation ought to have provided a rare opportunity to pause and consider the real issues that we talk about when we talk about accountable police use of biometrics and surveillance, a chance to design a legal framework that is a planned response to identified requirements rather than a retrospective reaction to highlighted shortcomings, but it is an opportunity missed.”

Now we see that the role of Surveillance Camera Commissioner is to be abolished in the new data protection Bill—talk about shooting the messenger. The much-respected Ada Lovelace Institute has called, in its report Countermeasures and the associated Ryder review in June this year, for new primary legislation to govern the use of biometric technologies by both public and private actors, for a new oversight body and for a moratorium until comprehensive legislation is passed.

The Justice and Home Affairs Committee stopped short of recommending a moratorium on the use of LFR, but I agree with the institute that a moratorium is a vital first step. We need to put a stop to this unregulated invasion of our privacy and have a careful review, so that its use can be paused while a proper regulatory framework is put in place. Rather than update and use toothless codes of practice, as we are urged to do by the Government, to legitimise the use of new technologies such as live facial recognition, the UK should have a root-and-branch surveillance camera and biometrics review, which seeks to increase accountability and protect fundamental rights. The committee’s report is extremely authoritative in this respect. I hope today that the Government will listen but, so far, I am not filled with optimism about their approach to AI governance.

Even so, I struggle to understand how the system works. Where do the instructions come from, and are they the right people? How do we know that the answers it produces are the right ones? Is the system open to cross-examination to test these issues? If so, how can this be done? I share the committee’s concern about where all this is leading. So far as the courts are concerned, AI comes especially into play in two ways. The first is in the provision of evidence in a criminal trial. The other is in its use in dispute resolution in the civil courts. Each of them presents very real challenges.

The report, for the most part, is directed at the use of advanced technologies by police forces. The courts become involved when evidence that has been gathered by this means is led at a criminal trial to secure a conviction. Some years ago—in fact, quite a number of years ago—I presided in a case before the criminal appeal court in which the appellant had been convicted on the basis of a primitive system of facial recognition technology. He insisted that it was a mistake and that its use was unfair because, due to problems with legal aid, he had no access to expert evidence to challenge it. It seemed to us that that amounted to a miscarriage of justice, so we set aside the conviction so that he could face trial again with expert assistance.

In the retrial, the jury—unfortunately, from his point of view—reached the same conclusion as the first jury on the recognition evidence and once again he was convicted. My point is that fairness and transparency, which the noble Baroness, Lady Primarolo, emphasised in her impressive speech, should be at the heart of any criminal trial. That requires that evidence of this kind should be open to challenge. As it happens, there was no suggestion that the evidence in that case had been manipulated; it was just said to be a mistake. The reference to the possibility of manipulation must give rise to real concerns, as shown by the very important selection of paragraphs 23 to 26 in the report, under the heading,

“The right to a fair trial”.

I support the recommendations that are referred to as numbers 1, 2 and 4 in the Government’s response. They are all designed to ensure the safe and ethical use of AI. The Government say they are confident that existing structures and organisations create a sufficient network of checks and balances, but the evidence that is narrated in this report suggests that that confidence may be misplaced. More safeguards than those that are available may be needed in this fast-moving area. I endorse the point made by the noble Lord, Lord Blunkett, which the noble Baroness mentioned: it is far better to do this now than later, when it would be too late and things would have moved on beyond recall.

As for AI’s use in dispute resolution in the civil courts, I pay tribute to the work of the Library and its very helpful briefing on the report. It contains a link to an article referred to by the noble Lord, Lord Hunt of Wirral, headed,

“Technology to become embedded in UK justice system by 2040, senior judge suggests”.

That contained a link to a speech that was given online in March this year by the Master of the Rolls, Sir Geoffrey Vos, about the future for dispute resolution in what he referred to as a “brave new world”.

If one wants to be enlightened of the huge advantages that AI can offer, they can be seen in Sir Geoffrey’s speech. He is an enthusiastic supporter, promoting AI’s use in the civil courts as fast as possible. He focuses particularly on the advantage of speed and simplicity, which gathering evidence in this way can produce. I am certainly not one of those who decries the use of AI; it is all a question of how it can be best operated.

According to Sir Geoffrey, factual disputes will themselves become a thing of the past, as so much of what we do will be indelibly recorded by AI. He referred, among other things, to number plate recognition. You cannot really dispute where your car has appeared, because AI no longer leaves any room for dispute about that. He says that we are more and more likely to find this a feature of dispute resolution in the civil courts.

He went on to say that some decisions, admittedly minor decisions, such as those about time limits and other procedural aspects, could be made by this system with no human intervention. Proposals for dispute resolution themselves would be “driven by AI”, as he puts it.

He acknowledged that public confidence is important, and that the public would need to understand what had been decided by a machine and what had not. He also said that, ultimately, there must be the ability to question an AI-driven decision before a human judge. That begs the question whether and how that can be done, and how far we can trust algorithms that are not open to being tested in that way.

I was encouraged by the statement in paragraph 32 of the Government’s response that they will work with the justice system with a view to

“better long term research and evaluation of the different circumstances in which predictive algorithms”

are described and used to support future decision-making. Of course, there is much that the courts themselves can do to control and regulate their use, but the extent of the ability of litigants to question and interrogate the algorithms is not open to control or guidance by the courts. That is why the recommendation in paragraph 155 of the report, which is dealt with in paragraph 18 of the Government’s response, is so important. It is about the need for a requirement on producers to embed explainability within the tools. If that requirement is there, one may be able open up a system of cross-examination to find out what is going on and see whether what has been produced can be relied on. I fear that the Government’s response in paragraph 35 hardly does justice to this crucial issue.

I hope that when he comes to reply the Minister will be able to reassure the noble Baroness that the Government will look again at the evidence and recommendations in the committee’s report, to see whether more can be done to regulate and control the way that AI is imposing itself on our lives. I suggest that if the Minister and his team have not already done so, they might like to read Sir Geoffrey’s speech, because it will show the advantages and concerns which surround this whole issue.

In one sense, the Government are right to make much of CAID because it was game-changing. For instance, a case with 10,000 images that would typically have taken up to three days to review could be done in an hour, thanks to CAID. Perpetrators could be apprehended more quickly, officers protected from the effects of viewing these images and more focus placed on identifying the victims. As someone who worked on child sexual abuse and exploitation at the Home Office when CAID was introduced, I assure the Minister that I completely understood—and understand—the value of new technologies in certain instances.

However, in the context of the report, I just do not think that it is a very helpful example. The Home Office itself helped to develop CAID in collaboration with the police and industry partners. Once piloted, it went live across all police forces and the NCA. To suggest that that is the norm would be misleading, and it should not be used as a reason not to address the clear problems that we identified in a system where all 43 forces, as has been mentioned, are free to individually commission whatever tools they like in a market that is, as we said, opaque at best and the Wild West at worst, in which the oversight mechanisms are, frankly, inadequate. The Home Office may think that we are overreacting, but the truth is that it would be hard-pushed to make that case because without a central register, as we suggested, it is impossible to know who is using what, how and to whom.

If we dig a little deeper, the Minister may see why we are concerned. Some of this has already been mentioned. On procurement, we heard from a police representative who said that procurement is not the comfort zone of all police forces. When the tools they are procuring may have consequences for human rights and the fairness of the justice system, as we have been talking about, never mind taking into account the complexities of the technologies market, where providers are reluctant to share information on the basis of commercial confidentiality, as the noble Baroness, Lady Hamwee, said, that is truly worrying.

Then there is the problem that, as the NCC Group told us,

“many claims made by [Machine Learning] product vendors, predominantly about products’ effectiveness in detecting threats, are often unproven, or not verified by independent third parties.”

There are the salespeople who—in an understandably overzealous way in a burgeoning market—according to one developer,

“take something they do not understand and shout a number that they do not understand”.

I would add that in many cases they then make it available to officers who do not understand it either. Incredibly, the police are not required to be trained to use different AI technologies—this is one of the things I found most shocking in our report—including facial recognition, because they are procured at a local level.

All this does not feel like a solid foundation on which to deploy such highly sensitive tools and, as the noble Baroness, Lady Hamwee, has already alluded to, there are some in the police and in the market who agree with us. At the excellent conference at the Alan Turing Institute last week, one speaker representing the police pointed out that in order to become a detective you have to pass an exam, and that the same should be true for technology. Another from a different force said: “Artificial intelligence is not on the tip of the tongue of the public yet, but we don’t want it to be another frontier of failure.”

One way in which we could help to build confidence is statutory specialist ethics committees, which would not only increase community involvement and understanding but help to create an institutional culture of accountability, something that we already know needs to be improved. I am afraid to say that that was another recommendation dismissed by the Home Office.

I am not blaming the police here. There are some brilliant forces, such as West Midlands, which have spotted the benefits but also the pitfalls, and which are working hard to get ahead of them. Without more commitment from the Government, though, I fail to see how the current system leads to anything but another frontier of failure. As people have said throughout the debate, at some point under the current free-for-all, when a police force that has not put in the protections that, say, West Midlands has, it feels inevitable that something is going to go very wrong.

It is not as though the Government are not doing anything. The Centre for Data Ethics and Innovation, which is based in DCMS, is piloting the public sector algorithmic transparency standard. We on the committee would all agree with that, and, genuinely, people around the world are looking at it. Can the Minister tell me, if you compare the work that is going on in DCMS with the response to our report, how closely do officials in the Home Office work with their counterparts in DCMS on this? This pilot includes some police forces, and it does not feel as if the two marry up.

Again, as others have said, I know that probably quite a few people may wish to put this report on the shelf and watch it gather dust. However, I think we all know that in practice, that is unlikely to happen because the concerns raised within it will surely become more apparent down the line.

Finally, we heard a great analogy at the conference last week with regard to training for those using AI. The speaker said: “For a car to be allowed on the road, it’s got to have an MOT, but the driver also has to have a licence.” I am afraid that at the moment, with regard to these technologies, we do not have either.

That is a comment on the rise of these very exciting new technologies in general but I suggest that, of all the spheres in which artificial intelligence and these new technologies are being employed, the criminal justice sphere is special. There are great potential benefits, as we have heard, but real dangers as well. Why are the criminal justice system and the ambit of the home department so special? It is because we are talking about people’s rights and freedoms. We are thinking about the right to life and to protect people, our communities and victims and potential victims, but we are also talking about the gravest rights, freedoms and liberties of the subject. That came through very clearly in both the evidence to and the private deliberations of our committee.

I remind noble Lords that it was just over 40 years ago that, in response to the Brixton riots in this city, Lord Scarman produced his report because there was a crisis of trust and confidence in policing in so many of our communities. Not long after that legendary Scarman report, a Conservative Thatcher Government produced the Police and Criminal Evidence Act 1984. There was inevitably some controversy attached to it but, none the less, I would consider it a piece of human rights legislation, because it attempted to set a framework of principles and law for governing police power.

We would not dream today of rescinding or repealing that Act. It has been amended, but it is still on the statute book. The idea is that police power, while essential, needs to be regulated and consolidated in one place. Of course, new and intrusive technologies have emerged. The PACE codes have had to be updated and the legislation itself has been amended, but some basic principles and ideas of accessibility and transparency in the use of intrusive police power hold still, over 40 years later.

I do not believe that noble Lords and Ministers would dream of rescinding that, and nor should the Government think that such a framework is not needed today in relation to these new powers—these powers which we cannot even see being used, or understand, because they are effectively in a black box, or in a jar in the form of the pill but I cannot say what is in the pill that I am taking. That is why regulation and framework legislation is required.

It is simply not enough to rely on the current arrangement of broad police discretion and the occasional police witness to our committee or some other forum to say, “Oh, but you know: proportionality”. We are compliant with human rights proportionality as it if is a mantra. That is not detailed enough for regulation. It would not be detailed enough for powers of arrest and it certainly would not be detailed enough for the use of drugs. We need to get into the black box: we need to prescribe it and to decide what is legitimate and proportionate in the use of this technology and its design. Legislation is absolutely essential to avoid what the noble Baroness, Lady Sanderson of Welton, called the Wild West—because that is exactly where we are now in the use of this technology in the criminal justice system and, to some extent, at the border in relation to its intrusive use.

In addition to this framework legislation—the Police and Criminal Evidence Act and an AI Act for the 21st century—we need a national body that will do the prescription and kitemarking. There is no doubt that we need this because of the black box. Lay citizens and even parliamentarians cannot understand the technologies, read and decipher the algorithms, and understand whether coded bias is being baked in—which is happening.

I commend the Netflix documentary on facial technology that features the noble Baroness, Lady Jones of Moulsecoomb, from this House. It is a wonderful documentary. I hope that noble Lords, Ministers and their officials—who are passing them notes, probably saying “Yes, it’s a great documentary”—will watch it.

Kitemarking is essential before any procurement of these technologies and algorithms within the criminal justice system. It should not be left to local police officers, or even PCCs, to have lunch with some people who are selling their wares and decide what is a good deal or not.

In addition to the kitemarking of the product, there is a great opportunity for His Majesty’s Government and the United Kingdom in going down the road being advocated in our report. We could be world leaders in the kitemarking and regulation of this technology. In years to come, if we take up the recommendations from this committee, there could be countries all over the world that say, “We go for the UK AI in criminal justice model”. It is the equivalent of saying they want to contract in English law or in Delaware law, or whatever it is. This technology is being developed and used all over the world, and if we get ahead of the kitemarking and regulation game, others may contract into our arrangements and adopt our technologies and systems over time.

It is completely without justification, it seems to me, for private companies to be experimenting on our populations, including with their intimate data and with policing and intelligence and so on, and then claiming that they will not engage with transparency or legality because of commercial sensitivities. That is a swindle and a scandal, and it needs to end. We would not allow arms companies or drugs companies to behave this way; we certainly should not be allowing it in these deals that are being done in the 43 forces with these people in the Wild West—I will not say who it is that rides around on horses in the Wild West, but the point is made.

To conclude, we are just asking for this technology to be governed by the rule of law, for Parliament to step up and, crucially, for Ministers to step up, as their predecessors did in the Thatcher Government in the 1980s in response to the Brixton riots and the Scarman report. Only this time, we are asking that this be done before a scandal and before a crisis of confidence that reaches the kind of levels where it will be harder to use the technology in a positive way in the future.

No wonder that the committee reports that it

“uncovered a landscape, a new Wild West, in which new technologies are developing at a pace that public awareness, government and legislation have not kept up with.”

It refers to the phenomenon of “digital excitement”—one could say the delight in boys’ toys, if that were not sexist—felt by some who get their hands on a new technology product. That is of course not a good rationale for purchase. It is hardly surprising that my noble friend Lady Hamwee, in her letter to the Home Secretary, said that the committee was “disheartened” by the Home Office’s response to its “constructive conclusions and recommendations”, saying it found the Home Secretary—I think my noble friend Lady Hamwee quoted this—

“more satisfied with the current position than is consonant with the evidence”

that the committee had received. That is quite a strong message.

My noble friend Lady Hamwee said with considerable feeling that the committee

“hoped that when the House debates the report, the Minister will be able to explore with us in more depth the points that we raised, and not simply be briefed to repeat the formal response”.

We very much look forward to that more realistic response today. The Government’s response was disappointing and complacent, and failed to do justice to the quality of the evidence, the report and the committee. The Government

“was not persuaded that a new independent national body and certification system should be created. It said whilst certification worked in some contexts, it could also create false confidence and be costly. It disagreed with the idea of making transparency a statutory principle. It said … making transparency a legal duty could limit the police’s current transparency efforts to whatever would be set out in statute.”

Also, the Government said that

“it could not make the police and the judiciary undertake training on ‘meaningful interaction with technologies’. This was because training was the responsibility of the College of Policing and Judicial College, rather than the government.”

However, as the noble Baroness, Lady Sanderson of Welton, said, we oblige drivers to have a licence as well as for the car to have an MOT. The Government

“disagreed that there should be statutory ethics groups created to scrutinise the use of technologies and veto deployment … because they would not be democratically elected.”

These all seem remarkably weak points. An alternative term would be “scraping the barrel”.

The committee said that:

“While we found much enthusiasm about the potential of advanced technologies… we did not detect a corresponding commitment to any thorough evaluation of their efficacy … there are no minimum scientific or ethical standards that an AI tool must meet before it can be used in the criminal justice sphere. Most public bodies lack the expertise and resources to carry out evaluations … we risk deploying technologies which could be unreliable, disproportionate, or simply unsuitable for the task in hand.”

Are the Government happy with that situation?

The committee found the institutional landscape confused and duplicative—no wonder, with at least 30 organisations, initiatives and programmes having some input or other—and found governance arrangements complex and disconnected, while the Government are appointing still more bodies which make the picture even more crowded. The committee said:

“We have heard no evidence that the Government has taken a cross-departmental strategic approach to the use of new technologies in the application of the law … Thorough review across Departments is urgently required.”

Can the Minister tell us that that at least will happen? The report mentions that a government White Paper is supposedly in the pipeline. Can the Minister tell us the envisaged date for that?

The report has a number of important proposals on governance, oversight and evaluation to address these various deficits. One very sensible proposal is a new national body to set scientific and quality standards and certify new products against those standards. The committee recommends “evaluate centrally, procure locally”.

The committee says its

“evidence reflected organisational confusion about what guidance, regulation and legislation applied”

and argues persuasively for a strong legal framework to remedy the fact that

“users are in effect making it up as they go along.”

No wonder it uses the term “Wild West”.

The report refers to the EU artificial intelligence regulation, or “AI Act”, that is in preparation—I am not sure where it has got to—and notes that it would ban systems that pose an “unacceptable risk’”, such as social scoring and many deployments of facial recognition. I hope the Government are still willing to learn from the EU.

The committee suggests legislation to set principles, supplemented by regulations to govern the use of specific technologies. If the Government object that there is a lack of parliamentary time, I suggest at least three Bills that could and should be dropped to make space: the Northern Ireland Protocol Bill, the revocation of EU law Bill and the Bill of Rights Bill, otherwise known as the Human Rights Act destruction Bill.

The committee found the market “worryingly opaque”, with buyers often pretty ignorant about the systems that they were buying due to companies’ insistence on commercial confidentiality. It found some “dubious selling practices” and untested, unproven claims about effectiveness of the products.

The committee therefore makes a number of important proposals for increased transparency and explainability, including consultations and published impact assessments. The committee reports that there is no central register, making it virtually impossible to find out where and how these systems are being used such that parliamentarians, the media, academics and those subject to them could scrutinise and challenge them. The committee rightly called for a mandatory register.

The Government published their consultation paper Data: A New Direction just over a year ago, promising

“a bold new data regime”,

a phrase that makes me wary. I am concerned about prejudice to our data adequacy decision from the European Commission but also worried if it makes the Government less vigilant about data protection and privacy issues.

The committee said it sees

“serious risks that an individual’s right to a fair trial could be undermined by algorithmically manipulated evidence”,

with defendants and indeed courts ignorant of what technologies might have been used in their case. That is a pretty dire state of affairs.

The report raises serious concerns that bias in data collection could lead to discriminatory policing, especially in predictive policing. It is well-known, as my noble friend Lord Clement-Jones pointed out, that facial recognition technology is not sound when used on female and ethnic minority subjects because the learning algorithms have leaned more on data from white men than from other groups. The committee also warned of the danger of overpolicing through the use of predictive tools, which could become a vicious circle of concentration on poorer people in more disadvantaged areas.

The committee is highly concerned at the lack of accountability for the misuse or failure of these AI technologies and hence the lack of recourse for people who might suffer from their use. It suggests that the Government appoint a taskforce to produce guidance on consistent lines of accountability.

This is a first-class and hugely valuable report. The Government’s complacency—I could say blinkered complacency—is profoundly unwise when defects and unfairness in the deployment of AI systems could create a backlash through a loss of trust or become, in the words quoted by the noble Baroness, Lady Sanderson of Welton, “another frontier of failure.”

The glittering prize for the UK is, in the words of the report, to be

“a frontrunner in the global race for AI while respecting human rights and the rule of law.”

I hope we hear a better response than we had in June and concrete plans now from the Minister.