That this House takes note of the Report from the Artificial Intelligence in Weapon Systems Committee Proceed with Caution: Artificial Intelligence in Weapon Systems (HL Paper 16).
My Lords, it is a pleasure to introduce this debate on the report of the AI in Weapon Systems Committee. I am very grateful to the business managers for arranging an early debate; this is a fast-moving subject and it is right that the House has an early opportunity to consider it.
It was a real privilege to chair the committee, for two reasons. The first was its most agreeable and expert membership, who were thoroughly collegiate and enthusiastic. The second was the outstanding staff support that we received. The excellent Alasdair Love led a first-class team by example. As well as Alasdair, I thank Sarah Jennings, our gifted policy adviser; Cathy Adams, who led us authoritatively through the complexities of international humanitarian law; Stephen Reed, who provided Rolls-Royce administration; and Louise Shewey, who was the ideal communications expert. Our two specialist advisers, Professor Dame Muffy Calder from the University of Glasgow and Adrian Weller from the Alan Turing Institute at the University of Cambridge, were invaluable.
AI will have a major influence on the future of warfare. Forces around the world are investing heavily in AI capabilities but fighting is still largely a human activity. AI-enabled autonomous weapon systems—AWS—could revolutionise defence technology and are one of the most controversial uses of AI today. How, for example, can autonomous weapons comply with the rules of armed conflict, which exist for humanitarian purposes?
There is widespread interest in the use of AI in autonomous weapons but there is concern as well. Noble Lords will be aware of recent reports that Israel is using AI to identify targets in the Gaza conflict, potentially leading to a high civilian casualty rate. In a society such as ours, there must be democratic endorsement of any autonomous weapon capability. There needs to be greater public understanding; an enhanced role for Parliament in decision-making; and the building and retention of public confidence in the development and potential use of autonomous weapons.
The Government aim to be “ambitious, safe, responsible”. Of course we agree in principle, but aspiration has not entirely lived up to reality. In our report, we therefore made proposals to ensure that the Government approach the development and use of AI in AWS in a way that is ethical and legal, providing key strategic and battlefield benefits, while achieving public understanding and democratic endorsement. “Ambitious, safe and responsible” must be translated into practical implementation. We suggest four priorities.
First, the Government should lead by example in international engagement on regulation of AWS. The AI Safety Summit was a welcome initiative, but it did not cover defence. The international community has been debating the regulation of AWS for several years. Outcomes could be a legally binding treaty or non-binding measures clarifying the application of international humanitarian law—and each approach has its advocates. Despite differences about form, an effective international instrument must be a high priority.
My Lords, it gives me great pleasure to follow the noble Lord, Lord Lisvane, the chairman of the committee, who was absolutely excellent in the way he carried out the job. I have no doubt that he had somewhat of an advantage over many of the rest of us on the committee, as he had spent quite a lot of time in the House of Commons on the Defence Select Committee, which must have given him great inside knowledge of what was going on in the defence field. That was very useful to all of us.
I am very glad to have been on the committee. I have always believed that, if we are to win wars, we need two major components. First, we have to train and motivate our troops properly. I do not think anybody doubts that the British are world leaders in training the military; indeed, we do it for many other countries as well. The professionalism that our Armed Forces show is the envy of the world. I wish I could say the same of the ability of the Ministry of Defence to procure equipment, which has been lacking for years, even in the days when I was responsible for some of it.
The interesting thing that has changed is that, in the old days, industry used to look to defence to spend taxpayers’ money on research and development, hoping that some of that technology would move over into the private sector and it would benefit. That has all completely changed now. The sums of money that have been spent on research and development by the private tech companies in the United States, for instance, are so enormous that technological change is moving at a very fast rate. Let us face it: defence is benefiting from the private sector rather than the other way around. As a result, technology is moving on so fast that it is very difficult for any of us to keep up with it.
So I am very keen that we should embrace AI. We will be left at a serious disadvantage if our enemies adopt AI with enthusiasm and we do not. It is extremely important that we take this on board and use it to save the lives of our troops and improve our chances of winning wars.
My Lords, I am pleased to follow the noble Lord, Lord Hamilton. Like him, I congratulate the committee’s chair, the noble Lord, Lord Lisvane, and thank him for securing an early debate on the report, for his comprehensive and powerful opening speech and for his adroit and inclusive chairing of the committee. It is most important in these circumstances to have a chair who is inclusive. It was a pleasure to serve under his chairmanship and it is an equal pleasure to recognise and thank the clerks, the staff and advisers for their exemplary support.
The report and the record of its proceedings contain a great deal that is of value, including testimony from a range of experts which would repay concerted attention from Ministers and officials. Mindful of the time constraint, I wish to focus on one specific area: our working definition of AWS, or rather, the absence of one, which presents very real difficulties, both domestic and international.
The gateway through which I entered this somewhat Kafkaesque debate was a sentence from the Ministry of Defence which purported to explain why this country does not have an agreed definition of AWS. Cited on page 4 of the report and intermittently thereafter, it suggests that we do not have a working definition of AWS because, as the noble Lord, Lord Lisvane, has quoted more broadly than I will, such terms
“have acquired a meaning beyond their literal interpretation”.
The floor of your Lordships’ House is not an appropriate forum for a detailed textual exegesis, although I do enjoy that. However, that sentence recalled Hazlitt’s criticism of the oratory of William Pitt, which he stigmatised as combining
“evasive dexterity, and perplexing formality”.
This impression reflects the conclusion of the committee that this explanation is plainly insufficient. How can we actively seek to engage with policy in regulating AWS if we cannot find even provisional words with which to define it? It is like attempting to make a suit for a man whose measurements are shrouded in secrecy and whose very existence is merely a rumour. These are, of course, enormously complex questions but in making good policy, complexity should not be a refuge but a rebuke. It is the job of Governments of any political stripe to be able to articulate their approach and have it tested by experts and dissenting voices.
My Lords, it is a delight to follow the noble Lord, Lord Browne, whose companionship in the committee was but one of its many delights.
I start by drawing attention to my relevant interests in the register, particularly my advisory role with three companies, Thales, Tadaweb and Whitespace, all of which have some interest in the exploitation of AI for defence purposes.
It is great to see a few dedicated attendees of the Chamber still here late into Friday. My motivation to speak is probably as much to do with group loyalty as the likelihood of further value added, so I will keep my comments short and more focused on some contextual observations on the committee’s work, rather than in the pursuit of additional insights. There is not much more I want to stress and/or prioritise regarding the actual conclusions and recommendations of the report, and our chairman’s opening remarks were typically excellent and comprehensive. However, there are some issues of context that it is worth giving some prominence to. I will offer half a dozen, all of which represent not the committee’s view but a personal one.
The first comment is that the committee probably thought itself confronted by a prevailing sense of negativity about the risks of AI in general and autonomous weapons systems in particular. The negativity was not among the committee’s membership but rather among many of our expert witnesses, some of whom were technical doom-mongers, while others seemed to earn their living by turning what is ultimately a practical problem of battlefield management into an ethical challenge of Gordian complexity.
My second comment is specifically on the nature of the technical evidence that we heard, which, if not outright conflicted, was at least extremely diverse in its views on risk and timescale, particularly on the risks of killer robots achieving what you might call self-generated autonomy. The result was that, despite much evidence to the contrary, it was very difficult to wholly liberate ourselves from a sense of residual ignorance of some killer fact. I judge, therefore, that this is a topic that will as we go forward require persistent and dynamic stewardship.
My Lords, I am grateful to the noble Lord, Lord Lisvane, for introducing this debate.
I read the report as soon it was published. I agree with it and with the position of HMG and the MoD. However, looking around the corner, I see that reality may conflict with what the report says. Its title is of course very appropriate—although we might wonder how we got it. I am relaxed about the MoD’s reluctance to define AWS. A definition has the danger of excluding certain unanticipated developments.
It may be helpful to the House if I illustrate a potential difficulty with a fully autonomous system, to show why we should not willingly go in this direction. Suppose His Majesty’s Armed Forces are engaged in a high-intensity conflict and an officer is in control of a drone system. He reads his intelligence summary—INTSUM—which indicates fragility in the cohesiveness of enemy forces. The officer controls the final decision for the drone to engage any target, in accordance with our current policy. The drone detects an enemy armoured battalion group but the AFVs are tightly parked in a square in the open, not camouflaged, and the personnel are a few hundred metres away, sitting around campfires. In view of the INTSUM, it would be obvious to a competent officer that this unit has capitulated and should not be engaged for a variety of reasons, not least international humanitarian law. It is equally obvious that a drone with AI might not recognise that the enemy unit is not actively engaged in hostilities. In its own way, the report recognises these potential difficulties.
My concern centres on the current war in Ukraine. Both sides will be using electronic warfare to prevent their opponent being able to receive data from their own drones or give those drones direction. That is an obvious thing to do. But if you are in a war of survival—and the Ukrainians certainly are—and you have access to a drone system with AI that could autonomously identify, select and attack a target, absent any relevant treaty you would have to use that fully autonomous capability. If you do not, you will lose the war or suffer heavy casualties because your enemy has made your own drones ineffective by means of electronic warfare. So long as drones are being used in the current high-intensity conflict, we need to recognise that it will be almost impossible to prevent AI being used fully autonomously. Equally, it will be hard to negotiate a suitable treaty, even if we attach a very high priority to doing so.
My Lords, I too welcome the excellent report from the committee and thank it for this work. My brief contribution will focus on AI in the maritime domain. My starting point is that if, like me, you believe we need a bigger Navy then it is obvious that we will need to use AI-enabled systems as an effective force multiplier.
We should therefore enthusiastically welcome the Royal Navy’s leadership in a wide range of maritime use cases. For example, in the surface fleet there is the so-called intelligent ship human autonomy teaming; in the subsurface environment, autonomous uncrewed mine hunting, partly supported by the new RFA “Stirling Castle”, as well as new sensor technologies and acoustic signature machine learning for anti-submarine warfare; and in maritime air defence, AI-enhanced threat prioritisation and kinetic response using tools such as Startle and Sycoiea, which are obviously vital in an era of drone swarms and ballistic and hypersonic missiles. These and other AI systems are undoubtedly strengthening our nation’s ability to deter and defend at sea. They also enhance the Royal Navy’s centuries-old global contribution to rules-based freedom of navigation, which underpins our shared prosperity.
Looking forwards, my second point is that Parliament itself can help. When it comes to experimentation and trialling, there is a sense in some parts of defence that peacetime risk-minimisation mindsets are not currently well calibrated to the evolving and growing threats that we now face. Parliament could therefore accept and encourage a greater risk appetite, within carefully set parameters. Many innovations will come from within the public sector and we should support investment, including in the excellent Dstl and DASA. But in parentheses, I am not convinced by the report’s recommendation at paragraph 17 that the MoD should be asked to publish its annual spending on AI, given that it will increasingly become ubiquitous, embedded and financially impossible to demarcate.
My Lords, I am grateful to the noble Lord, Lord Lisvane, for his opening summary of this important report and to the noble Lord, Lord Stevens, for his remarks just delivered, reminding us of the maritime context of this debate as well. I also thank those involved in the creation of the report. Perhaps this alone is worth noting: AI did not produce this report; human beings did.
My friend the right reverend Prelate the former Bishop of Coventry was a member of the committee producing the report and he will be delighted that it is receiving the attention it deserves. He is present today, and I hope he does not mind me speaking on his behalf in this regard.
The principles of just war are strongly associated with the Christian moral tradition, in which it is for politicians to ensure that any declaration of war is just and then for the military to pursue that war’s aims by just means. In both cases, justice must be measured against the broader moral principles of proportionality and discrimination. This, then, is where AI begins to raise important and urgent questions. AI opens new avenues of military practice that cannot be refused, together with new risks that must not be ignored. The report rightly says that we must “proceed with caution”, but it does say “proceed”. Here, there is an opportunity for the UK to fulfil its commitment to offer leadership in this sphere in the international field.
There is a risk of shifting the decision-making process and the moral burden for each outcome on to a system that has no moral capacity and too much opacity. To implement AI’s benefits well, military cultural values need to be named, explained and then translated into AWS’ command and control—especially where the meaning of “just” diverges from the kind of utilitarian calculus that most easily “aligns” digital processes with moral choice.
Inherent human values, including virtue, should also be embedded in the development, not just the deployment, of new AI-enabled weapon systems. As recent use of AI systems shows in the context of global conflict, AI changes questions of proportionality and discrimination. When a database identifies 37,000 potential targets using “apparent links” to an enemy, or an intelligence officer says
My Lords, this has been a very useful report, because it gets us thinking properly about these things.
I declare an interest in the whole world of generative AI and LLMs, with Kaimai and FIDO, which is looking at curated databases to extract information. It is very useful for that sort of thing. With that, as mentioned in the report, comes the whole issue of training materials. The results you get depend on what the AI is looking at. If you fire it off against a huge amount of uncurated stuff, you might, and you can, get all sorts of rubbish back. Certain tests have found AI returning things that were 70% to 80% inaccurate. On the other hand, if put against something carefully targeted, such as medical research, where everything that has gone into the dataset has been studied and put together by someone, it will find stuff that no one has had time to read and put together, and realise that it is relevant.
In the same way, AI will be very useful in confusing scenarios facing a military commander, or in military decisions, to help them weed out what is right and what is wrong and what is true and what is not. I seem to remember, though I cannot remember when it was, that there was nearly a nuclear war because, at one point, various sensors had gone wrong and they thought there was a military attack on the United States. They nearly triggered all the defences, but someone suddenly said, “Hang on, this doesn’t look quite right”. It may well be that an artificial intelligence system, which may not be confused by some of the fluff, might have spotted that more easily or accurately, and reported it and said, “Don’t believe everything you’re looking at; there is another problem in the system”. On the other hand, it might have done the opposite. This is the trouble, which is why the human intervention point is very important.
We also have to remember that, although AI started being developed in the 1980s, with neural networks and things like that, it is only really getting into its stride now. We do not know quite where things will end up, and so it is very difficult to regulate it.
2:05 pm
20 of 39 shown
A key element in pursuing international agreement will be prohibiting the use of AI in nuclear command, control and communications. On one hand, advances in AI offer greater effectiveness. For example, machine learning could improve detection capabilities of early warning systems, make it easier for human analysts to cross-analyse intelligence, surveillance and reconnaissance data, and improve the protection of nuclear command, control and communications against cyberattacks.
However, the use of AI in nuclear command, control and communications could spur arms races or increase the likelihood of states escalating to nuclear use during a crisis. AI will compress the time for decision-making. Moreover, an AI tool could be hacked, its training data poisoned or its outputs interpreted as fact when they are statistical correlations—all leading to potentially catastrophic outcomes.
Secondly, the Government should adopt an operational definition of AWS which, surprisingly, they do not have. The Ministry of Defence is cautious about adopting one because
“such terms have acquired a meaning beyond their literal interpretation”,
and an
“overly narrow definition could become quickly outdated in such a complex and fast-moving area and could inadvertently hinder progress in international discussions”.
I hear what the Government say, but I am not convinced. I believe it is possible to create a future-proofed definition. Doing so would aid the UK’s ability to make meaningful policy on autonomous weapons and engage fully in discussions in international fora. It would make us a more effective and influential player.
Thirdly, the Government should ensure human control at all stages of an AWS’s lifecycle. Much of the concern about AWS is focused on systems in which the autonomy is enabled by AI technologies, with an AI system undertaking analysis on information obtained from sensors. However, it is essential to have human control over the deployment of the system, to ensure both human moral agency and legal compliance. This must be buttressed by our absolute national commitment to the requirements of international humanitarian law.
Finally, the Government must ensure that their procurement processes can cope with the world of AI. We heard that the Ministry of Defence’s procurement suffers from a lack of accountability and is overly bureaucratic—not the first time such criticisms have been levelled. In particular, we heard that it lacks capability on software and data, both of which are central to the development of AI. This may require revolutionary change. If so, so be it—but time is short.
Your Lordships have the Government’s reply to our report. I am grateful for the work that has gone into it. There are six welcome points, which I will deal with expeditiously.
First, there is a commitment to ensuring meaningful human control and human accountability throughout the lifecycle of a system and the acknowledgement that accountability cannot be transferred to machines.
Secondly, I welcome their willingness to explore the establishment of an
“‘AI-enabled military operator’ skill set”
and to institute processes for the licensing and recertification of operators, including training that covers technical, legal and ethical compliance.
Thirdly, I welcome the commitment to giving force to the ethical principles in “ambitious, safe and responsible”. The Government must become a leader in setting responsible standards at every stage of the lifecycle of AWS, including responsible development and governance of military AI.
Fourthly, I am glad that the Government are reviewing the role of its AI ethics advisory panel, including in relation to our recommendation to increase transparency, which is key if the Government are to retain public confidence in their policies.
Fifthly, I welcome the recognition of the importance of retaining ultimate ownership over data, and making this explicit in commercial arrangements with suppliers, as well as the importance of pursuing data-sharing agreements and partnerships with allies. This is crucial for the development of robust AI models.
Finally, I welcome the Government’s readiness to make defence AI a more attractive profession, including by making recruitment and retention allowances more flexible, enabling more exchange between the Government and the technology sector and by appointing a capability lead for AI talent and skills. This is essential if MoD civil servants are to deal on equal terms with the private sector.
Two cheers so far—the Government could do more. They have no intention of adopting an operational definition of AWS, and I think they must if the UK is to be a more serious player. Perhaps the Minister can update us on a trip down the Damascus road on that one, but at the moment there appears to be no movement.
They do not commit to an international instrument on lethal AWS, arguing that current international humanitarian law is sufficient. If the Government want to fulfil their ambition to promote the safe and responsible development of AI around the world, they must be a leader in pressing for early agreement on an effective international instrument. The reports of the use of AI in the Gaza conflict are clear evidence of the urgency.
Our recommendation on the importance of parliamentary accountability is accepted, but the Government seemingly make no effort to show how accountability would be achieved. Parliament must be at the centre of decision-making on the development and use of AWS, but that depends on the transparency and availability of information, on anticipating issues rather than reacting after the event and on Parliament’s ability to hold Ministers effectively to account.
The Government accept that human control should be ensured in nuclear command, control and communications, but they do not commit to removing AI entirely. However, the risk of potentially apocalyptic consequences means that the Government should at least lead international efforts to achieve its prohibition.
The Government have accepted the need to scrutinise procurement offers more effectively and our recommendation to explore bringing in external expertise through an independent body, but they provide no detail on how they would create standards relating to data quality and sufficiency, human-machine interaction and the transparency and resilience of AWS.
Overall, the Government’s response to our report was “of constructive intent”. I hope that that does not sound too grudging. They have clearly recognised the role of responsible AI in our future defence capability, but they must embed ethical and legal principles at all stages of design, development and deployment. Technology should be used when advantageous, but not at an unacceptable cost to the UK’s moral principles. I beg to move.
There have been a number of very alarmist stories going around. It caused me a certain amount of concern that the committee might think that this is a business that should be regulated out of business altogether due to the possibility of things going wrong. Indeed, while we were on the committee, there was a report in the newspapers of a piece of AI equipment being trialled by the United States that went completely wrong on the simulator and ended up killing the operator and then blowing itself up. We asked the United States what its reaction had been to this. The answer we got was that it had never happened. That might be true—who knows?—but it is slightly sad as we want to learn lessons from all these things, rather like the airlines do when things go wrong. They share the information with everybody in the business and that makes the whole airline business much safer than it would otherwise be. I imagine that, if this did happen, the United States withdrew the whole system from its inventory and went back to the manufacturers and told them to get their act together and not make these sort of mistakes in the future.
My concerns about the committee being somewhat Luddite were misplaced. The report we have produced recognises that we have to take on AI in our defence equipment and that, if we do not, we will be put at a singular disadvantage.
The noble Lord, Lord Lisvane, mentioned the question of international humanitarian law. I am not as much in support of this as perhaps I should be, having signed up to the committee report. I am absolutely certain that nothing whatever is going to happen on this front. The committee was given clear evidence that there is no international agreement to tighten up international humanitarian law. I do not think that we should look to international humanitarian law as an answer to our problems.
The noble Lord, Lord Lisvane, also mentioned the question of nuclear. Unilaterally, we have to ensure that human control remains an element in the whole use of nuclear weapons. I share his concerns about leaving all this to machines: machines can very easily go wrong.
We took a lot of evidence from people who called themselves Stop Killer Robots. I did not understand why so many of them seemed to be put in front of us, but we ended up with these people. When we asked them about Phalanx, they said that this did not apply to that. Your Lordships will know that Phalanx is a point defence system on most of our major Royal Navy ships. It can be used manually or as a completely automated weapons system, identifying targets and opening fire on them if they are coming towards the ship. I would be surprised if it was not being used as an AWS in the Red Sea, where there is the constant threat of Houthi missiles coming in. That system saves the lives of sailors in our Royal Navy.
We should always recognise that AI has a very important role to play. We should be careful about saying that we want to stop all lethal robots, given that they could make all the difference between us winning and losing wars.
In advising the Government to adopt a definition, the committee was careful. While it suggested that a future-proofed definition would be desirable, the report makes it clear that even a more provisional operational definition would be useful. We understand that this could change as the technology dictates, but it would at least have the advantage of reflecting the Government’s contemporary thinking. In lieu of that, we are forced to make a series of inferential leaps in guessing details of the Government’s approach to this question. We are given to understand that the Government use the NATO definition of “autonomous”, which takes us a small step forward, although as the report makes clear, that leaves terms such as “desired”, “goals”, “unpredictable” and, extraordinarily, “parameters”, entirely undefined.
Questions about an agreed definition have vexed policymakers in other domestic and international fora, but we should at least be working towards a definition that would bring some measure of clarity to our regulatory and developmental efforts. I would urge, therefore, the Minister to reconsider the question of an operational definition.
In so doing, I remind the Minister of the evidence of Professor Stuart Russell, who noted that the lack of specificity was creating damaging ambiguity in intergovernmental co-operation, and of Professor Taddeo’s concern that the current definitional latitude allows unscrupulous states to develop AWS without ever describing them as such, and her further exhortation upon this Government to develop
“a definition that is realistic, that is technologically and scientifically grounded, and on which we can find agreement in international fora to start thinking about how to regulate these weapons”.
My third observation relates to the Damascus road. I think that the committee experienced a conversion to an understanding of how, in stark contrast, for example, to financial services, the use of lethal force on the modern battlefield is already remarkably well regulated, at least by the armed forces of more civilised societies. In this context, I think that the committee achieved a more general understanding, confirmed by military professionals, that humans will nearly always be the deciding factor in the use of lethal force when any ethical or legal constraint is in play. Identifying the need to preserve the pre-eminence of human agency is perhaps the single most important element of the committee’s findings.
My fourth comment is that the committee’s deliberations played out in the context of the obscene brutality in Ukraine and Gaza. It was a constant concern not to deny our own people of, if you like, the benefits of ethical autonomy. There is so much beneficial advantage to be derived from AI in autonomy that we would be mad not to proceed with ways to exploit it, even if the requirements of regulations will undoubtedly constrain us in ways that patently will not trouble many of our potential enemies.
My fifth comment, it follows, is on our chosen title, Proceed with Caution. I forget whether this title was imposed by our chairman or more democratically agreed by the whole committee. I wholly accept that “proceed with reckless abandon” would not have survived the secretariat’s editorship, but, on a wholly personal level, I exhort the Minister to reassure us that the Government will not allow undue caution to inhibit progress. I fear that defence is currently impoverished, so it has to be both clever and technically ambitious.
I want to say something by way of wider context. The object of our study, AI in autonomous weapons systems, necessarily narrowed the focus of the committee’s attention on conflict above the threshold of formalised warfare. However, I think the whole committee was conscious of the ever-increasing scale of conflict in what is termed the grey zone, below the threshold of formalised warfare, where the mendacious use of AI to create alternate truth, undermine democracy and accelerate the decline of social integrity is far less regulated and far more existentially threatening to our way of life. This growing element of international conflict undoubtedly demands our urgent attention.
The whole nature of land warfare is changing very rapidly—the noble Lord, Lord Lisvane, used the phrase “fast-moving”—and we do not know what the end state will be. However, we can try to influence it and anticipate where it will end up.
Where Parliament can help is by recognising that most innovation in this space will probably involve partnerships with the commercial sector, often with dual-use civil and military elements, as the noble Lord, Lord Hamilton, argued. In fact, figures from Stanford published in Nature on Monday this week show that the vast majority of AI research is happening in the private sector, rather than in universities or the public sector. The MoD’s and the Navy’s accounting officers and top-level budget holders should be given considerable latitude to use innovative procurement models and development partnerships, without post-hoc “Gotcha” censure from us.
This brings me to my third and final point, which is that we need to be careful about how we regulate. The Royal Navy is, rightly, not waiting for new international public law but is pragmatically applying core UNCLOS requirements to the IMO’s four-part typology of autonomous maritime vehicles and vessels. As for the Navy’s most profound responsibility, the UK’s continuous at-sea deterrent, the Lords committee’s report rightly reasserts that nuclear weapons must remain under human control. Anyone who doubts that should Google “Stanislav Petrov” or “Cold War nuclear close calls”. But the report is also right to argue, at paragraph 51, that this paradigmatic case for restraint is not wholly generalisable. Parliament would be making a category mistake if we attempted to regulate AI as a discrete category of weapon system, when in fact it is a spectrum of rapidly evolving general-purpose technologies.
An alternative approach is set out in the 2023 Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy, which includes key ethical and international humanitarian law guard-rails. That framework is now endorsed by more than 50 countries, including the US, France and the UK, but, regrettably, not by the other two permanent members of the UN Security Council, Russia and China, nor of course by Iran or North Korea. Work should continue, however, to expand its reach internationally.
To conclude, for the reasons I have set out, AI systems clearly offer enormous potential benefits in the maritime environment. Parliament can and should help our nation capitalise on them. Although the committee’s report is titled Proceed with Caution, for the reasons I have given today, the signal we should send to the Royal Navy should be: continue to proceed with speed.
“The machine did it coldly. And that made it easier”,
it is vital to attach human responsibility to each action.
AWS designed according to military culture will, at best, practically strengthen the moral aspects of just war by reducing or eliminating collateral damage, but we should guard against a cultural rewiring or feedback loop that dilutes or corrodes the moral human responsibility that just war depends on. It is reassuring, therefore, as other noble Lords have noted, to see a clear statement that accountability cannot be delegated to a machine in the Government’s response to the report, together with the Government’s commitment to fully uphold national and international law.
Current events across the globe and the rapid pace of development of AI in both civil and military contexts make this a timely and important debate. I commend the committee, and those in government and in the MoD charged with transforming its helpful insights and practical recommendations into concrete action.
Public confidence and democratic endorsement of any plans the Government might have in the development of AI are vital. I therefore urge the Government to commit to ensuring public confidence and education in their ongoing response to this report.
My interest in this stems from the fact that I served with the TA for 15 years, and so I am interested in this country’s ability to defend itself. I worry about what would happen if we start trying to shackle ourselves to a whole lot of things that reduce that capability—I entirely agree with the noble Lord, Lord Hamilton. We should worry about that, because many countries may well pay lip service to international humanitarian law but an awful lot of them will use it to try to shackle us, who tend to obey it, while they themselves will not feel constricted by it. Take, for instance, the international Convention on Cluster Munitions. We are signed up to that, and so are many good countries, but there are one or two very serious countries, including one of our allies, that did not sign up to it. I personally agree with it, absolutely—it is a most appalling munition, because of the huge problems with the aftermath and the tidy-up.
I was also amused by conclusion 8 in the report, which mentioned testing AI “against all possible scenarios”. I seem to remember that there was a famous German general who said, “When anybody has only two possible courses of action, he will always adopt the third”. That is the trouble. I think the British are quite good at finding the third way in these things; that is possibly how we run, because of the unlikelihood of what we do.
The other thing I worry about with autonomous weapons systems is collateral damage. If you start programming a thing with facial recognition—you program in a face and ask it to take out a particular person or group of people, and off shoots the drone to make a decision on it—how do you tell it how much collateral damage to allow, if any? That is a problem. Particularly recently, we have seen that with other things, where people have decided that the target is so important that it is all right killing a few others. But it is not really—at least, I do not feel so. When you create a lot of collateral damage, particularly if it is not in a war but an insurgent situation, you reinforce the insurgents’ desire to be difficult, and that of their family and friends and all the other people. People forget that.
The other thing is that parliamentary scrutiny will be too slow. We are no good at scrutinising at high speed, and things will be changing quite rapidly in this area. We need scrutiny, we need human control at the end, and we need to use AI when it is useful for making that decision, but we must be very careful about trying to slow everything down with overbearing scrutiny.