AI Systems: Risks

Question for Short Debate

Asked by

To ask His Majesty’s Government what steps they are taking to ensure that advanced AI development remains safe and controllable, given the recent threat update warning from the Director General of MI5 that there are “potential future risks from non-human, autonomous AI systems which may evade human oversight and control”.

My Lords, I am grateful to have this opportunity to discuss the most pressing issue facing humanity: the advent of superintelligence. I am particularly grateful to have had the support of ControlAI, a company working in this area, in preparing this speech.

Several weeks ago, MI5 Director-General Ken McCallum warned in his annual lecture about

“risks from non-human … AI systems which may evade human oversight and control”.

But this warning only follows on from Nobel Prize winners, leading AI scientists and CEOs of AI companies that:

“Mitigating the risk of extinction from AI should be a global priority”.

In my opinion, it should be the global priority because of the seriousness of the situation.

The fact is that the leading AI companies are racing and competing with each other to develop superintelligent AI systems, despite the risks they acknowledge that such systems pose to humanity, possibly as early as 2030. For example, the CEO of Anthropic, which as many noble Lords will know is one of the leading AI companies, has assessed the chance of AI destroying humanity at between 10% and 25%. Most worryingly, AI companies are developing machines that can autonomously improve themselves, possibly leading to a superintelligence explosion.

The AI companies are in fact bringing into existence, for the first time, an entity that is more intelligent than humans, which is obviously extremely serious. People talk about pulling the plug out, but they simply would not allow us to pull the plug out. I do not know whether any noble Lords have seen a wonderful film about all this called “Ex Machina”, where the AI does not allow the plug to be pulled out.

In the face of these threats, I urge the Government to take the following steps: first, to publicly recognise that superintelligence poses an extinction threat to humanity; secondly, for the UK to prevent the development of superintelligence on its soil; and, thirdly, for the UK to resume its leadership in AI safety and to champion an international agreement to prohibit the development of superintelligent systems.

If noble Lords can believe it, I was terribly young when I first spoke on this subject: I was here in my 20s as a hereditary Peer. I have had a lifelong interest in this area after reading a book called The Silicon Idol by a brilliant Oxford astrophysicist, and I spoke about my concerns all those years ago. When I spoke on this subject two years ago, I quoted the well-known words of Dylan Thomas, which many noble Lords will recognise:

“Do not go gentle into that good night.

Rage, rage against the dying of the light”.

That, of course, is the dying of the human light. But now I will add WB Yeats’s equally famous poem:

My Lords, I thank my noble friend Lord Fairfax for securing this very important debate. We are already seeing the potential of artificial intelligence being realised as a driving force throughout this country. Businesses are profiting from it, and the country as a whole is benefiting. The United Kingdom has positioned itself well to exploit the benefits of this new technology, with only the United States and China being ahead of the UK in total investment into AI.

With this growing industry come the external threats and risks of a new unregulated technology. National security is perhaps the most pressing issue, but every week there is a new story in which AI has been used for malicious ends. Whether because of deepfake images or fraud, it is clear that the technology employed needs appropriate oversight.

The last Conservative Government were committed to ensuring that the UK becomes an AI superpower with thorough safety rules. As Prime Minister, Rishi Sunak hosted the AI Safety Summit, which saw the signing of the Bletchley declaration, bringing academia and industry together with the representatives of 28 countries. The AI Safety Institute was launched, set up to test new types of AI before and after they are released. The institute now has more researchers than anywhere else in the world and provides the safety check on new technology without stunting innovation.

The question remains as to how this Government will eventually seek to regulate artificial intelligence. To give AI companies complete free rein would be imprudent, but the Government’s current approach has not established business confidence. Despite promising to bring forth binding regulations on the most powerful AI models, the Government have stalled. Regulation was supposed to be introduced this parliamentary Session but, as we approach Prorogation, the policy is still yet to be seen. On the one hand, companies are expected to prepare for potentially heavy-handed measures, yet on the other they are left in a state of flux. That is not the way to encourage growth in a booming industry.

My Lords, I commend the noble Lord, Lord Fairfax, on raising what is rightly a fundamental question of our time: the risk of AI systems becoming more powerful than their human creators. Advanced AI does not become unsafe in a vacuum; it becomes unsafe by design when it is developed without accountability, driven by profit incentives of private actors and embedded in infrastructure that the state can neither inspect nor exit. Alongside concerns of runaway capability is the risk of dependency. Long before any dramatic accident or attack, we risk a growing reliance on a narrow set of foreign-owned technologies, leaving the UK unable to act as a sovereign state with values, choices and technologies of its own.

Just this week, the Ministry of Defence awarded a £240 million contract for “critical operational decision-making support” to Palantir. The issue here is not one company but a pattern of outsourcing our national infrastructure to American firms, backed by a US Administration whose national security strategy states plainly: “In everything we do, we are putting America first”.

Across the economy, we have normalised deep vendor lock-in to US companies to the point where security, critical industries and sensitive government departments cannot credibly switch suppliers, even as the risks or the terms of engagement shift. One security expert recently described it to me as economic warfare, where they create strategic advantage by advancing their domestic industry and technology while simultaneously degrading the same capacity in adversaries and allies alike. Where the state cannot inspect, audit or exit the systems that shape its decisions or handle sensitive data, it has no sovereignty.

The US and China are determinedly ahead, but many AI experts believe that the next phase of AI will favour systems that are reliable, auditable and governed by understood rules. That is where the United Kingdom has an opportunity.

My Lords, I also want to congratulate the noble Lord, Lord Fairfax of Cameron, on securing this question for short debate on such a timely issue. AI is an incredible development for many reasons—R&D, innovation, economic growth, productivity, faster health diagnoses and many other areas. However, this must be balanced with a regulatory environment which allows and encourages all those positive things and provides safeguards against harms. We must be risk-aware, and I hope that the Minister will be able, in closing this debate, to set out where the Government are with their risk analysis and action plan to deal with those risks.

Two sorts of harm can occur with autonomous AI systems. The first is intentional harm, which I hope could be identified and regulated in a straightforward manner. It is the second type of harm—unintentional or reckless harm—which may be more difficult to detect and, therefore, to regulate. So-called superintelligent AI is the riskiest type of AI. Sometimes, as the MI5 director, Kenneth McCallum noticed, it would be reckless to ignore.

Serious harms from AI have already begun to materialise. Before Christmas, I asked the Education Minister in the House a question about the fact that toys with AI, such as speaking teddy bears, were unregulated and had the potential to be very dangerous indeed to very young children. If children interact with AI chatbots and toys instead of their parents, guardians and friends, that could lead to serious harms. There have been documented cases of health deterioration and tragic instances where young people have taken their lives after forming attachments to these systems.

Modern AI systems, I understand, are not built in a piece-by-piece fashion, like a machine, but grown. That means that no one, not even the initial AI developers, understand the AI they create. That is frightening indeed.

My Lords, I also thank my noble friend Lord Fairfax of Cameron for securing this hugely important debate. Like other noble Lords, I very much acknowledge the transformative potential of AI, not least in areas such as medicine.

However, there are dangers. We would be mad to ignore them because many of the same people who built this technology—people who have won Nobel Prizes and Turing Awards—are warning that AI poses an extinction risk to humanity. Hundreds of AI experts recently co-signed a letter that said:

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war”.

Among the signatories, you will see the names of Sam Altman—CEO of OpenAI, as noble Lords will know—and Geoffrey Hinton, often referred to as the godfather of AI.

A separate letter signed by Elon Musk and Steve Wozniak, among many others, reads:

“AI systems with human-competitive intelligence can pose profound risks to society and humanity”.

It also calls for a moratorium on the next generation of AI until we know more about it. These people cannot be dismissed as Luddites or technophobes. They are the architects of this brave new world. They recognise that superintelligent AI is far more powerful than any of us can understand and has the capacity to overwhelm us.

It is not just that we do not understand where things will end up; we do not even understand where things are today. The CEO of Anthropic, one of the world’s largest AI companies, admitted:

“Maybe we … understand 3% of how”

AI systems work. That is not that reassuring. We have already had a glimpse of what losing control looks like; for example, in an experiment, an Anthropic AI system attempted to blackmail its managing engineer when told that it was going to be shut down.

My Lords, I thank the noble Lord, Lord Fairfax, for securing this debate. I begin from a position of some scepticism. As many experts say, there is a great deal of hype around AGI—so-called human-level intelligence—but increasing numbers of experts are saying that, unless we get to at least physically embodied intelligence, the large language model-type approach will never get us to that point.

However, my intervention is about a different risk from the generative AI being promoted so much now: resilience. It starts with an incident that is not about AI at all. I shall take noble Lords to Berlin on Saturday, where 45,000 households—including some 2,000 businesses, four hospitals, 74 care homes and 20 schools—found themselves without power. It was not until Wednesday that a significant number of those were reconnected. Unsurprisingly, in Berlin there is now a lot of discussion about vulnerability and today, our own local think tank, the Council on Geostrategy, is highlighting how by cutting just 60 undersea cables, or a percentage of those, we could see a 99% cut in data flows. Imagine the financial impacts and the impacts in societal chaos.

To make this practical, I note that the Government are funding Northumberland County Council, through the £200 million flood and coastal innovation programmes, to trial a flood prediction service for six catchments in that county that are particularly vulnerable to flash floods. Such flashy catchments have a big problem with traditional models of flood warning, so maybe AI can provide the solution. But of course, that is dependent on electricity, data flows and cyber systems that are functional and have not been hit by some kind of malevolent force.

More than that, what kind of data are we relying on? Is it spoiled or polluted data? It is not impossible to imagine that being through human agency. I see that Northumberland is also trialling the use of AI in considering flood risk in planning applications. There could be a lot of money at risk there. Even if there is not an active agent, is it taking adequate account of the changing weather resulting from the climate emergency? Maybe with enough data it could allow for that, but would it also allow for changing human behaviour, ageing populations or loss of trust in authority?

My Lords, I think we are all grateful to the noble Lord, Lord Fairfax, for giving us this valuable opportunity today. When, within our lifetimes, we experienced the introduction of the world wide web and the internet, we could see the opportunities that that technology brought—but it also brought harms, something we have learned far too late to deal with. We debate almost every week in the Chamber how we are going to deal with those harms. If intelligence is about anything, it is about learning from that past experience and using that knowledge to avoid repeating the mistakes with AI. It is not Luddite to express the concerns that have prompted this debate today on the safety and development of AI. I believe we are already behind the curve.

I am not an expert on this subject but, as an older person, I have concern for future generations, not just in my own family but in this country and the world, and that is not an exaggeration. It is not just on a domestic basis that we see this; it is interesting that we have seen some of those organisations that one might think have a vested interest already expressing their concern. We have received a briefing for this debate from the Institution of Engineering and Technology, and it is interesting that it should say:

“AI safety and the assessment of risk must go beyond the physical, to look at financial, societal, reputational and risks to mental health, amongst other harms”.

If that is what industry is telling us the potential harms are, we should already know how we are going to control it.

I hope that today’s debate will ensure that, when the Minister responds, he will give us some information not only about what the Government are intending but about what timescale the Government are working to, because clearly industry also thinks that we are behind the curve. The institution also talks about standards and transparency, and says:

My Lords, like others, I thank the noble Lord, Lord Fairfax. I agree with almost everything he said, bar one aspect that I think was optimistic, which is that we have a five-year window—I fear it might be even less than that. I also agree with the concerns of the noble Baroness, Lady Foster, about the impact of AI chatbots on the well-being of children, but like all of us I am even more worried about the risks of development of superintelligence.

I can discard some of my quotes because your Lordships have heard them already, so I can be a bit briefer, but I will give another Anthropic quote from Jack Clark, who is the co-founder and head of policy. He said:

“We have what I think of as appropriate anxiety and a fear of hubris. This is a huge responsibility that shouldn’t be left only to companies. One of the things that we advocate for is for sensible policy frameworks that make our development practices transparent. I think a larger swath of society is going to want to make decisions about these systems. It would be a failure for only the companies to be making all of the judgment calls about how to build this”.

If the AI executives are worried, I am worried and we all should be worried. Although the AI Security Institute is a step in the right direction, despite manifesto commitments, as your Lordships have already heard, we do not yet have legislation, and without a legislative framework we are really at significant risk.

The AI company executives talk about how they are taking decisions to try to teach their AI systems to value human life above the AI superintelligence, but they should not be the ones having to think that is a good thing to do. We must act before we reach the point of no return and the genie is out of the bottle.