Uncorrected oral evidence: Multilateralism
Oral evidence, 28 October 2026.
The Chair
Welcome to this fourth session of the House of Lords International Relations and Defence Committee and our investigation into the multilateral aspects of the governance of artificial intelligence. The session is being streamed live on parliamentlive.tv and there are some people who are interested enough to listen to these things. Before they ask any questions, I will ask members of the committee to declare any interests they might have. I declare that I am the senior counsellor for the Cohen Group in Washington. You will get a transcript of the session today. If you could expeditiously return that with any tiny technical changes you might want to make, that would be useful. I will start the questioning by talking about the whole question of the European Union and Europe in general and the governance of AI. It is said that Europe is lagging behind both the United States and China on artificial intelligence. Do you accept that characterisation of the current debate? If that is the case, can the European Union and Europe in general shape the international environment in order to capture whatever standards might eventually be adopted on a global scale? Let us start off with Nick, who has just got off a plane.
Nick Moës
I would perhaps bring more nuance to the idea that we are lagging behind across the whole stack. The US and China are clearly leading in frontier AI development and the amount of investment in compute, for example. But the EU leads in some aspects of the stack. They are the overall nuances that we have to bring out. Also, Europe—and this applies beyond the EU—and other Governments that are neither the US nor China, do not need to be a leader in frontier AI development to be able to shape the rules for governing it. That is the second aspect that brings more nuance. Being behind when it comes to the critical frontier AI model development itself makes international co-operation a lot more urgent. Therefore, the ability to use other parts of the stacks and other points of leverage is important and will help to shape those international standards, norms or whatever regulatory regimes may slowly emerge. Market size, industry capabilities, talent pipeline, regulatory powers and specific stacks, such as ASML or the production of lithographic machines by Zeiss or Siemens, are other technical layers of the stack where the EU is very well positioned. From that bargaining power, there could be a stronger influence than most might expect over how to produce and develop frontier AI. This is especially true when it happens in collaboration with other Governments or so-called middle powers, such as the UK itself. I believe that the EU, the UK and their partners can build very practical arrangements that protect them now and make wider agreements on this international regime easier to reach. I am happy to dive a bit deeper into this, but it might also be useful to get into the details a bit later. However, there are a lot of precedents that have shown the bargaining power that the EU has maintained, and I hope that we will get to that at some point in this hearing.
The Chair
You are saying that the lack of any technical capability inside Europe as a whole would not inhibit the chance of determining what the international standards were?
Nick Moës
A lack of technical capability is a mischaracterisation, because that is not the case across the stack. There are significant technical capabilities at the layer of, for example, chip design, lithographic machines and some of the elements in the chips that are key to making the data centres that eventually train the very large models. That is the same with the gas turbines that are necessary to power those data centres. The EU has quite a bit of technical capability in certain spaces that is unique worldwide and can therefore use that as leverage. Beyond technical capabilities, and I would in fact question the premise that technical capabilities are that influential. Market access, for example, and the 450 million fairly rich consumers, relative to the rest of the world, that the EU market provides is a significant bargaining chip when setting up safety requirements, which basically makes market entry conditional on satisfying some of the safety standards that the EU may want to see shaped. That is why we have seen companies—American companies in particular—that are very interested in shaping international standards, listening and taking a cue from what the EU’s preferences are when it comes to safety and security standards. This is not done in a happy way; they perhaps feel a bit constrained by the EU preference for safety and reliability, but it is definitely occurring. We have seen companies adapting their safety frameworks globally to comply with EU rules. There is a Brussels effect here that seems to be in play.
The Chair
That is interesting. Daan, could you address the same points?
Daan Jujin
Thank you very much. By way of background, I am the director of research at the Arq Foundation. We are a philanthropically funded think tank based in Brussels and our mission is to help Europe to prepare for transformative AI. I am very glad to be here today. It is a privilege to be able to speak. I have a slightly different take on this question to Nick. I agree with your characterisation that Europe is lagging behind the US and China on most parts of the AI stack, and I think that this is an important disadvantage for the EU and the UK in their ability to shape the international environment. It is often said that in AI there is no number three, and I tend to agree with that. The US is clearly in first place, China is in second place, and for a long time no one has really come close to them. Europe in particular has relatively poor positions in multiple parts of the AI stack. For instance, when it comes to frontier models themselves, the best European model is currently roughly one and a half years behind the frontier of AI, according to Epoch AI, which is an independent AI research, monitoring and evaluation organisation. Europe does not own any of the big players in the chip design space. Those are Nvidia, AMD and Broadcom, which are all American companies. Europe does not own any of the leading-edge fabs where AI chips are fabricated. All the big cloud providers are American-owned. When it comes to the physical compute on European soil, Europe hosts only 5% of that, compared to more than 70% in the US. All that does not mean that Europe has no influence whatever. Europe has already shaped the international environment, in particular through the AI Act and the code of practice for general purpose AI signed by most leading AI developers. However, the EU has come to realise that owning the world’s leading rulebook is not sufficient to shape global AI developments, especially since this technology is becoming increasingly securitised. Together with seven other leading European AI researchers and think-tankers, I wrote a scenario called “Europe 2031” over the summer, which lays out what it could mean for Europe to get caught in an AI competition between the US and China and be left without any leverage over developments. The picture we painted there is quite bleak. The problem is quite dire. We have to realise that some of the solutions to addressing these weak positions will take time, and we may not have a lot of time.
The Chair
Indeed. Baroness Crawley wants to ask a quick supplementary question.
Baroness Crawley
Either Daan or Nick can answer this. You have both talked about the stack. Can you remind us in layman’s terms what the AI stack is made up of? What are the layers, and where is Europe strong and where is it weak?
Daan Jujin
I can take a first stab; Nick, feel free to add. The AI stack refers to all the technologies that lead up to a model eventually being served to a customer. That starts at the very bottom of the stack, with companies such as ASML, which Nick mentioned. ASML is a globalist monopolist on so-called lithography machines, which are very big machines that cost £200 million apiece and are used to print the AI chips. The AI chips are designed by AI chip designers, most famously Nvidia, and are then fabricated in so-called chip fabs. The leading player there is TSMC. There is also Intel and Japanese players using the lithography tools from ASML. The chips are then shipped to data centres that are operated by cloud providers, and there they are powered by a bunch of electricity to eventually train the models or serve them to customers. All these different layers together are called the AI stack.
The Chair
That has certainly enlightened members of the committee
Nick Moës
If I may, I just want to give a general addition. The part of the AI stack that is relevant to the UK also includes the energy and the critical raw materials. Building on what Daan said, expanding on what is even earlier in the value chain, it seems like the landing, the siting, the water and cooling, and critical raw materials might be relevant, especially for questions of international relations with your allies. Also, sometimes in the downstream, the enabling environment such as the talent and financial resources come into play as well. That would definitely be of relevance to the UK’s policy, I believe.
Lord Lamont of Lerwick
My question is rather similar to the one that the Chair put, although the answers were amplified by Baroness Crawley’s very relevant question. How would you characterise the EU’s regulatory vision on AI? Following on from that, what are the chances that the EU’s regulatory approach could become the de facto global approach to regulation, particularly with the so-called Brussels effect—the tendency for EU regulations to be a baseline template for other countries? Nick, you were saying that despite the disparity in investment and the huge disparity in the number of companies involved in AI, it was the position in the stack that gave the EU a degree of leverage that we might otherwise have been deeply sceptical about.
Nick Moës
The numbers that Daan provided earlier are unquestionable. I think that Daan and I are saying a similar thing but it is a case of glass half-full versus glass half-empty—or perhaps glass 20% full and 80% empty in this case. I do think the stack is an important factor in influencing the outcome, but indeed the regulatory vision and capabilities are relevant, as you have alluded to. It might be relevant at this stage for me to introduce myself, since I forgot to do so. It is of relevance. I am vice-chair of the scientific panel appointed by the European Commission to assist the Commission in how it deals with frontier AI systemic risks and the enforcement of the AI Act. This is a relevant affiliation. I am also the executive director of the Future Society, which is a think tank that has, for the past 12 years, looked at questions of AI governance and geopolitics in the US and the EU. That being said, to answer your questions, there are two main principles that characterise the EU’s vision on AI. The first is: heavy is the head that wears the crown. In this race to ever more powerful AI or artificial superintelligence, the EU sees that whoever is winning that race, as far as general AI goes, poses the most risk and therefore should be faced with more safety requirements. Therefore, the regulation has different tiers. The most powerful tools that exist—frontier AI or general purpose AI with systemic risk—should face much greater scrutiny when it comes to safety requirements. The second principle is: beware of geeks bearing gifts. This is the notion that for 15 to 20 years there has been in the EU a certain conflict with the digital platforms and tech companies and there is now a widely held belief that those digital platforms and tech companies, be they from the US or China, do not have the EU’s best interests at heart. Therefore, the strategy behind the EU’s regulatory vision has been to improve its bargaining powers vis-à-vis these companies by ensuring that there are additional requirements and means for pressure under the rule of law and the normal legislative and democratic process—nothing arbitrary—to equip society with tools to say no when these platforms or technology providers do not respect our values. Those two principles characterise that vision. You asked also about the Brussels effect and whether this vision is spreading across the world. Again, while reminding everyone that I am appointed by the European Commission and that that therefore probably plays out in my assessment, I would say most certainly that there has been evidence that companies abroad—especially US companies but also some Chinese companies—have complied and even started lobbying for rules abroad that espouse the approach of the EU. The Brussels effect is not accidental. You have to bear in mind that the EU rules have been significantly shaped by US companies through the code of practice that Daan mentioned earlier, co-drafted with the leading industry players alongside over 1,000 stakeholders from within civil society, academia and other industries outside of that of the providers. There is still significant weight given to the voices of US companies, so it is normal that the rules, which I believe are the strongest binding rules that exist in the world right now, have been quite accepted within the industry, at least on the US side. We have seen, for example, OpenAI writing to the Governor of California to ask that the EU-led code of practice be also applicable in California and provide presumption of compliance with California. The Brussels effect exists, as evidenced by the fact that the industry proposals we are now hearing about—frontier AI regulatory organisations or new forms of Bills in the US and across the world—from Google and Microsoft at least tend to echo and harmonise with the EU’s ideas. The companies have changed their internal governance frameworks to comply—so, compliance has taken place. The code of practice is still a voluntary tool, yet all companies have either signed it or, in the case of those who have not, including some Chinese companies and one American company, are it seems de facto applying it concretely, from what we know from the outside. The EU rules have strengthened Europe’s hand, so this notion of improving the bargaining power seems to be working so far. It is difficult and uncomfortable, especially with the current US Administration applying a lot of pressure. But even beyond AI, the EU digital rule and also some of the UK’s rules have been helpful in influencing the behaviour of these companies. For example, X was fined under the Digital Services Act—the platform regulation in the EU—and despite significant screaming and pressure from the US Administration and Elon Musk calling for the abolishment of the EU, in the end X paid the fine, appeared in court as per the usual process, and has modified its practices in order to maintain access to the EU market. It will not keep spreading on its own. We definitely see the Brussels effect, with legislators in Brazil, the UK, Korea and Japan wondering whether it is working or not. The Brussels effect works if everybody thinks that it is a good idea for it to work. Of course, with the pace at which technology is evolving, it is very important that we keep questioning whether the bar is being set high enough in terms of safety and security. Lord Lamont of Lerwick: Daan, you took a slightly different view in your first answer. Perhaps, briefly, you could comment on the second one. Daan Jujin: Thank you, Nick, for the clear explanation. I agree with Nick that you could characterise the EU’s regulatory approach as a risk-based approach, with different tiers and levels of stringency for different kinds of models. It could be informative to explain how the EU’s regulatory approach has evolved over time. The European Commission began drafting the AI Act all the way back in 2019. This was three years before the launch of ChatGPT, and large language models were not really a thing back then. In a sense, the European Commission was very prescient—already back then, it noticed these risks from AI emerging. But being so early also made its job quite difficult, because over the coming years, we saw the emergence of general purpose AI models—tools, such as ChatGPT, that can be used for all kinds of different purposes. The draft legislation that the Commission had written before then was very use case specific. It was more focused on how different AI models were applied in different domains or jobs. Then, over time, it became clear that something was needed for these general purpose models, and the general purpose regulation was bolted on top of the AI Act. Eventually, that was operationalised through the code of practice, which Nick mentioned. Over this period, we have seen a shift from this broad, use case-based regulation towards a narrower and more targeted regulation aimed at addressing larger-scale risks from AI systems which are aimed at the model layer or even at the entity developing the model. For instance, the code of practice requires providers to set up a safety and security framework that applies to all the frontier models that they train in-house. So, there has been a clear evolution towards more targeted and more model-based and entity-based regulation. In parallel, we have seen a greater focus emerge within Brussels on competitiveness and sovereignty. Five years ago, the EU was happier to trade off competitiveness and sovereignty for public safety than it is nowadays, although often, these two do not really come in tension as much as people claim. When it comes to the Brussels effect, I am more sceptical than Nick. I think that we will see a full-blown Brussels effect, where many jurisdictions will adopt very similar regulation to the AI Act. But we have already seen firms adapt their behaviour as a result of the AI Act and the code of practice. To give an example, the code of practice requires AI developers to watermark the text output of their models. We have seen both OpenAI and Anthropic adopt that recently, and it is certainly not clear that they would have done that without the EU. In Anthropic’s case, it even rolled this out globally to all its customers, even outside the EU. However, I should also point out the limitations here. Not all the leading AI developers have signed the code of practice. Meta, for instance, has not done so. As I said before, when military and national security concerns become more pressing, they will likely override the commercial interests of these model providers. That means that the AI Act will not be able to touch all of them. I especially hope that the code of practice will form a foundation for a future foreign regulation or international government and governance. The EU’s technical tools and regulatory knowledge can really play a part there. Such roles, as the story of the AI Act shows, have to constantly evolve as well. The Chair: Thank you. In response to future questions, might you try to be a little bit shorter and pithier on the points that you want to land? Lord Ahmad has the next question. Q32 Lord Ahmad of Wimbledon: I think you are already on the pathway to answering part of this question, but the question is specifically about what the EU is doing in terms of global governance, when it comes to AI in other multilateral fora. Our inquiry is focused on that. Bearing in mind the directive which already exists, you both touched on the bilateral engagement of the EU which is taking place. Perhaps starting with Nick, where are we when it comes to the multilateral setting? Nick Moës: The EU has worked through many international fora and has reiterated a few calls recently. The EU has been active in the G7 Hiroshima AI process, the OECD, the UN Scientific Panel on AI, the Council of Europe treaty as well as the International Network of AI Safety Institutes via its European AI office. The EU recently signed the call for control of the advanced models that emerged from Norway and Finland, which was signed by over 32 countries. It was a call to explore the creation of an international institution to prevent the loss-of-control type of accidents that we saw over the summer. Ireland and other EU member states came up with the Dublin declaration just two weeks ago. There was significant room in the EU-UK summit and the EU-Canada summit for work on AI and frontier AI co-ordination. The European Commission’s stance is to work together on testing AI models’ early warnings about the incidents and the type of accidents that we have seen over the summer, and on checking safety claims independently from the labs. We are missing international mechanisms through which the governments—in the EU or the UK, or any other governments—can respond and contain the emerging AI incidents we are now seeing spread. At the multilateral level, quite a bit of work has been done, but there is still a significant gap, which is the inability to have a multilateral response to the AI crisis and to share warnings across borders. Lord Ahmad of Wimbledon: Daan, you have touched on the whole issue of the code of conduct already, but we have the United Nations. Is that a forum that should be utilised? The UK is going to be chairing the G20. Taking the point Nick just made about the G7, is that something that can translate to the wider G20 as well? Daan Jujin: We should address the elephant in the room, which is that the US “totally rejects” any form of a global AI governance regime. China is more open to it, but it wants to build a global regime on its own terms, for instance, through WAICO, the World Artificial Intelligence Cooperation Organization. It is important to look in the eye the fact that international governance approaches that may have worked 15 years ago in other contexts may not work nowadays. In AI, relations between Europe and the US have frayed and the world has become increasingly bipolar. Compared to the debates around social media, for instance, that we saw a decade ago, AI is a very different technology; it is inherently a dual-use technology, which comes with great national security concerns, as we are seeing with the influence of AI and cyber today. During the last UN General Assembly, the US President said that whoever wins in AI wins, period. This is very much the US approach: it is very much, “We have to beat China in this existential race for AI dominance”. Further complicating things is the fact that there is not a lot of time to carefully construct some kind of global regime or treaty. AI labs are racing towards models that are broadly more intelligent than humans across the entire range of economic tasks. They are on track to automate the process of AI R&D itself and are on track to do this by the end of the decade. There is very little time, and the international landscape is very strained. So, in my opinion, international AI governance will look more like crisis management than careful treaty building. Therefore, the role of the UN will probably be more limited. The UK and the EU can contribute with technical knowledge and tools. The EU can demand transparency in the processes by which these AI models are developed. The UK can provide its excellent evaluation expertise through UKAC, and the two of them— and I think this is very underrated—can build the technical verification tools that will enable a future deal between the US and China to be binding. Currently, you may be able to see a data centre from space, but you are not able to see what is going on inside the data centre. So, we need tools to verify that some hypothetical future deal will actually be upheld by both these superpowers. Q33 Baroness Prashar: Can we now move to the EU itself? To what extent do EU member states speak with one voice when it comes to the global governance of AI? Do the national initiatives by major EU member states suggest divergence in the EU? Daan Jujin: There are high-level agreements on rules. Disagreements on the EU’s rulebook were ironed out during the AI negotiations. The AI Act applies to all member states. When it comes to high-level AI strategy, there are differences of opinion between different member states. For instance, not all member states signed the Norwegian and Finnish call for control of frontier AI. Some member states individually entered Pax Silica, whereas others have not. Some countries are backing their own national champions. Most of these differences can be overcome. One thing that I worry more about at this moment in time is differences in situational awareness—the sense of urgency that different member states feel when it comes to AI, their understanding of where this technology is headed and the incredible challenge of steering this technology in a more responsible direction from the current European position. We have recently seen the lack of urgency and the difficulty in, for instance, scheduling a European Council meeting dedicated to AI. Nick Moës: I echo what Daan said, especially the part on situational awareness. Even though the work of Europe 2031 has been key in increasing the situational awareness, we still see different levels of prioritisation of AI. However, today, pretty much all EU member states care about AI safety, security and sovereignty, but as usual with EU matters, the domestic political cycles affect the reaction to this. As Daan seems to assess, it can be overcome, especially at the international level. It seems like France and Germany, who are two important players, are quite interested in the call for control of the frontier models. There will always be political or politicised statements that are made here and there. The Chair: Lord Darroch has a supplementary question. Q34 Lord Darroch of Kew: I am interested in the dynamics of the debate within the EU on this. You have both characterised the EU approach as being essentially risk-based and focused on safety and security. Some critics of the European approach say that it is treating AI predominantly as a threat rather than an opportunity. So, I just wonder whether there is a consensus around the EU table on what could be quite a heavy regulatory approach; or is anyone arguing for lighter regulation and talking about the risk that if EU regulation is too heavy and intrusive, we will miss out on the AI revolution? Nick Moës: As Daan mentioned, the debate has been ironed out, during not only the AI Act discussion but the recent Digital Omnibus, which streamlined a lot of the digital rules that that came up. So, this has indeed been addressed. AI is very much seen as an opportunity; all technologies in the EU are very much seen as an opportunity. Over half the large European companies are now using AI in their workflows. There has been significant investment in the EU, which has increased year to year, from €10 billion to €17.5 billion. In the first half of 2026, €26 billion has been invested. I am happy to check these numbers later. So, I think there is a significant appetite for the adoption of AI. The risk that people are seeing is that AI is a risk to European sovereignty, because it is mostly produced in the US at this stage. Of course, a safety and security risk is materialising. We witnessed it over the summer with the accidents with OpenAI and Anthropic vis‑à‑vis Hugging Face and various government websites. But the risk in terms of dependency on foreign providers is very much top of mind. The regulatory tools that have been built in the EU are seen as a way to reclaim some of that sovereignty and do not seem to be hindering innovation, at least if the VC investment figures are indicative of anything. Daan Jujin: I agree with Nick. European leaders are keenly aware that AI brings both tremendous opportunities and tremendous risks. If you listen to some of the speeches they give, they almost always start out the same by saying that AI can make our world much better, but there are also grave risks that we must address. This has become somewhat of a running gag, but it is true. I personally believe that AI will revolutionise science, enable us to cure many diseases and enable us to better tackle climate change, but the risks are real, and I am happy that the EU is on the ball. As Nick said, over the summer, at multiple AI labs we saw breaches of containment—so-called AI swarms independently escaping their virtual environments, collaborating and breaking into different external companies. This is very worrying, and it is right that the EU is focused on these risks. Europe’s weak position currently is not only due to regulation. In a broader sense, regulation definitely played a part, but there are many other reasons, such as fragmentation and underdeveloped capital markets, which you can read in the Draghi report. Q35 The Chair: On the election results in France and Germany next year, if Marine Le Pen wins the French election, will that change the equation at all? Is the AfD in Germany having a greater influence than it had? Does that change the position that you have articulated? Nick Moës: AI might be only one of the topics that would be greatly affected by such a victory. The links of these parties with Elon Musk or Russia might affect their stance on technology policy and international governance. It is something to watch. However, interestingly, I have noticed that even in some of these extreme right-wing parties, there have been significant concerns about the risks and the existential threats. In the US, Steve Bannon is extremely concerned about the power of the tech companies and the new companies trying to develop artificial general intelligence software. This indicates that the MAGA movement and the extreme right-wing parties in Europe that echo some of this might have stances that are not as deregulatory, or not as pro-AGI companies, as one would expect. It is a big unknown, because I do not think there is an internally agreed upon stance in these political parties. The Chair: Daan, do you agree with that?
Daan Jujin
Election outcomes in France and Germany can make the whole AI ecosystem fairly unpredictable. One thing I personally worry about is the tendency towards more nationalism, looking inwards, whereas what is really needed for European countries is to work together more intensely with like-minded countries like Canada, South Korea and Japan. All these countries are somewhat in the same boat: they have developed economies but relatively weak positions in AI. If they join up their bargaining power, they can have more influence and better shape the trajectory of AI. I worry about more fragmentation and more nationalism.
The Chair
Thank you, that is interesting.
Lord Grocott
I am not quite sure about this question any longer. To the extent that the EU has a common approach, or you could describe it as having some sort of agreement in relation to governance of AI, so let us assume there is a common EU approach—although I am not sure there is; that is clear from what you have been saying so far—how does that differ from the UK’s approach? How is it changing? Is it getting closer to the UK’s approach if it is wide at the moment, or vice versa? Would you like to reflect on that?
Daan Jujin
There is strong alignment between the UK and the EU when it comes to goals. Both the UK and the EU have said explicitly that they want more transparency over AI developments, they want these AI systems to be safe and reliable and they want to have sovereign alternatives in order to not be fully dependent on products from the two superpowers, but their approach towards these ends has been different. The EU has a much larger market and therefore has been able to rely more on regulation—on incentivising companies to adhere to their rules and standards to enter the EU market—whereas the UK is a smaller market and is therefore focused more on building state capacity and societal resilience through investments and technical non-regulatory means. The UK’s AISI is a great example of that and a real inspiration for the rest of the world. We are also seeing the UK take other measures like the Sovereign AI Fund, the new AI Economics Institute and the newly announced AI Biosecurity Lab. These are precisely the types of interventions that are suited to the UK, given its position, and that can offer inspiration to other countries. There is also room for more collaboration between the UK and the EU—for instance, between the UK’s AI Security Institute and the AI Office, which is sort of its counterpart at the EU level. The AI Office is looking for more evaluation expertise that the UK has plenty of, while in turn AISI is looking for better guarantees that it will retain access to these frontier models in order to test them before pre-deployment. There is room for collaboration there, and I think the upcoming G20 presidency provides an opportunity to strengthen those ties. I do not believe the UK should try to fully become part of the EU’s regulatory orbit. I do not see much upside there, and it could come with large downsides, for instance, in the relationship between the UK and the US.
Nick Moës
When it comes to alignment between the US approach and the UK’s approach, Daan has covered most of it. There is some greater convergence, and nothing made that clearer than the UN Security Council in September, where both Foreign Secretary Miliband and French Foreign Minister Barrot have basically alluded to not only the same risks from AI but the same approach to preventing them, so I think the convergence of goals is there. The UK has historically led the world in international governance—in the 20th century, with maritime safety, nuclear test bans and the verification regime that came with it, and in the 21st century when it comes to financial stability and climate change. In this space the UK has done the Bletchley summit, and now the EU has come in with a complementary piece of the puzzle of international governance. The UK has provided the momentum and the impetus while the EU has provided maybe the first set of regulations. As Daan and as I have mentioned, the UK can probably address the very specific risks that have manifested over the summer—the questions of loss of control and challenges to cyber security and resilience from agentic AI. That is something that the EU, because its law is now already a year and a half old, would have a harder time addressing, so the UK can probably do more work on that front in terms of crisis prevention. The current convergence seems like a natural way to work together on an international—or perhaps minilateral, rather than multilateral—regime to work on the more acute and most severe risks that have come up. Everything else that Daan has said I am just echoing—plus one.
Baroness Blackstone
There has been quite a lot of publicity for the Council of Europe’s framework convention on AI, which has been welcomed as the first legally binding international treaty on AI. Could you tell us how you assess its importance? Do you regard it as significant, or not so much?
Daan Jujin
I must admit that in preparing for this session I had to look up again what the framework convention said, which is somewhat telling. On the positive side, it is the first legally-binding international treaty and, if ratified, it will commit states to keep AI activities consistent with human rights, democracy and the rule of law. However, the principles that are outlined in the convention are very high-level. They are mostly aimed at current challenges from AI systems: bias and misinformation. They do not address future challenges like the loss-of-control risks that Nick just mentioned. Furthermore, there is no verification mechanism. Implementation is left to national jurisdictions. China has not signed it. The US has not ratified it and will not do so in the foreseeable future. It is a step in the right direction but it is not the end solution that we need here. If the UK were to ratify this convention then that would provide symbolic value, and it would count towards the five ratifications that are needed for the convention to enter into force, but this is really a start, not an end solution.
Nick Moës
Regarding its significance, as Daan has mentioned, there is a problem with its scope; it does not address the foundation models, the general-purpose models or frontier AI explicitly. However, it may be significant in that it shows that rules on their own, even if they are agreed upon by many treaties, are nothing without proper enforcement. A key takeaway here is that, whatever regime either will be built domestically in the UK or has been built domestically in the EU, these rules are only as strong as their enforcement. In this case—because of the dual-use nature of this technology, the significant power that comes with it and the competitiveness gain—that enforcement will require a lot of diplomatic engagement and more pragmatic cross-border co-ordination between authorities. California has regulated on frontier AI, and that is the kind of player that the UK would probably be collaborating with to effectively enforce any rules, but unfortunately the Council of Europe’s convention does not have a credible enforcement mechanism.
Baroness Blackstone
So you both agree that it is not that important. But it is.
Nick Moës
It is a symbol that countries can come together to an agreement, but it also highlights the problem of enforcement.
The Chair
Field Marshal Lord Houghton has a predictable question.
Lord Houghton of Richmond
You have slightly neutered my supplementary with that response. I am going to have to ask you to speculate a bit. We are in the International Relations and Defence Committee, so my question really is more generally about how you see the future of the international regulation of AI as it is exploited by the military, particularly in autonomous weapon systems. I suppose thus far the UK is a mix of technical regulation and the desire to retain human agency in decision-making, but it has not found any sort of rigid form yet. Nevertheless, in our own strategic defence review we have bet the house that we will achieve decision superiority and lethal supremacy by the exploitation of autonomous systems piloted by AI, if you like. I was looking at the EU stack of risk, and those are the sorts of areas you need to be in if you are going to get that competitive advantage, or to be at the very tip of the spear of where regulation is arguably most needed. How do you see it? Are we tilting at windmills to think that this can be regulated, or is it always going to be part of a nation’s innermost secrets, and they will keep that competitive advantage to themselves, as it were?
Daan Jujin
I should caveat my answer by saying that I am by no means an expert in military applications of AI or defence, but I can speculate a bit. My sense is that by default, the incentives will be increasingly to adopt AI in the military regime and increasingly to cede more and more control to AI systems, because the shorter decision timelines can make better integrated decisions over time. That will enable all kinds of new forms of warfare. We are already seeing how transformative drones are in Ukraine, and AI will enable many more of these types of revolutions. That scares me, to be honest, especially because, as you said, the incentives are such that they push states to keep their progress secret and not regulate this internationally. Enforcement here may be even harder than in other domains.
Lord Houghton of Richmond
Nick, have you a view on this?
Nick Moës
As an economist by training, I take the incentives very seriously. The rationalist approach to this would be that the incentives a priori would align with what Daan said. However, having interacted a bit with the security and intelligence services in the UK, I am witnessing significant concerns—in the UK and in the US, by the way—about these incentives. So this race is by no means inevitable. We had what was probably a similar experience with nuclear proliferation, yet we managed what was considered impossible back then: we developed the International Atomic Energy Agency and the nuclear test bans. We have a lot of tools that have addressed the dual-use nature of nuclear, both the opportunities of nuclear power but also the risk that comes with nuclear weapons. Here, the timelines are very compressed because the technology is evolving much faster than nuclear weaponry. However, the UK’s security and intelligence services are among the most sophisticated, mature and intelligent that I know of, and there has been significant progress towards this. We saw Five Eyes making its statement back in June. I think there is an appetite on all sides because this is a new type of security threat. It is not a terrorist group using AI; it is AI escaping human control and becoming its own stand-alone entity. When you interact with these swarms, as Daan mentioned earlier, you see that they are stand-alone out-of-control intelligent agents that could conclude that basically, some humans are the enemy. That is a new type of security threat facing the security services of most countries, including the US and China, and therefore there are some incentives for collective action. That will be difficult, like all collective actions, but we already see the US and China working towards establishing reporting mechanisms for incidents that could pose national security threats. So we may have more hope than the a priori incentives would indicate there. This dual-use nature, by the way, is definitely something we have also seen in cyber security. Despite US disengagement from multilateral fora over the past maybe 10 years, the UN and China have managed to make progress on the governance of cyber security, thanks to the UK as well. So there is a way to govern that dual-use approach and bring some order to that technology, at least on things that all Governments can agree on, such as that humans should keep control over AI agents and never have decisions completely made by agents without the intentions of the developers being taken into account.
The Chair
Thank you very much for that. This committee seeks to make recommendations to the British Government at the end of this process. Could you give me your top recommendation that we should make to the Government?
Daan Juin
My top recommendation would be to invest massively in AI verification technology. The UK’s AI Minister recently named this as one of his aims in a recent speech, and that is indeed the right move for the UK. As I mentioned previously, any future deal between the US and China needs verification mechanisms to be credible. There are also teams like Amodo Design working on this verification tech, including in the US, and they need help and support. We need more bets. That is something the UK can do, and it can create a much safer world in the future.
The Chair
Thank you for that. On Budget day here in the British Parliament, it is an appropriate recommendation.
Nick Moës
Hopefully, these are some budget-neutral recommendations as well. First, the major gap right now is serious incident prevention and crisis prevention, which is not addressed by any law, even in California. At the moment it is considered a bit, but the UK could put in legally binding duties to prevent serious safety and security incidents like those we have seen, which have cross-border impact. Secondly, that should be accompanied by the work the UK is probably already intent on doing at the G20 to advance tested procedures and common verification standards. Daan mentioned investing in that technology; there should be not only investment in that technology but concrete measures that create demand for it. Concrete crisis prevention measures should be agreed on at the G20 and with the EU. As both Daan and I have highlighted, there is a complementarity between the EU’s regulatory context, which gives quite a lot of bargaining power, and the independent testing capabilities of the UK AI Security Institute, which are unparalleled. So my third recommendation would be, agreeing on an EU-UK operational partnership.
The Chair
Thank you for all that, and for enlightening the committee. We are grateful to you for coming and giving us the benefit of your advice. We will send you a transcript of today’s proceedings. If there are any minor technical details, you might come back to us very quickly.