No, the EU is not regulating AI by outrage

Last week, I covered the recently discussed amendments to the EU AI Act which concern AI chatbots. According to some commentators, the amendments being considered in the European Parliament allegedly proposed to put generative AI tools, such as ChatGPT, wholesale into the high-risk basket.

If true, that would mean that the developers of such tools would be in all cases subject to stringent requirements, including the need to implement a risk management system and pass system certification — safeguards usually reserved for evidently high-stakes AI systems, such as those used by the judiciary and law enforcement for life-altering decision-making.

Having seen the discussion around that topic, I, as many others, became concerned, as the alleged proposals to consider AI chatbots, regardless of concrete use cases, as always high-risk, might be viewed as disproportional to the limited risk potential of these systems.

As some other colleagues, I have made some comments to that end, adding a disclaimer that I would really wish to see the actual legislative text.

Prominent coverage

Primarily, two sources have received prominence. The one that attracted most worried comments on LinkedIn, including mine, was the opinion piece by the Center for Data Innovation contributor Patrick Grady, “ChatGPT Amendment Shows the EU is Regulating by Outrage”.

The essence of the article is captured in its first paragraph (emphasis mine):

The EU is considering placing generative artificial intelligence (AI) tools, such as ChatGPT, in a “high risk” category in its upcoming AI bill, thereby subjecting such tools to burdensome compliance requirements. This sloppy addition needlessly stunts creativity and shows the EU is hitting the panic button instead of carefully considering the benefits and risks of new technologies.

It further mentions Grammarly, GitHub Copilot and some other similar AI tools as allegedly falling into high-risk category under the proposed amendments.

Another, earlier coverage was presented in a report by the EURACTIV journalist Luca Bertuzzi, “AI Act: EU Parliament’s crunch time on high-risk categorisation, prohibited practices” (emphasis mine):

A residual category was introduced to cover generative AI systems like ChatGPT. Any AI-generated text that might be mistaken for human-generated is considered at risk unless it undergoes human review and a person or organisation is legally liable for it.

Lacking access to the actual wording of the amendments, I preferred to rely on Luca Bertuzzi’s coverage, as it seemed to me that the opinion piece by Patrick Brady was missing, and building on the absence of, an important distinction I’ve emphasised above.

My earlier comment

Seeing the coverage above, in my last article, I preferred to stay cautious and noted:

[…] the reporting […] suggests that AI systems that generate text are not proposed to be deemed high-risk unless they operate in a deceiving way (by not disclosing that the text is computer-generated, in contexts where this is not evident).

Further, in the same newsletter, I explained that, as per the proposal of the Council of the European Union (6 December 2022 version), all “purely accessory” use of AI systems is supposed to be exempt from the rules applicable to high-risk AI uses.

That is, an otherwise high-risk AI use will not be deemed as such if the AI system does not automatically cause decisions or other actions in the real world and instead allows human decision makers to rule on whether and how to modify, contextualise and use the system’s output.

This is obviously the case with ChatGPT, Grammarly, GitHub Copilot and many other text-generating AI systems, at least considering how they are currently designed to operate. They cannot be classified as high-risk if we follow the Council’s proposed wording.

And then I discovered the actual wording of the amendments

Following the publication of my previous article and some subsequent discussions, I decided again to try and identify what wording may have been discussed by MEPs. Courtesy of Google Search and the European Parliament website, I’ve managed to find the likely primary source, the published draft report on the batch of amendments 3020 — 3312 to the EU AI Act.

On pages 92-93 of the report, among others, there are amendments 3237 and 3238 by Dragoş Tudorache and colleagues and by Brando Benifei and colleagues, respectively. These amendments appear to be the ones most similar to the description given in Luca Bertuzzi’s EURACTIV report.

In the part which relates to the text-generating AI systems, the amendment 3237 proposes to expand the list of high-risk AI systems in Annex III as follows (emphasis mine):

8 a. Other applications:

(a) AI systems intended to be used to generate, on the basis of limited human input, complex text content that would falsely appear to a person to be human generated and authentic, such as news articles, opinion articles, novels, scripts, and scientific articles, with the exception of AI systems used exclusively for content that undergoes human review and for the publication of which a natural or legal person established in the Union is liable or holds editorial responsibility;

Amendment 3237 is largely similar, and repeats the important caveat that the content must falsely appear to be human generated, but a further exception part is formulated differently:

8 a. Other applications:

(a) AI systems intended to be used to generate, on the basis of limited human input, complex text content that would falsely appear to a person to be human generated and authentic, such as news articles, opinion articles, novels, scripts, and scientific articles, except where the content forms part of an evidently artistic, creative or fictional and analogous work;

As it should be obvious to everyone who actually tried using most popular AI text-generating tools such as ChatGPT, they do not publish anything automatically without human review. Rather, they present their outputs to the human prompter in the respective interface, such as, in the case of ChatGPT, a chat box. It is then up to the human prompter to review the output and potentially use it or not (for more complex scenarios involving API integrations, see my previous article).

Naturally, this also implies that to the human prompter, these outputs cannot falsely appear to be human generated, as he or she is interacting with an AI chatbot which is presented as such. Whether the human prompter then decides to misrepresent the content as human generated is another matter, but in terms of the concrete wording of these legislative proposals, ChatGPT, and of course Grammarly and the like, are obviously out of scope.

The latter is both true in the context of the Council’s proposal (due to the provision exempting “purely accessory” outputs) and the MEPs’ proposals.

In conclusion

As a result, I do not really see how Patrick Grady could have come to the conclusion which resulted in the headline: “ChatGPT Amendment Shows the EU is Regulating by Outrage”. Whatever sources he may have used to prepare his article, they were apparently incomplete.

This doesn’t mean that the proposals in amendments 3237 and 3238 are entirely unproblematic, though.

As currently envisaged, the structure of the EU AI Act is such that it gives the European Commission certain powers to supplement the list of high-risk AI systems. But these powers are limited in that any future extension of this list must be constrained to eight areas of AI use exhaustively specified in Annex III.

Currently, these areas are specific enough and comprise clearly high-stakes use cases, such as, for example, in critical infrastructure, essential services, law enforcement and administration of justice.

The proposals in amendments 3237 and 3238, however, as currently worded, are completely out of line with this approach, as they presuppose adding a ninth AI use area entitled “8a. Other applications”. The benefit of this approach is that it creates a catch-all AI use category, so if a completely new high-risk AI application may come to light, it may be quickly made subject to the rules relevant for other high-risk AI uses.

The downside is of course that it dramatically reduces legal certainty for AI developers and AI users and allows the European Commission to bypass the usual lawmaking process (involving the Parliament and the Council) on a matter of high importance both to the business community and to the society at large.

Hopefully, this issue will be addressed during further discussions between the European policymakers.

If you like my newsletter, please consider supporting me on Patreon. Thanks!