The AI Act and the EU's own use of digital technologies: the example of ETIAS
-
Nathan Genicot
Author
What does the EU’s use of digital technologies in the field of external border management tell us about its regulatory approach to AI?
EU and Artificial Intelligence: a two-way relationship
For several years, the EU has been insisting on the need to develop a European framework to both promote and regulate AI. The publication of the Commission’s Proposal on Artificial Intelligence (the AI Act) –through which the EU has made clear its ambition to become the world leader in the regulation of AI and digital technologies– constituted a landmark moment of this strategy. One year after its publication, all eyes are on this text and its future evolution, while little attention is paid to how the European institutions themselves use AI (and digital technologies). But these uses may precisely be of some interest as they highlight some paradoxical aspects of the EU’s attitude to AI. One such application is the European Travel Information and Authorisation System (ETIAS), an algorithmic system that will profile non-EU citizens travelling in Europe.
What is the “trustworthy approach” to AI?
To begin with, let’s go back to this European approach to AI that the Commission started to build in the 2018 Communication AI for Europe, and continued in another Communication, Building Trust in Human-Centric AI, and in the White Paper on AI. In essence, the Commission’s position is that the development of AI is a huge step forward that will lead to major advances, but that it also presents dangers and threats that we may not neglect. For this reason, the EU needs to develop a regulatory framework guaranteeing that AI applications are trustworthy and human-centric: “AI is not an end in itself, but a tool that has to serve people with the ultimate aim of increasing human well-being” says the Commission. In this sense, the Commission identified 7 key requirements that AI applications should necessarily respect (such as human agency and oversight, diversity, non-discrimination and fairness, etc.).
The AI Act proposal is thus understood as the concretisation of the approach laid out in these previous publications. Its basic principle is to impose a set of constraints and legal guarantees on AI systems, which are all the more demanding as these systems are at risk. Hence, so-called high-risk AI systems –such as those used in criminal law, employment, credit assessment– will have to comply with a series of requirements and an ex-ante conformity assessment. Among the list of these high-risk systems, we also find AI application used for migration, asylum and border control purposes. The fact that this area has been included in the list is not insignificant as the EU itself has been particularly prone to using invasive technologies to control its own external borders.
The European digital surveillance and profiling of third-country nationals
In the context of globalisation and the resulting increased mobility, digital technologies were quickly perceived worldwide at the end of the 20th century as a useful tool for sorting the ‘good’ travellers (tourists and business people) from the ‘bad’ (poor migrants). The EU, which became competent in the late 1990s for the control of external borders, asylum and immigration (following the establishment of the area of freedom, security and justice), is no exception. It has since developed a range of policies for border and migration management and has relied heavily on technological solutions to do so. In particular, numerous databases and information systems have been created, such as Eurodac (which collects data related to asylum seekers) or the Visa Information System (related to visa data). In this regard, experts note the continuous trend to collect more and more data and for increasingly diverse purposes.
Recently, the EU decided to create new IT systems. One of them is called ETIAS and is supposed to be launched in 2023. ETIAS will screen travellers from visa-exempt countries wishing to make a short stay in the EU and will automatically assess the risk they pose in terms of security, health and illegal immigration. If the traveller is considered to be non-risky, he or she will automatically receive a travel authorisation. On the contrary, if the traveller is considered to be risky, his or her case will be referred to the national authority of the destination country.
But how will this assessment be done in practice? The ETIAS algorithmic system will lie on two main logics: first, it will be checked whether the applicant’s data is already stored in other databases (such as Eurodac, VIS, etc.), if so the applicant will be deemed at risk; second, a risk assessment will be carried out on the basis on various criteria that the applicant will have to provide (i.e. sex, age, nationality, country and city of residence, level of education, and current occupation).
These two operations are very different. The first one is all the more classical in the sense that it refers to the situation where a person has already been flagged in other EU databases and therefore needs to be closely monitored. The second is a profiling operation: a completely unknown traveller could be considered a potential illegal migrant or terrorist solely on the basis of its age, sex, level of education, etc. Why? Because previous data collected by the EU and Member States would indicate that a certain category of travellers are future terrorists, illegal migrants, etc. Presumably, ETIAS will single out the poorest and most vulnerable travellers. This is indeed often the case, as former UN Special Rapporteur Philip Alston has pointed out.
Of course, one could argue that as long as the algorithm does not make the final decision, there is no reason to worry. National authorities will have no obligation to withhold travel authorisation from travellers identified as risky, as the algorithm only acts as a first filter. But this may be putting too much faith in the agents who will issue short-stay visas considering, for example, the automation bias according to which humans tend to follow suggestions coming from automated decision-making systems.
How the EU is evading its own obligations
Thus, while the EU presents itself as a champion of human rights-friendly and trustworthy AI, its pervasive use of digital technologies for surveillance and sorting purposes may seem surprising, if not paradoxical. But at least - you might say - there is the forthcoming AI Act that will impose a minimum set of requirements and legal safeguards to ensure that systems like ETIAS are not completely dysfunctional or fed with biased data. A close look at the AI Act (in particular Article 83) reveals however that a specific exception is made for all EU IT systems dealing with border control and migration management, including ETIAS: the regulation will not apply to them. Interestingly, the European human-centric approach to AI seems thus to be of variable geometry, to say the least…
Recommended citations:
- This article is directly inspired from a scientific article I co-authored in which you will find a detailed study of ETIAS: Charly Derave, Nathan Genicot and Nina Hetmanska, “The risks of trustworthy AI: The case of ETIAS” (2022) European Journal of Risk Regulation forthcoming.
