The Tiered Approach Explained


The tiered approach in the context of the EU AI Act is a regulatory approach where foundation models beyond a threshold are regulated with requirements that are more demanding than models that are below this threshold. The principles that inspired the tiered approach in AI are applied in many sectors of EU regulation: finance, social media, privacy etc. 


Number of providers of models for varying thresholds, Moes et al. (2023)


Answering to the "How?" requires to answer three questions:

Current risks arise from increasingly general and capable models. Hence, measuring risks among foundation models is mostly about measuring capabilities. There are currently five metrics that correlate well with the levels of capabilities of a system. All of those metrics are contained in this one graphic below summarizing how capabilities (each curve of color summarizes a particular set of capabilities that are evaluated) evolves as a function of computing power (in FLOP) , data, the size of the model, and the loss (how far is a model from perfect prediction) of a model. 

As you can see, capabilities and hence risks increase relatively predictably with the following four metrics: 

Beyond that, the science of measurement of capabilities (called evals) is moving forward and is allowing us to estimate increasingly precisely the capabilities of a system. This is a fifth criterion that should be used as we make it increasingly precise. 

Thresholds based on a combination of two of the five criteria are sufficiently precise to be hard to circumvent and well correlated with risks

2. What Thresholds? 

Everyone agrees that thresholds should be improved over time, as the science of measurement of capabilities and risks improves. We explain in 3) how to do so. But until that improves, we still need to manage risks. There are discussions about what temporary threshold makes the most sense for a tiered approach. 

What threshold in the short-run?

Most agree that for a tiered approach, the threshold implemented in the short-run should be between 10^23 and 10^26 FLOP. For reference, here are models, the compute they need to be trained, and the amounts of money at stake to acquire the necessary hardware: 

Some of the options discussed are: 

Signatory Professor Bengio supports a threshold of 10^26 FLOP on the basis that

Signatory SaferAI supports a threshold of 10^23 FLOP on the basis that models trained approximately with this amount of computing power like LLaMa-2 require risk management, because they

LLaMa-2, as available on the internet (Touvron et al., 2023)

3. How do we ensure that we can update thresholds and criteria?

Adapting thresholds or criteria without the need a new regulation to be changed is not uncommon in EU law. There are various ways to achieve that, one of the most commonly referred to being the use of delegated acts, allowing the European Commission to supplement or amend elements of a legislation within a limited scope. This could apply to the criteria and thresholds that have been defined for the tiered approach.