comments on contamination + saturation

app/src/content/article.mdx

@@ -47,7 +47,21 @@ Now that you have an idea of why evaluation is important to different people, le
 
 ## Evaluating with existing benchmarks
 
-Now that you've gotten (re)acquainted with required basics on how tokenization and inference work, and what are the caveats when doing evalution, let's look at actual benchmarking! We'll first do a small tour of 2025 evaluations, then discuss what to look at in a benchamrk, and why you probably can't reproduce announcements scores. Lastly, we'll cover the special case of selecting good benchmark to evaluate training with the FineWeb team
+Now that you've gotten (re)acquainted with required basics on how tokenization and inference work, and what are the caveats when doing evalution, let's look at actual benchmarking! We'll first do a small tour of 2025 evaluations, then discuss what to look at in a benchamrk, and why you probably can't reproduce announcements scores. Lastly, we'll cover the special case of selecting good benchmark to evaluate training with the FineWeb team.
+
+<Note title="Important concepts"  emoji="⚠️" variant="info">
+In this section, you'll see two concepts mentionned quite a lot: contamination and saturation.
+
+**Saturation** is when model performance on a benchmark passes human performance. More generally, the term is used for datasets that are no longer considered useful, as they have lost discriminative power between models. 
+<Sidenote> It's what you observe in the banner picture! </Sidenote>
+
+*If all models have close to the highest possible score on your evaluation, it's no longer a discriminative benchmark. It's similar to evaluating high school students on pre-school problems: success tells you nothing (though failure is indicative).*
+
+**Contamination** is when an evaluation dataset ended up in the training dataset of models, in which case the performance of models is artificially inflated, and does not reflect real world performance on the task. 
+
+*It's a bit like evaluating a student on questions it already knows in advance.*
+
+</Note>
 
 ### Benchmarks to know in 2025
 

app/src/content/chapters/general-knowledge/2025-evaluations-for-useful-models.mdx

@@ -5,7 +5,11 @@ title: "2025 evaluations"
 import Note from "../../../components/Note.astro";
 import Sidenote from "../../../components/Sidenote.astro";
 
-You can evaluate **specific capabilities** on their own - it's usually quite interesting to get signal when training, or when comparing base/pretrained models. (However, if you select and validate your training methods with the following evaluations, reporting on them on the final model is slightly biased as you have already oriented your training method towards good results on them).
+You can evaluate **specific capabilities** on their own - it's usually quite interesting to get signal when training, or when comparing base/pretrained models. (However, if you select and validate your training methods with the following evaluations, reporting on them on the final model is slightly biased as you have already oriented your training method towards good results on them). 
+
+<Note>
+Feel free to skim this section if you're not very familiar with evaluation yet, and come back to it once you need to find a dataset for a specific capability :)
+</Note>
 
 #### Reasoning and commonsense
 
@@ -155,7 +159,9 @@ A similar approach is used to generate questions in [Arbitrage](https://arxiv.or
 
 In a similar vein, you'll also find arenas where LLMs are provided with money to actively trade on financial markets (like Alpha Arena or Trading Agents) - these experiments are less likely to give meaningful results, as, because of their costs, they tend to be run once per model only, so you get no statistical significance there.
 
-<Note title="TLDR" emoji="🎯">
+#### Recommendations
+
+<Note title="TLDR" emoji="🎯" variant="info">
 The landscape of evaluation has evolved with the jumps in capabilities, from testing isolated skills to measuring integrated performance in more realistic scenarios. 
 
 As of Nov 2025, I recommend using:
@@ -170,4 +176,8 @@ The field is moving toward evaluations that test capability orchestration rather
 <Sidenote>
 I hope the field moves towards putting more emphasis on functional testing rather than model judges, and generally understandable datasets and tasks. 
 </Sidenote>
-</Note>
+</Note>
+
+<Note>
+If you want to explore even more datasets, you'll find a big list of older interesting benchmarks [here](https://github.com/huggingface/evaluation-guidebook/blob/main/contents/automated-benchmarks/some-evaluation-datasets.md) with my notes.
+</Note>