Clémentine
commited on
Commit
·
dc8d285
1
Parent(s):
81747d0
rephrase
Browse files
app/src/content/chapters/troubleshooting/troubleshooting-inference.mdx
CHANGED
|
@@ -9,14 +9,12 @@ import Sidenote from "../../../components/Sidenote.astro";
|
|
| 9 |
|
| 10 |
### My results are very bad
|
| 11 |
|
| 12 |
-
The first thing to do is always to inspect your model generations in detail.
|
| 13 |
-
- too strict model output parsing (before computing the metric) which leads to the answer being lost
|
| 14 |
-
- Fixing: adapt your parsing
|
| 15 |
-
- unability of the models to follow your output format in few shot (frequent in recent models trained with instructions data, like llama 3.2 or Qwen 2.5)
|
| 16 |
-
- Fixing: either adapt your prompt format, or just assume that models should be able to follow it in few shot
|
| 17 |
-
- exceedingly verbose model which never gets to the correct answer (more frequent in long context models and something we observed with Qwen and CommandR models)
|
| 18 |
-
- Fixing: either increase the allowed context length, add instructions to be concise in the task prompt, or just assume that models should be able to answer succinctly
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
### My model is very slow!
|
| 22 |
➡️ Changing the batch size
|
|
|
|
| 9 |
|
| 10 |
### My results are very bad
|
| 11 |
|
| 12 |
+
The first thing to do is always to inspect your model generations in detail.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
+
Some frequent problems you should look for when troubleshooting are:
|
| 15 |
+
- Is your model output parsing too strict before computing the metric? It can lead to the answer being lost (obvious fix is to make it less strict, but you'll get more false positives!)
|
| 16 |
+
- Is your model struggling to follow your output format in few shot? This frequently happens in recent models trained on too specific evaluation formats, and you can either adapt your prompt format, or just state that models should be able to follow it and that the ones struggling are not good enough for the task you are considering.
|
| 17 |
+
- Is your model exceedingly verbose? In this case, it likely never gets to the correct answer - this is more frequent in long context models (we observed it with Qwen and Command R models in 2024) and reasoning models, especially if the tasks stops generation too soon. You can either increase the allowed context length, add instructions to be concise in the task prompt, or just assume that models should be able to answer succinctly.
|
| 18 |
|
| 19 |
### My model is very slow!
|
| 20 |
➡️ Changing the batch size
|