---
license: other
license_name: nvidia-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license
base_model: nvidia/Llama-4-Maverick-17B-128E-Eagle3
tags:
- speculative-decoding
- eagle3
- llama3
- llama4
- vllm
- speculators
---

# Llama-4-Maverick-17B-128E-Instruct-speculators.eagle3

## Model Overview
- **Verifier:** meta-llama/Llama-4-Maverick-17B-128E-Instruct
- **Speculative Decoding Algorithm:** EAGLE-3
- **Model Architecture:** Eagle3Speculator
- **Release Date:** 09/17/2025
- **Version:** 1.0
- **Model Developers:** RedHat

This is a speculator model designed for use with [meta-llama/Llama-4-Maverick-17B-128E-Instruct](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct), based on the [EAGLE-3](https://arxiv.org/abs/2503.01840) speculative decoding algorithm.
It was converted into the [speculators](https://github.com/neuralmagic/speculators) format from the model [nvidia/Llama-4-Maverick-17B-128E-Eagle3](https://huggingface.co/nvidia/Llama-4-Maverick-17B-128E-Eagle3).
This model should be used with the [meta-llama/Llama-4-Maverick-17B-128E-Instruct](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct) chat template, specifically through the `/chat/completions` endpoint.

## Use with vLLM

```bash
vllm serve meta-llama/Llama-4-Maverick-17B-128E-Instruct \
  -tp 8 \
  --speculative-config '{
    "model": "RedHatAI/Llama-4-Maverick-17B-128E-Instruct-speculators.eagle3",
    "num_speculative_tokens": 3,
    "method": "eagle3"
  }'
```

## Evaluations

<h3>Use cases</h3>
<table>
  <thead>
    <tr>
      <th>Use Case</th>
      <th>Dataset</th>
      <th>Number of Samples</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Coding</td>
      <td>HumanEval</td>
      <td>168</td>
    </tr>
    <tr>
      <td>Math Reasoning</td>
      <td>gsm8k</td>
      <td>80</td>
    </tr>
    <tr>
      <td>Text Summarization</td>
      <td>CNN/Daily Mail</td>
      <td>80</td>
    </tr>
  </tbody>
</table>

<h3>Acceptance lengths</h3>
<table>
  <thead>
    <tr>
      <th>Use Case</th>
      <th>k=1</th>
      <th>k=2</th>
      <th>k=3</th>
      <th>k=4</th>
      <th>k=5</th>
      <th>k=6</th>
      <th>k=7</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Coding</td>
      <td>1.83</td>
      <td>2.45</td>
      <td>2.94</td>
      <td>3.26</td>
      <td>3.47</td>
      <td>3.57</td>
      <td>3.62</td>
    </tr>
    <tr>
      <td>Math Reasoning</td>
      <td>1.86</td>
      <td>2.56</td>
      <td>3.08</td>
      <td>3.53</td>
      <td>3.73</td>
      <td>3.91</td>
      <td>4.02</td>
    </tr>
    <tr>
      <td>Text Summarization</td>
      <td>1.69</td>
      <td>2.12</td>
      <td>2.37</td>
      <td>2.52</td>
      <td>2.60</td>
      <td>2.63</td>
      <td>2.63</td>
    </tr>
  </tbody>
</table>

<h3>Performance benchmarking (2xA100)</h3>
<div style="display: flex; justify-content: center; gap: 20px;">

  <figure style="text-align: center;">
    <img src="assets/Llama-4-Maverick-17B-128E-Instruct-coding.png" alt="Coding" width="100%">
  </figure>

  <figure style="text-align: center;">
    <img src="assets/Llama-4-Maverick-17B-128E-Instruct-math_reasoning.png" alt="Math Reasoning" width="100%">
  </figure>

  <figure style="text-align: center;">
    <img src="assets/Llama-4-Maverick-17B-128E-Instruct-summarization.png" alt="Text Summarization" width="100%">
  </figure>
</div>

<details> <summary>Details</summary>
<strong>Configuration</strong>

- temperature: 0.6
- top_p: 0.9
- repetitions: 3
- time per experiment: 3min
- hardware: 8xB200
- vLLM version: 0.11.0
- GuideLLM version: 0.3.0

<strong>Command</strong>
```bash
GUIDELLM__PREFERRED_ROUTE="chat_completions" \
guidellm benchmark \
  --target "http://localhost:8000/v1" \
  --data "RedHatAI/speculator_benchmarks" \
  --data-args '{"data_files": "HumanEval.jsonl"}' \
  --rate-type sweep \
  --max-seconds 180 \
  --output-path "Llama-4-Maverick-HumanEval.json" \
  --backend-args '{"extra_body": {"chat_completions": {"temperature":0.6, "top_p":0.9}}}'
```
</details>


## Citation

If you use this model, please cite both the original NVIDIA model and the Speculators library:

```bibtex
@misc{nvidia2025llama4maverick,
  title={Llama 4 Maverick 17B Eagle3},
  author={NVIDIA Corporation},
  year={2025},
  publisher={Hugging Face}
}

@misc{speculators2024,
  title={Speculators: A Unified Library for Speculative Decoding},
  author={Neural Magic},
  year={2024},
  url={https://github.com/neuralmagic/speculators}
}
```

## Acknowledgments

- Original model by NVIDIA Corporation
- Conversion and formatting for Speculators/vLLM compatibility
- Based on Eagle3 architecture with Llama3 draft head targeting Llama4 verifier