README.md

---
base_model:
- VanishD/Agentic-R1
language:
- en
license: mit
pipeline_tag: text-generation
library_name: transformers
tags:
- qwen2
- reasoning
- tool-use
- llm
---

# Agentic-R1: Distilled Dual-Strategy Reasoning

The model was presented in the paper [Agentic-R1: Distilled Dual-Strategy Reasoning](https://huggingface.co/papers/2507.05707).

Code: https://github.com/StigLidu/DualDistill

## Abstract

Current long chain-of-thought (long-CoT) models excel at mathematical reasoning but rely on slow and error-prone natural language traces. Tool-augmented agents address arithmetic via code execution, but often falter on complex logical tasks. We introduce a fine-tuning framework, DualDistill, that distills complementary reasoning strategies from multiple teachers into a unified student model. Using this approach, we train Agentic-R1, which dynamically selects the optimal strategy for each query, invoking tools for arithmetic and algorithmic problems, and using text-based reasoning for abstract ones. Our method improves accuracy across a range of tasks, including both computation-intensive and standard benchmarks, demonstrating the effectiveness of multi-strategy distillation in achieving robust and efficient reasoning.

## Key Features

-   **Efficient Training**: Integrates tool use into long-chain-of-thought (CoT) reasoning using only 4 × A6000 GPUs
-   **Unified Reasoning**: Fuses heterogeneous reasoning traces from multiple teacher models into a single student model

<div align="center">
  <img src="https://raw.githubusercontent.com/StigLidu/DualDistill/main/fig/overview.png" alt="Overview of DualDistill methodology" width="500">
  <p><em>Overview of DualDistill methodology</em></p>
</div>

## Datasets

| Dataset | Description | Link |
|---------|-------------|------|
| **Training Set** | Complete training dataset with teacher trajectories | [🤗 HuggingFace](https://huggingface.co/datasets/VanishD/DualDistill) |
| **Test Set** | Evaluation benchmarks | `dataset/test/` |

## Results

<div align="center">
  <img src="https://raw.githubusercontent.com/StigLidu/DualDistill/main/fig/result.png" alt="Performance comparison of Agentic-R1 models" width="700">
</div>

-   **Agentic-R1** demonstrates significant performance gains on **DeepMath-L** and **Combinatorics300**, where both complex reasoning and tool use are crucial for success.
-   **Agentic-R1-SD** (Self-Distilled) further enhances performance through our self-distillation approach, consistently outperforming baseline models across nearly all evaluation tasks.

## Quick Start

### Installation

1.  **Clone the repository**:
    ```bash
    git clone https://github.com/StigLidu/DualDistill.git
    cd DualDistill
    ```

2.  **Create environment** (optional but recommended):
    ```bash
    conda create -n dualdistill python=3.11
    conda activate dualdistill
    ```

3.  **Install dependencies**:
    ```bash
    pip install -r requirements.txt
    pip install flash-attn --no-build-isolation
    ```

### Inference Server and Evaluation

To run inference and evaluation using the provided scripts:

1.  **Start inference server**:
    ```bash
    bash script/eval_script/start_inference_server.sh [model_path] [display_name] [port]
    ```

2.  **Run Evaluation**:
    ```bash
    bash script/eval_script/eval_remote_server.sh \
      [url] [display_name] [data_path] [code_mode] [max_token]
    ```

    **Example**:
    ```bash
    bash script/eval_script/eval_remote_server.sh \
      "http://localhost:8080/v1" "agentic-r1" "dataset/test/math.json" "true" "4096"
    ```

## Trained Models

| Model | Description | HuggingFace Link |
|-------|-------------|------------------|
| **Agentic-R1-7B** | Base model with teacher distillation | [🤗 Download](https://huggingface.co/VanishD/Agentic-R1) |
| **Agentic-R1-7B-SD** | Enhanced model with self-distillation | [🤗 Download](https://huggingface.co/VanishD/Agentic-R1-SD) |

## ⚠️ Important Notes

-   **Code Execution Safety**: The evaluation scripts execute model-generated code locally. Only use trusted models before execution.
-   **Inference Config**: If you are using vLLM (a recent version) and encounter an error regarding the maximum context length. You may need to modify the `model_max_length` in `tokenizer_config.json`.
-   **Self-Distillation Warning**: The self-distillation step requires sampling many trajectories and can be time-consuming.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## Acknowledgments

We thank the following open-source projects for their foundational contributions:

-   [OpenHands](https://github.com/All-Hands-AI/OpenHands) - Agent framework
-   [DeepMath-103K](https://huggingface.co/datasets/zwhe99/DeepMath-103K) - Mathematical reasoning dataset
-   [vLLM](https://github.com/vllm-project/vllm) - High-performance inference engine

## Contact

For questions or support, please contact:

-   **Weihua Du**: [weihuad@cs.cmu.edu](mailto:weihuad@cs.cmu.edu)

## Citation

If you find our work useful, please consider citing:

```bibtex
@article{du2025agentic,
  title={Agentic-R1: Distilled Dual-Strategy Reasoning},
  author={Du, Weihua and Aggarwal, Pranjal and Welleck, Sean and Yang, Yiming},
  journal={arXiv preprint arXiv:2507.05707},
  year={2025}
}
```

---

<div align="center">
  <p>⭐ Star us on GitHub if this project helped you!</p>
</div>