nielsr HF Staff commited on
Commit
cd3334e
Β·
verified Β·
1 Parent(s): caa0744

Update model card with metadata, paper link and usage

Browse files

Hi! I'm Niels from the Hugging Face community science team.

This PR improves the model card for d3LLM-Dream by:
- Adding the `library_name: transformers` tag (verified by the presence of `auto_map` in `config.json`).
- Adding the `license: apache-2.0` tag.
- Adding the `base_model` identifier.
- Linking the model to its research paper: [d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation](https://huggingface.co/papers/2601.07568).
- Providing a standard `transformers` usage example.
- Adding the official citation from the paper.

These changes help improve the discoverability and usability of your model on the Hugging Face Hub.

Files changed (1) hide show
  1. README.md +42 -9
README.md CHANGED
@@ -1,30 +1,63 @@
1
  ---
2
  datasets:
3
  - d3LLM/trajectory_data_dream_32
 
 
 
 
4
  tags:
5
  - diffusion
6
  - text-generation
7
  - fast-inference
8
  - d3llm
9
- pipeline_tag: text-generation
10
  ---
11
 
12
-
13
  # d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation πŸš€
14
 
 
 
 
 
 
 
 
15
  ## Model Description
16
 
17
- **d3LLM-Dream** is an ultra-fast diffusion language model that achieves high generation speed while maintaining competitive performance. Built on the Dream architecture.
18
 
19
  ## Key Features
20
 
21
- - πŸš€ High throughput: **4.5Γ— faster** than autoregressive models (Qwen-2.5-7B) on H100 GPU, **2.5Γ— faster** on A100 GPU. Achieves **235.34 tokens/s** on H100 (vs 57.32 for AR baseline) on GSM8K-CoT Dataset.
22
- - πŸ“Š High AUP (Accuracy Under Parallelism) scores across benchmarks
23
- - πŸ”§ Optimized for coding and math reasoning tasks
24
 
25
  ## Usage
26
 
27
- For detailed usage instructions, evaluation scripts, training datasets, and training code, please refer to the official GitHub repository and our blog:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
- - πŸ‘‰ Code repo: **[https://github.com/hao-ai-lab/d3LLM](https://github.com/hao-ai-lab/d3LLM)**
30
- - 🌐 Blog: **[https://hao-ai-lab.github.io/blogs/text-diffusion/](https://hao-ai-lab.github.io/blogs/text-diffusion/)**
 
 
 
 
 
 
 
 
1
  ---
2
  datasets:
3
  - d3LLM/trajectory_data_dream_32
4
+ pipeline_tag: text-generation
5
+ library_name: transformers
6
+ license: apache-2.0
7
+ base_model: Dream-org/Dream-v0-Instruct-7B
8
  tags:
9
  - diffusion
10
  - text-generation
11
  - fast-inference
12
  - d3llm
 
13
  ---
14
 
 
15
  # d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation πŸš€
16
 
17
+ This repository contains the **d3LLM-Dream** model, an ultra-fast diffusion language model introduced in the paper [d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation](https://huggingface.co/papers/2601.07568).
18
+
19
+ - πŸ“„ **Paper**: [arXiv:2601.07568](https://huggingface.co/papers/2601.07568)
20
+ - πŸ‘‰ **Code repo**: [https://github.com/hao-ai-lab/d3LLM](https://github.com/hao-ai-lab/d3LLM)
21
+ - 🌐 **Blog**: [https://hao-ai-lab.github.io/blogs/text-diffusion/](https://hao-ai-lab.github.io/blogs/text-diffusion/)
22
+ - πŸ•ΉοΈ **Demo**: [https://d3llm-team.github.io/](https://d3llm-team.github.io/)
23
+
24
  ## Model Description
25
 
26
+ **d3LLM-Dream** is an ultra-fast diffusion language model that achieves high generation speed while maintaining competitive performance. It strikes a balance between accuracy and parallelism by using **pseudo-trajectory distillation** during training and **entropy-based multi-block decoding** during inference.
27
 
28
  ## Key Features
29
 
30
+ - πŸš€ **High throughput**: 4.5Γ— faster than autoregressive models (Qwen-2.5-7B) on H100 GPU, 2.5Γ— faster on A100 GPU. Achieves **235.34 tokens/s** on H100 on GSM8K-CoT.
31
+ - πŸ“Š **High AUP**: Optimized for Accuracy Under Parallelism across benchmarks.
32
+ - πŸ”§ **Specialized**: Optimized for coding and math reasoning tasks.
33
 
34
  ## Usage
35
 
36
+ You can load and use the model with the πŸ€— Transformers library. Note that `trust_remote_code=True` is required as the model uses a custom architecture.
37
+
38
+ ```python
39
+ from transformers import AutoModel, AutoTokenizer
40
+
41
+ model_id = "d3LLM/d3LLM_Dream"
42
+
43
+ # Load model and tokenizer
44
+ tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
45
+ model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
46
+
47
+ # For detailed inference scripts (multi-block decoding),
48
+ # please refer to the official GitHub repository.
49
+ ```
50
+
51
+ For more comprehensive examples and evaluation scripts, visit the [official repository](https://github.com/hao-ai-lab/d3LLM).
52
+
53
+ ## Citation
54
 
55
+ ```bibtex
56
+ @article{arxiv'26:d3llm,
57
+ title = {d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation},
58
+ author = {Yu-Yang Qian and Junda Su and Lanxiang Hu and Peiyuan Zhang and Zhijie Deng and Peng Zhao and Hao Zhang},
59
+ journal = {ArXiv preprint},
60
+ volume = {arXiv:2601.07568},
61
+ year = {2026}
62
+ }
63
+ ```