Update README.md
Browse files
README.md
CHANGED
|
@@ -31,7 +31,9 @@ Terminal Technology Department, Alipay, Ant Group.
|
|
| 31 |
* EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation. [GitHub](https://github.com/antgroup/echomimic_v2)
|
| 32 |
|
| 33 |
## 📣 Updates
|
| 34 |
-
* [
|
|
|
|
|
|
|
| 35 |
* [2024.11.22] π₯ [GradioUI](https://github.com/antgroup/echomimic_v2/blob/main/app.py) is now available. Thanks @gluttony-10 for the contribution.
|
| 36 |
* [2024.11.22] π₯ [ComfyUI](https://github.com/smthemex/ComfyUI_EchoMimic) is now available. Thanks @smthemex for the contribution.
|
| 37 |
* [2024.11.21] π₯ We release the EMTD dataset list and processing scripts.
|
|
@@ -132,14 +134,26 @@ Terminal Technology Department, Alipay, Ant Group.
|
|
| 132 |
</tr>
|
| 133 |
</table>
|
| 134 |
|
| 135 |
-
## βοΈ Installation
|
| 136 |
### Download the Codes
|
| 137 |
|
| 138 |
```bash
|
| 139 |
git clone https://github.com/antgroup/echomimic_v2
|
| 140 |
cd echomimic_v2
|
| 141 |
```
|
|
|
|
|
|
|
| 142 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
### Python Environment Setup
|
| 144 |
|
| 145 |
- Tested System Environment: Centos 7.2/Ubuntu 22.04, Cuda >= 11.7
|
|
@@ -206,6 +220,11 @@ Run the python inference script:
|
|
| 206 |
python infer.py --config='./configs/prompts/infer.yaml'
|
| 207 |
```
|
| 208 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 209 |
### EMTD Dataset
|
| 210 |
Download dataset:
|
| 211 |
```bash
|
|
@@ -219,6 +238,7 @@ Process dataset:
|
|
| 219 |
```bash
|
| 220 |
python ./EMTD_dataset/preprocess.py
|
| 221 |
```
|
|
|
|
| 222 |
|
| 223 |
## π Release Plans
|
| 224 |
|
|
@@ -228,7 +248,8 @@ python ./EMTD_dataset/preprocess.py
|
|
| 228 |
| β
| Pretrained models trained on English and Mandarin Chinese on HuggingFace | 21st Nov, 2024 |
|
| 229 |
| β
| Pretrained models trained on English and Mandarin Chinese on ModelScope | 21st Nov, 2024 |
|
| 230 |
| β
| EMTD dataset list and processing scripts | 21st Nov, 2024 |
|
| 231 |
-
|
|
|
|
|
| 232 |
| π | Online Demo on ModelScope to be released | TBD |
|
| 233 |
| π | Online Demo on HuggingFace to be released | TBD |
|
| 234 |
|
|
@@ -248,13 +269,12 @@ If we missed any open-source projects or related articles, we would like to comp
|
|
| 248 |
If you find our work useful for your research, please consider citing the paper :
|
| 249 |
|
| 250 |
```
|
| 251 |
-
@misc{
|
| 252 |
title={EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation},
|
| 253 |
author={Rang Meng, Xingyu Zhang, Yuming Li, Chenguang Ma},
|
| 254 |
year={2024},
|
| 255 |
eprint={2411.10061},
|
| 256 |
-
archivePrefix={arXiv}
|
| 257 |
-
primaryClass={cs.CV}
|
| 258 |
}
|
| 259 |
```
|
| 260 |
|
|
|
|
| 31 |
* EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation. [GitHub](https://github.com/antgroup/echomimic_v2)
|
| 32 |
|
| 33 |
## 📣 Updates
|
| 34 |
+
* [2025.01.03] ππ₯ **One Minute is All You Need to Generate Video**. [Accelerated EchoMimicV2](https://github.com/antgroup/echomimic_v2/blob/main/infer_acc.py) are released. The inference speed can be improved by 9x (from ~7mins/120frames to ~50s/120frames on A100 GPU).
|
| 35 |
+
* [2024.12.16] π₯ [RefImg-Pose Alignment Demo](https://github.com/antgroup/echomimic_v2/blob/main/demo.ipynb) is now available, which involves aligning reference image, extracting pose from driving video, and generating video.
|
| 36 |
+
* [2024.11.27] π₯ [Installation tutorial](https://www.youtube.com/watch?v=2ab6U1-nVTQ) is now available. Thanks [AiMotionStudio](https://www.youtube.com/@AiMotionStudio) for the contribution.
|
| 37 |
* [2024.11.22] π₯ [GradioUI](https://github.com/antgroup/echomimic_v2/blob/main/app.py) is now available. Thanks @gluttony-10 for the contribution.
|
| 38 |
* [2024.11.22] π₯ [ComfyUI](https://github.com/smthemex/ComfyUI_EchoMimic) is now available. Thanks @smthemex for the contribution.
|
| 39 |
* [2024.11.21] π₯ We release the EMTD dataset list and processing scripts.
|
|
|
|
| 134 |
</tr>
|
| 135 |
</table>
|
| 136 |
|
| 137 |
+
## βοΈ Automatic Installation
|
| 138 |
### Download the Codes
|
| 139 |
|
| 140 |
```bash
|
| 141 |
git clone https://github.com/antgroup/echomimic_v2
|
| 142 |
cd echomimic_v2
|
| 143 |
```
|
| 144 |
+
### Automatic Setup
|
| 145 |
+
- CUDA >= 11.7, Python == 3.10
|
| 146 |
|
| 147 |
+
```bash
|
| 148 |
+
sh linux_setup.sh
|
| 149 |
+
```
|
| 150 |
+
## βοΈ Manual Installation
|
| 151 |
+
### Download the Codes
|
| 152 |
+
|
| 153 |
+
```bash
|
| 154 |
+
git clone https://github.com/antgroup/echomimic_v2
|
| 155 |
+
cd echomimic_v2
|
| 156 |
+
```
|
| 157 |
### Python Environment Setup
|
| 158 |
|
| 159 |
- Tested System Environment: Centos 7.2/Ubuntu 22.04, Cuda >= 11.7
|
|
|
|
| 220 |
python infer.py --config='./configs/prompts/infer.yaml'
|
| 221 |
```
|
| 222 |
|
| 223 |
+
Run the python inference script for accelerated version. Make sure to check out the configuration for accelerated inference:
|
| 224 |
+
```bash
|
| 225 |
+
python infer_acc.py --config='./configs/prompts/infer_acc.yaml'
|
| 226 |
+
```
|
| 227 |
+
|
| 228 |
### EMTD Dataset
|
| 229 |
Download dataset:
|
| 230 |
```bash
|
|
|
|
| 238 |
```bash
|
| 239 |
python ./EMTD_dataset/preprocess.py
|
| 240 |
```
|
| 241 |
+
Make sure to check out the [discussions](https://github.com/antgroup/echomimic_v2/discussions) to learn how to start the inference.
|
| 242 |
|
| 243 |
## π Release Plans
|
| 244 |
|
|
|
|
| 248 |
| β
| Pretrained models trained on English and Mandarin Chinese on HuggingFace | 21st Nov, 2024 |
|
| 249 |
| β
| Pretrained models trained on English and Mandarin Chinese on ModelScope | 21st Nov, 2024 |
|
| 250 |
| β
| EMTD dataset list and processing scripts | 21st Nov, 2024 |
|
| 251 |
+
| β
| Jupyter demo with pose and reference image alignmnet | 16st Dec, 2024 |
|
| 252 |
+
| β
| Accelerated models | 3st Jan, 2025 |
|
| 253 |
| π | Online Demo on ModelScope to be released | TBD |
|
| 254 |
| π | Online Demo on HuggingFace to be released | TBD |
|
| 255 |
|
|
|
|
| 269 |
If you find our work useful for your research, please consider citing the paper :
|
| 270 |
|
| 271 |
```
|
| 272 |
+
@misc{meng2024echomimicv2,
|
| 273 |
title={EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation},
|
| 274 |
author={Rang Meng, Xingyu Zhang, Yuming Li, Chenguang Ma},
|
| 275 |
year={2024},
|
| 276 |
eprint={2411.10061},
|
| 277 |
+
archivePrefix={arXiv}
|
|
|
|
| 278 |
}
|
| 279 |
```
|
| 280 |
|