|
|
--- |
|
|
language: |
|
|
- zh |
|
|
- en |
|
|
- ja |
|
|
- ko |
|
|
- es |
|
|
license: cc-by-nc-4.0 |
|
|
pipeline_tag: automatic-speech-recognition |
|
|
library_name: transformers |
|
|
tags: |
|
|
- music |
|
|
- art |
|
|
--- |
|
|
|
|
|
# HeartTranscriptor-oss |
|
|
|
|
|
HeartTranscriptor is a robust lyric recognition model optimized for real-world music scenarios, part of the **HeartMuLa** family of Music Foundation Models. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
HeartTranscriptor is a Whisper-based model specifically fine-tuned for lyric transcription across multiple languages. It is designed to handle the complexities of real-world music, where vocals are often mixed with rich instrumental backgrounds. It supports languages including but not limited to English, Chinese, Japanese, Korean, and Spanish. |
|
|
|
|
|
- **Developed by:** Ario Scale Global |
|
|
- **Model Type:** Whisper-based lyric transcription |
|
|
- **Language(s):** English, Chinese, Japanese, Korean, Spanish |
|
|
- **License:** [CC-BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
- **Github Repo:** https://github.com/HeartMuLa/heartlib |
|
|
- **Paper:** https://arxiv.org/abs/2601.10547 |
|
|
- **Demo:** https://heartmula.github.io/ |
|
|
- **Related Models:** |
|
|
- [HeartMuLa-oss-3B](https://huggingface.co/HeartMuLa/HeartMuLa-oss-3B) |
|
|
- [HeartCodec-oss](https://huggingface.co/HeartMuLa/HeartCodec-oss) |
|
|
|
|
|
## Get Started |
|
|
|
|
|
Check the GitHub repository [HeartMuLa/heartlib](https://github.com/HeartMuLa/heartlib) for a quickstart guide and local deployment instructions. Usage examples for HeartTranscriptor can be found in the `examples` directory of the repository. |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you find HeartMuLa or HeartTranscriptor useful, please cite: |
|
|
|
|
|
```bibtex |
|
|
@misc{yang2026heartmulafamilyopensourced, |
|
|
title={HeartMuLa: A Family of Open Sourced Music Foundation Models}, |
|
|
author={Dongchao Yang and Yuxin Xie and Yuguo Yin and Zheyu Wang and Xiaoyu Yi and Gongxi Zhu and Xiaolong Weng and Zihan Xiong and Yingzhe Ma and Dading Cong and Jingliang Liu and Zihang Huang and Jinghan Ru and Rongjie Huang and Haoran Wan and Peixu Wang and Kuoxi Yu and Helin Wang and Liming Liang and Xianwei Zhuang and Yuanyuan Wang and Haohan Guo and Junjie Cao and Zeqian Ju and Songxiang Liu and Yuewen Cao and Heming Weng and Yuexian Zou}, |
|
|
year={2026}, |
|
|
eprint={2601.10547}, |
|
|
archivePrefix={arXiv}, |
|
|
primaryClass={cs.SD}, |
|
|
url={https://arxiv.org/abs/2601.10547}, |
|
|
} |
|
|
``` |
|
|
|
|
|
## Contact |
|
|
If you are interested in HeartMuLa, feel free to reach us at heartmula.ai@gmail.com |
