|
|
--- |
|
|
license: cc-by-nc-nd-4.0 |
|
|
language: |
|
|
- en |
|
|
pipeline_tag: feature-extraction |
|
|
library_name: timm |
|
|
datasets: |
|
|
- AI4Pathology/pathorchestra-image-features |
|
|
--- |
|
|
license: cc-by-nc-nd-4.0 |
|
|
language: |
|
|
- en |
|
|
pipeline_tag: feature-extraction |
|
|
library_name: timm |
|
|
--- |
|
|
|
|
|
# PathOrchestra_V1.0.0 – Foundation Model for Computational Pathology |
|
|
|
|
|
## 🔒 Access Policy |
|
|
|
|
|
Access to the pretrained weights of **PathOrchestra_V1.0.0** is restricted for **academic research purposes only**. |
|
|
Please ensure that your Hugging Face account is associated with an **official/institutional email** and request access accordingly. |
|
|
|
|
|
> **License:** CC BY-NC-ND 4.0 – non-commercial use only; modifications and redistribution are not permitted. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🧠 Model Overview |
|
|
|
|
|
**PathOrchestra** is a scalable vision foundation model for computational pathology, pretrained using self-supervised learning on a corpus of **300,000 whole-slide images (WSIs)** spanning **20 organs/tissue types** from multiple medical centers. |
|
|
|
|
|
The model was evaluated across **112 clinical-grade diagnostic tasks**, leveraging a combination of **61 private** and **51 public datasets**, demonstrating strong generalizability in multi-organ and multi-task settings. |
|
|
|
|
|
--- |
|
|
|
|
|
## 🔧 Usage: Load as a Vision Encoder |
|
|
|
|
|
To load the model via `timm` with pretrained weights from the Hugging Face Hub: |
|
|
|
|
|
```python |
|
|
import timm |
|
|
from huggingface_hub import login |
|
|
|
|
|
# Authenticate with your User Access Token (https://huggingface.co/settings/tokens) |
|
|
login(token=your_hf_token) |
|
|
|
|
|
model = timm.create_model( |
|
|
"hf-hub:AI4Pathology/PathOrchestra", |
|
|
pretrained=True, |
|
|
init_values=1e-5, |
|
|
dynamic_img_size=True, |
|
|
) |
|
|
|
|
|
model.eval() |
|
|
``` |
|
|
|
|
|
|
|
|
## 🧪 Feature Extraction Example |
|
|
|
|
|
```python |
|
|
import torch |
|
|
|
|
|
from PIL import Image |
|
|
from torchvision import transforms |
|
|
from huggingface_hub import hf_hub_download |
|
|
|
|
|
|
|
|
# Define preprocessing transform |
|
|
transform = transforms.Compose([ |
|
|
transforms.Resize(224), |
|
|
transforms.ToTensor(), |
|
|
transforms.Normalize(mean=(0.485, 0.456, 0.406), |
|
|
std=(0.229, 0.224, 0.225)) |
|
|
]) |
|
|
|
|
|
image_path = hf_hub_download(repo_id="AI4Pathology/PathOrchestra", filename="example.png") |
|
|
image = Image.open(image_path).convert("RGB") |
|
|
|
|
|
image = transform(image).unsqueeze(0) # Add batch dimension |
|
|
|
|
|
with torch.inference_mode(): |
|
|
features = model(image) # Extract patch-level embedding |
|
|
|
|
|
print(feature_emb.shape) |
|
|
``` |
|
|
--- |
|
|
|
|
|
## 📫 Contact |
|
|
For access requests, collaboration inquiries, or academic use cases, please contact the corresponding authors listed in the official repository. |
|
|
|
|
|
## 🙏 Acknowledgements |
|
|
We thank the authors of DINOv2 and UNI for foundational contributions to vision model development. |
|
|
|
|
|
## 📖 Citation |
|
|
If you use PathOrchestra in your research, please cite: |
|
|
|
|
|
```bibtex |
|
|
@article{yan2025pathorchestra, |
|
|
title={PathOrchestra: A comprehensive foundation model for computational pathology with over 100 diverse clinical-grade tasks}, |
|
|
author={Yan, Fang and Wu, Jianfeng and Li, Jiawen and Wang, Wei and Lu, Jiaxuan and Chen, Wen and Gao, Zizhao and Li, Jianan and Yan, Hong and Ma, Jiabo and others}, |
|
|
journal={arXiv preprint arXiv:2503.24345}, |
|
|
year={2025} |
|
|
} |
|
|
``` |
