Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -29,6 +29,8 @@ widget:
|
|
| 29 |
|
| 30 |
# LCM-LoRA SD1.5 - Checkpoint 1600
|
| 31 |
|
|
|
|
|
|
|
| 32 |
## Final Training - Mature
|
| 33 |
|
| 34 |
<div align="center">
|
|
@@ -60,76 +62,6 @@ This checkpoint represents training at **1600 steps** in our LCM-LoRA progressio
|
|
| 60 |
- π§ **Easy Integration**: Drop-in replacement using diffusers
|
| 61 |
- π **Proven Quality**: See comparison grid above
|
| 62 |
|
| 63 |
-
---
|
| 64 |
-
|
| 65 |
-
## π¦ All Checkpoints in Series
|
| 66 |
-
|
| 67 |
-
Click any checkpoint to view its specific characteristics and outputs:
|
| 68 |
-
|
| 69 |
-
### [Checkpoint 400 - Early Training - Foundation](https://huggingface.co/Mercity/lcm-lora-sd1.5-400)
|
| 70 |
-
|
| 71 |
-
**Characteristics:** Early training checkpoint showing foundational LCM capabilities. Provides decent quality with room for refinement.
|
| 72 |
-
|
| 73 |
-
**Best for:** Fast experimentation, understanding early LCM behavior
|
| 74 |
-
|
| 75 |
-
**Quality:** Good baseline quality, all prompts work correctly
|
| 76 |
-
|
| 77 |
-
<div align="center">
|
| 78 |
-
<img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-400/resolve/main/comparison_grid.png" alt="Checkpoint 400" width="100%">
|
| 79 |
-
</div>
|
| 80 |
-
|
| 81 |
-
[β View Full Details for Checkpoint 400](https://huggingface.co/Mercity/lcm-lora-sd1.5-400)
|
| 82 |
-
|
| 83 |
-
---
|
| 84 |
-
|
| 85 |
-
### [Checkpoint 800 - Mid Training - Vibrant Style](https://huggingface.co/Mercity/lcm-lora-sd1.5-800)
|
| 86 |
-
|
| 87 |
-
**Characteristics:** Mid-training checkpoint with vibrant, artistic outputs. Strong visual impact with saturated colors and expressive style.
|
| 88 |
-
|
| 89 |
-
**Best for:** Artistic applications, vibrant aesthetic, expressive style
|
| 90 |
-
|
| 91 |
-
**Quality:** High visual impact, strong artistic direction, vivid colors
|
| 92 |
-
|
| 93 |
-
<div align="center">
|
| 94 |
-
<img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-800/resolve/main/comparison_grid.png" alt="Checkpoint 800" width="100%">
|
| 95 |
-
</div>
|
| 96 |
-
|
| 97 |
-
[β View Full Details for Checkpoint 800](https://huggingface.co/Mercity/lcm-lora-sd1.5-800)
|
| 98 |
-
|
| 99 |
-
---
|
| 100 |
-
|
| 101 |
-
### [Checkpoint 1200 - High Training - Refined](https://huggingface.co/Mercity/lcm-lora-sd1.5-1200)
|
| 102 |
-
|
| 103 |
-
**Characteristics:** Higher training with more refined outputs. Some prompts may show signs of overfitting (e.g., occasional missing details).
|
| 104 |
-
|
| 105 |
-
**Best for:** Balanced colors, natural tones, specific use cases
|
| 106 |
-
|
| 107 |
-
**Quality:** High technical quality but may miss some prompt details
|
| 108 |
-
|
| 109 |
-
<div align="center">
|
| 110 |
-
<img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-1200/resolve/main/comparison_grid.png" alt="Checkpoint 1200" width="100%">
|
| 111 |
-
</div>
|
| 112 |
-
|
| 113 |
-
[β View Full Details for Checkpoint 1200](https://huggingface.co/Mercity/lcm-lora-sd1.5-1200)
|
| 114 |
-
|
| 115 |
-
---
|
| 116 |
-
|
| 117 |
-
### β¨ Checkpoint 1600 - Final Training - Mature (Current)
|
| 118 |
-
|
| 119 |
-
**Characteristics:** Final training checkpoint with mature, consistent outputs. Well-balanced and reliable across all prompts.
|
| 120 |
-
|
| 121 |
-
**Best for:** Most training, consistent results, production use
|
| 122 |
-
|
| 123 |
-
**Quality:** Excellent consistency, balanced outputs, reliable
|
| 124 |
-
|
| 125 |
-
<div align="center">
|
| 126 |
-
<img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-1600/resolve/main/comparison_grid.png" alt="Checkpoint 1600" width="100%">
|
| 127 |
-
</div>
|
| 128 |
-
|
| 129 |
-
---
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
---
|
| 134 |
|
| 135 |
## Checkpoint Comparison
|
|
@@ -158,79 +90,25 @@ Compare training progression and characteristics:
|
|
| 158 |
|
| 159 |
---
|
| 160 |
|
| 161 |
-
##
|
| 162 |
|
| 163 |
-
|
| 164 |
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
### π Underwater Scene
|
| 168 |
|
| 169 |
-
|
| 170 |
|
| 171 |
-
|
| 172 |
|
| 173 |
-
|
| 174 |
-
<img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-1600/resolve/main/validation/ood_underwater.png" alt="Underwater Scene" width="100%" style="max-width: 512px;">
|
| 175 |
-
<p><em>Image: ood_underwater.png</em></p>
|
| 176 |
-
</div>
|
| 177 |
|
| 178 |
-
|
| 179 |
|
| 180 |
-
|
| 181 |
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
**Tests:** Zero gravity, cosmic environment, space rendering (not in training)
|
| 185 |
-
|
| 186 |
-
<div align="center">
|
| 187 |
-
<img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-1600/resolve/main/validation/ood_space.png" alt="Space/Astronomy" width="100%" style="max-width: 512px;">
|
| 188 |
-
<p><em>Image: ood_space.png</em></p>
|
| 189 |
-
</div>
|
| 190 |
-
|
| 191 |
-
---
|
| 192 |
-
|
| 193 |
-
### π° Food Photography
|
| 194 |
-
|
| 195 |
-
**Prompt:** *"gourmet chocolate cake with berries on elegant plate, professional food photography, soft studio lighting"*
|
| 196 |
-
|
| 197 |
-
**Tests:** Food textures, studio lighting, product photography (not in training)
|
| 198 |
-
|
| 199 |
-
<div align="center">
|
| 200 |
-
<img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-1600/resolve/main/validation/ood_food.png" alt="Food Photography" width="100%" style="max-width: 512px;">
|
| 201 |
-
<p><em>Image: ood_food.png</em></p>
|
| 202 |
-
</div>
|
| 203 |
-
|
| 204 |
-
---
|
| 205 |
-
|
| 206 |
-
### π΄ Human Portrait
|
| 207 |
-
|
| 208 |
-
**Prompt:** *"close-up portrait of elderly man with weathered face and kind eyes, dramatic side lighting, black and white"*
|
| 209 |
-
|
| 210 |
-
**Tests:** Human facial features, skin texture, B&W conversion (training had cat portrait, not human closeup)
|
| 211 |
-
|
| 212 |
-
<div align="center">
|
| 213 |
-
<img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-1600/resolve/main/validation/ood_portrait.png" alt="Human Portrait" width="100%" style="max-width: 512px;">
|
| 214 |
-
<p><em>Image: ood_portrait.png</em></p>
|
| 215 |
-
</div>
|
| 216 |
-
|
| 217 |
-
---
|
| 218 |
-
|
| 219 |
-
### π¨ Abstract Art
|
| 220 |
-
|
| 221 |
-
**Prompt:** *"abstract watercolor painting with flowing colors, pink and blue gradient, artistic ethereal style"*
|
| 222 |
-
|
| 223 |
-
**Tests:** Non-representational art, color blending (training was all representational)
|
| 224 |
-
|
| 225 |
-
<div align="center">
|
| 226 |
-
<img src="https://huggingface.co/Mercity/lcm-lora-sd1.5-1600/resolve/main/validation/ood_abstract.png" alt="Abstract Art" width="100%" style="max-width: 512px;">
|
| 227 |
-
<p><em>Image: ood_abstract.png</em></p>
|
| 228 |
-
</div>
|
| 229 |
-
|
| 230 |
-
---
|
| 231 |
-
|
| 232 |
-
All validation images and prompts can be found in the `validation/` directory.
|
| 233 |
|
|
|
|
| 234 |
|
| 235 |
|
| 236 |
---
|
|
@@ -299,7 +177,7 @@ guidance_scale = 1.0 # Required for LCM
|
|
| 299 |
|
| 300 |
---
|
| 301 |
|
| 302 |
-
##
|
| 303 |
|
| 304 |
The comparison grid above shows outputs from this checkpoint at 2, 4, and 6 inference steps, compared to standard SD1.5 at 50 steps.
|
| 305 |
|
|
@@ -319,6 +197,34 @@ All sample images for this checkpoint are available in the `samples/` directory.
|
|
| 319 |
|
| 320 |
</details>
|
| 321 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 322 |
---
|
| 323 |
|
| 324 |
## Performance
|
|
@@ -341,44 +247,6 @@ All sample images for this checkpoint are available in the `samples/` directory.
|
|
| 341 |
|
| 342 |
---
|
| 343 |
|
| 344 |
-
## Advanced Usage
|
| 345 |
-
|
| 346 |
-
### Speed Optimization
|
| 347 |
-
|
| 348 |
-
```python
|
| 349 |
-
# Fuse LoRA for faster inference
|
| 350 |
-
pipe.fuse_lora(lora_scale=1.0)
|
| 351 |
-
|
| 352 |
-
# Use xformers for memory efficiency
|
| 353 |
-
pipe.enable_xformers_memory_efficient_attention()
|
| 354 |
-
|
| 355 |
-
# Compile model (PyTorch 2.0+)
|
| 356 |
-
pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True)
|
| 357 |
-
```
|
| 358 |
-
|
| 359 |
-
### Multiple LoRAs
|
| 360 |
-
|
| 361 |
-
```python
|
| 362 |
-
# Combine with other LoRAs
|
| 363 |
-
pipe.load_lora_weights("other_style.safetensors", adapter_name="style")
|
| 364 |
-
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-1600", adapter_name="lcm")
|
| 365 |
-
|
| 366 |
-
# Adjust weights
|
| 367 |
-
pipe.set_adapters(["style", "lcm"], adapter_weights=[0.8, 1.0])
|
| 368 |
-
```
|
| 369 |
-
|
| 370 |
-
### Switch Between Checkpoints
|
| 371 |
-
|
| 372 |
-
```python
|
| 373 |
-
# Load different checkpoints from this series
|
| 374 |
-
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-400")
|
| 375 |
-
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-800")
|
| 376 |
-
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-1200")
|
| 377 |
-
pipe.load_lora_weights("Mercity/lcm-lora-sd1.5-1600")
|
| 378 |
-
```
|
| 379 |
-
|
| 380 |
-
---
|
| 381 |
-
|
| 382 |
## Series Information
|
| 383 |
|
| 384 |
### Training Progression
|
|
|
|
| 29 |
|
| 30 |
# LCM-LoRA SD1.5 - Checkpoint 1600
|
| 31 |
|
| 32 |
+
**Author:** Juhi Singh | [HuggingFace](https://huggingface.co/juhirats)
|
| 33 |
+
|
| 34 |
## Final Training - Mature
|
| 35 |
|
| 36 |
<div align="center">
|
|
|
|
| 62 |
- π§ **Easy Integration**: Drop-in replacement using diffusers
|
| 63 |
- π **Proven Quality**: See comparison grid above
|
| 64 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
---
|
| 66 |
|
| 67 |
## Checkpoint Comparison
|
|
|
|
| 90 |
|
| 91 |
---
|
| 92 |
|
| 93 |
+
## Visual Comparison Across All Checkpoints
|
| 94 |
|
| 95 |
+
See how outputs evolve across the training series. Each grid shows: Baseline SD1.5 (50 steps) vs LCM-LoRA at 2, 4, and 6 steps.
|
| 96 |
|
| 97 |
+
### [Checkpoint 400](https://huggingface.co/Mercity/lcm-lora-sd1.5-400)
|
|
|
|
|
|
|
| 98 |
|
| 99 |
+

|
| 100 |
|
| 101 |
+
### [Checkpoint 800](https://huggingface.co/Mercity/lcm-lora-sd1.5-800)
|
| 102 |
|
| 103 |
+

|
|
|
|
|
|
|
|
|
|
| 104 |
|
| 105 |
+
### [Checkpoint 1200](https://huggingface.co/Mercity/lcm-lora-sd1.5-1200)
|
| 106 |
|
| 107 |
+

|
| 108 |
|
| 109 |
+
### Checkpoint 1600 (This Checkpoint)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 110 |
|
| 111 |
+

|
| 112 |
|
| 113 |
|
| 114 |
---
|
|
|
|
| 177 |
|
| 178 |
---
|
| 179 |
|
| 180 |
+
## Sample Outputs
|
| 181 |
|
| 182 |
The comparison grid above shows outputs from this checkpoint at 2, 4, and 6 inference steps, compared to standard SD1.5 at 50 steps.
|
| 183 |
|
|
|
|
| 197 |
|
| 198 |
</details>
|
| 199 |
|
| 200 |
+
### Out-of-Distribution (OOD) Validation Images
|
| 201 |
+
|
| 202 |
+
To test generalization beyond the training distribution, we generated images for 5 OOD prompts that are deliberately different from training prompts:
|
| 203 |
+
|
| 204 |
+
1. **π Underwater Scene**
|
| 205 |
+
- *"underwater coral reef with colorful fish and sea anemones, crystal clear water, natural sunlight filtering through"*
|
| 206 |
+
- Tests: Water effects, marine life, underwater lighting (not in training)
|
| 207 |
+
|
| 208 |
+
2. **π Space/Astronomy**
|
| 209 |
+
- *"astronaut floating in space with earth in background, stars and galaxies, cinematic lighting, 4k"*
|
| 210 |
+
- Tests: Zero gravity, cosmic environment, space rendering (not in training)
|
| 211 |
+
|
| 212 |
+
3. **π° Food Photography**
|
| 213 |
+
- *"gourmet chocolate cake with berries on elegant plate, professional food photography, soft studio lighting"*
|
| 214 |
+
- Tests: Food textures, studio lighting, product photography (not in training)
|
| 215 |
+
|
| 216 |
+
4. **π΄ Human Portrait**
|
| 217 |
+
- *"close-up portrait of elderly man with weathered face and kind eyes, dramatic side lighting, black and white"*
|
| 218 |
+
- Tests: Human facial features, skin texture, B&W conversion (training had cat portrait, not human closeup)
|
| 219 |
+
|
| 220 |
+
5. **π¨ Abstract Art**
|
| 221 |
+
- *"abstract watercolor painting with flowing colors, pink and blue gradient, artistic ethereal style"*
|
| 222 |
+
- Tests: Non-representational art, color blending (training was all representational)
|
| 223 |
+
|
| 224 |
+
**Why OOD Validation?** These prompts test whether the model truly learned general concepts rather than just memorizing training prompts. Good OOD performance indicates robust generalization.
|
| 225 |
+
|
| 226 |
+
All validation images can be found in the `validation/` directory. See [`validation/prompts.txt`](validation/prompts.txt) for the complete list of prompts used.
|
| 227 |
+
|
| 228 |
---
|
| 229 |
|
| 230 |
## Performance
|
|
|
|
| 247 |
|
| 248 |
---
|
| 249 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 250 |
## Series Information
|
| 251 |
|
| 252 |
### Training Progression
|