File size: 26,497 Bytes
8ecc6fa
 
e934b8f
8ecc6fa
 
 
 
 
 
 
 
727ebbb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5500263
727ebbb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5500263
727ebbb
5500263
 
727ebbb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8ecc6fa
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
---
license: mit
title: Yuuki-api
sdk: docker
emoji: πŸš€
colorFrom: purple
colorTo: pink
pinned: true
thumbnail: >-
  https://cdn-uploads.huggingface.co/production/uploads/68a8bd1d45ff88ffe886e331/64rKm_tmEj0DPMdzTAWCM.png
---
<div align="center">

<br>

<img src="https://img.shields.io/badge/%E2%9C%A6-YUUKI--API-000000?style=for-the-badge&labelColor=000000" alt="Yuuki API" height="50">

<br><br>

# Local Inference API for Yuuki Models

**FastAPI server. Docker deployment. Multi-model support. Zero external dependencies.**<br>
**Run Yuuki models directly on CPU with lazy loading and automatic caching.**

<br>

<a href="#features"><img src="https://img.shields.io/badge/FEATURES-000000?style=for-the-badge" alt="Features"></a>
&nbsp;&nbsp;
<a href="https://huggingface.co/spaces/OpceanAI/Yuuki-api"><img src="https://img.shields.io/badge/LIVE_API-000000?style=for-the-badge" alt="Live API"></a>
&nbsp;&nbsp;
<a href="https://github.com/sponsors/aguitauwu"><img src="https://img.shields.io/badge/SPONSOR-000000?style=for-the-badge" alt="Sponsor"></a>

<br><br>

[![License](https://img.shields.io/badge/MIT-222222?style=flat-square&logo=opensourceinitiative&logoColor=white)](LICENSE)
&nbsp;
[![FastAPI](https://img.shields.io/badge/FastAPI-222222?style=flat-square&logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com/)
&nbsp;
[![Docker](https://img.shields.io/badge/Docker-222222?style=flat-square&logo=docker&logoColor=white)](https://www.docker.com/)
&nbsp;
[![PyTorch](https://img.shields.io/badge/PyTorch-222222?style=flat-square&logo=pytorch&logoColor=white)](https://pytorch.org/)
&nbsp;
[![Transformers](https://img.shields.io/badge/Transformers-222222?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/docs/transformers/)
&nbsp;
[![HuggingFace](https://img.shields.io/badge/Spaces-222222?style=flat-square&logo=huggingface&logoColor=white)](https://huggingface.co/spaces)

<br>

---

<br>

<table>
<tr>
<td width="50%" valign="top">

**Self-hosted inference server.**<br><br>
Three Yuuki model variants.<br>
Lazy loading with memory caching.<br>
REST API with OpenAPI docs.<br>
Health check and model list endpoints.<br>
CORS enabled for web clients.<br>
Automatic model downloads at build time.<br>
CPU-optimized with ~50 tokens/second.

</td>
<td width="50%" valign="top">

**Production-ready deployment.**<br><br>
Dockerized for HuggingFace Spaces.<br>
Health checks with auto-restart.<br>
Request/response timing metrics.<br>
Configurable token limits.<br>
Temperature and top-p sampling.<br>
<br>
No API keys. No rate limits. Just inference.

</td>
</tr>
</table>

<br>

</div>

---

<br>

<div align="center">

## What is Yuuki-API?

</div>

<br>

**Yuuki-API** is a self-hosted inference server for the [Yuuki language models](https://huggingface.co/YuuKi-OS). It provides a FastAPI-based REST API that loads models on-demand, caches them in memory, and serves predictions via simple HTTP endpoints. Unlike cloud APIs, this runs entirely locally -- no API keys, no rate limits, no external dependencies.

The server supports three Yuuki model variants: **Yuuki-best** (flagship checkpoint), **Yuuki-3.7** (balanced), and **Yuuki-v0.1** (lightweight). Models are lazy-loaded on first use and cached for subsequent requests. All inference runs on CPU with PyTorch, optimized for resource-constrained environments like HuggingFace Spaces Free tier.

Built with **FastAPI**, **PyTorch**, **Transformers**, and packaged in a **Docker** container. Pre-downloads model weights during the build step to minimize startup time. Interactive API documentation available at `/docs`.

<br>

---

<br>

<div align="center">

## Features

</div>

<br>

<table>
<tr>
<td width="50%" valign="top">

<h3>Multi-Model Support</h3>

Three Yuuki variants: `yuuki-best`, `yuuki-3.7`, and `yuuki-v0.1`. Each model maps to its HuggingFace checkpoint. Clients specify the model via the `model` field in POST requests. Default is `yuuki-best` if not specified.

<br>

<h3>Lazy Loading & Caching</h3>

Models are loaded into memory only when first requested, not at server startup. Once loaded, they remain cached for the lifetime of the process. This allows the server to start instantly while supporting multiple models without consuming memory upfront.

<br>

<h3>REST API with Docs</h3>

Standard REST endpoints: `GET /` for API info, `GET /health` for status, `GET /models` for available models, and `POST /generate` for inference. FastAPI automatically generates interactive OpenAPI documentation at `/docs` and JSON schema at `/openapi.json`.

<br>

<h3>CORS Enabled</h3>

Configured with permissive CORS headers to allow requests from any origin. Essential for browser-based clients like [Yuuki Chat](https://github.com/YuuKi-OS/Yuuki-chat) or web demos.

</td>
<td width="50%" valign="top">

<h3>Request Validation</h3>

Pydantic models validate all inputs: `prompt` (1-4000 chars), `max_new_tokens` (1-512), `temperature` (0.1-2.0), and `top_p` (0.0-1.0). Invalid requests return structured error messages with HTTP 400/422 status codes.

<br>

<h3>Response Timing</h3>

Every `/generate` response includes a `time_ms` field showing inference latency in milliseconds. Useful for performance monitoring and client-side UX (e.g., showing "Generated in 2.1s").

<br>

<h3>Dockerized Deployment</h3>

Multi-stage Dockerfile that pre-downloads all three model checkpoints during the build step. This means the container starts with models already cached, eliminating cold-start delays. Optimized for HuggingFace Spaces but works anywhere Docker runs.

<br>

<h3>Health Checks</h3>

Built-in `/health` endpoint returns server status and lists which models are currently loaded in memory. Docker health check configured to auto-restart on failures.

</td>
</tr>
</table>

<br>

---

<br>

<div align="center">

## API Reference

</div>

<br>

### `GET /`

Returns API metadata and available endpoints.

```bash
curl https://opceanai-yuuki-api.hf.space/
```

**Response:**
```json
{
  "message": "Yuuki Local Inference API",
  "models": ["yuuki-best", "yuuki-3.7", "yuuki-v0.1"],
  "endpoints": {
    "health": "GET /health",
    "models": "GET /models",
    "generate": "POST /generate",
    "docs": "GET /docs"
  }
}
```

<br>

### `GET /health`

Health check endpoint showing server status and loaded models.

```bash
curl https://opceanai-yuuki-api.hf.space/health
```

**Response:**
```json
{
  "status": "ok",
  "available_models": ["yuuki-best", "yuuki-3.7", "yuuki-v0.1"],
  "loaded_models": ["yuuki-best"]
}
```

<br>

### `GET /models`

Lists all available models with their HuggingFace identifiers.

```bash
curl https://opceanai-yuuki-api.hf.space/models
```

**Response:**
```json
{
  "models": [
    {"id": "yuuki-best", "name": "OpceanAI/Yuuki-best"},
    {"id": "yuuki-3.7", "name": "OpceanAI/Yuuki-3.7"},
    {"id": "yuuki-v0.1", "name": "OpceanAI/Yuuki-v0.1"}
  ]
}
```

<br>

### `POST /generate`

Generate text completion from a prompt.

```bash
curl -X POST https://opceanai-yuuki-api.hf.space/generate \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "def fibonacci(n):",
    "model": "yuuki-best",
    "max_new_tokens": 100,
    "temperature": 0.7,
    "top_p": 0.95
  }'
```

**Request Body:**

| Field | Type | Required | Default | Range | Description |
|:------|:-----|:---------|:--------|:------|:------------|
| `prompt` | string | **Yes** | - | 1-4000 chars | Input text to complete |
| `model` | string | No | `yuuki-best` | - | Model ID to use |
| `max_new_tokens` | integer | No | 120 | 1-512 | Maximum tokens to generate |
| `temperature` | float | No | 0.7 | 0.1-2.0 | Sampling temperature |
| `top_p` | float | No | 0.95 | 0.0-1.0 | Nucleus sampling threshold |

**Response:**

```json
{
  "response": "    if n <= 1:\n        return n\n    return fibonacci(n-1) + fibonacci(n-2)",
  "model": "yuuki-best",
  "tokens_generated": 25,
  "time_ms": 2033
}
```

| Field | Type | Description |
|:------|:-----|:------------|
| `response` | string | Generated text (excluding the original prompt) |
| `model` | string | Model ID that was used |
| `tokens_generated` | integer | Number of new tokens produced |
| `time_ms` | integer | Inference time in milliseconds |

<br>

**Error Responses:**

```json
// Invalid model
{"detail": "Invalid model. Available: ['yuuki-best', 'yuuki-3.7', 'yuuki-v0.1']"}

// Token limit exceeded
{"detail": [{"type": "less_than_equal", "loc": ["body", "max_new_tokens"], 
"msg": "Input should be less than or equal to 512", "input": 1024}]}

// Server error
{"detail": "Model inference failed: Out of memory"}
```

<br>

---

<br>

<div align="center">

## Models

</div>

<br>

| Model ID | HuggingFace Path | Parameters | Description | Speed (CPU) |
|:---------|:-----------------|:-----------|:------------|:------------|
| `yuuki-best` | `OpceanAI/Yuuki-best` | 124M | Flagship checkpoint with best quality. Trained to step 2000. | ~50 tok/s |
| `yuuki-3.7` | `OpceanAI/Yuuki-3.7` | 124M | Balanced checkpoint for speed and quality. | ~50 tok/s |
| `yuuki-v0.1` | `OpceanAI/Yuuki-v0.1` | 124M | Lightweight first-generation model. Fastest inference. | ~55 tok/s |

All models are based on GPT-2 architecture (124M parameters) and trained on CPU (Snapdragon 685) with zero cloud budget. Model weights are ~500MB each. The server caches loaded models in RAM (~1.5GB total if all three are loaded).

<br>

---

<br>

<div align="center">

## Tech Stack

</div>

<br>

| Technology | Version | Purpose |
|:-----------|:--------|:--------|
| **FastAPI** | 0.115.0 | Web framework, request validation, auto-docs |
| **Uvicorn** | 0.30.6 | ASGI server for running FastAPI |
| **PyTorch** | 2.4.1 | Deep learning framework for model inference |
| **Transformers** | 4.45.0 | HuggingFace library for loading and running LLMs |
| **Pydantic** | 2.9.0 | Request/response validation |
| **Accelerate** | 0.34.2 | Model loading optimizations |

<br>

### System Requirements

| Resource | Minimum | Recommended |
|:---------|:--------|:------------|
| CPU | 2 cores | 4+ cores |
| RAM | 2GB | 4GB (8GB if loading all models) |
| Storage | 2GB | 3GB |
| Python | 3.10+ | 3.10+ |

<br>

---

<br>

<div align="center">

## Architecture

</div>

<br>

```
                           Client (Browser/CLI)
                                    |
                                    | HTTP POST /generate
                                    v
         +-------------------------------------------------------+
         |           Yuuki-API (FastAPI + Uvicorn)              |
         |                                                       |
         |   /generate endpoint                                 |
         |         |                                            |
         |         v                                            |
         |   load_model(model_key)                              |
         |         |                                            |
         |         v                                            |
         |   +-----------------+                                |
         |   | Cache Check     | <-- loaded_models dict         |
         |   +-----------------+                                |
         |         |                                            |
         |    Model cached?                                     |
         |    /           \                                     |
         |  YES            NO                                   |
         |   |              |                                   |
         |   |              v                                   |
         |   |      AutoModelForCausalLM.from_pretrained()      |
         |   |      AutoTokenizer.from_pretrained()             |
         |   |              |                                   |
         |   |              v                                   |
         |   |      Store in loaded_models cache                |
         |   |              |                                   |
         |   +<-------------+                                   |
         |         |                                            |
         |         v                                            |
         |   tokenizer.encode(prompt)                           |
         |         |                                            |
         |         v                                            |
         |   model.generate()                                   |
         |         |                                            |
         |         v                                            |
         |   tokenizer.decode(output)                           |
         |         |                                            |
         |         v                                            |
         |   {"response": "...", "tokens_generated": N,         |
         |    "time_ms": T, "model": "yuuki-best"}              |
         +----------------------+--------------------------------+
                                |
                                v
                        JSON Response to Client
```

<br>

### Request Flow

1. **Client sends POST** to `/generate` with `prompt`, `model`, and parameters
2. **FastAPI validates** request body via Pydantic models
3. **load_model()** checks if model is cached in memory
4. **If not cached:** Downloads from HuggingFace, loads with PyTorch, stores in cache
5. **If cached:** Retrieves from `loaded_models` dict
6. **Tokenizer encodes** prompt to token IDs
7. **Model generates** continuation with specified parameters
8. **Tokenizer decodes** new tokens to text
9. **Response returned** with generated text, token count, and timing

<br>

---

<br>

<div align="center">

## Installation

</div>

<br>

### Local Development

```bash
# Clone repository
git clone https://github.com/YuuKi-OS/Yuuki-api
cd Yuuki-api

# Create virtual environment
python3.10 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Run server
uvicorn app:app --host 0.0.0.0 --port 7860
```

Server will start at `http://localhost:7860`. Visit `http://localhost:7860/docs` for interactive API documentation.

<br>

### Docker

```bash
# Build image
docker build -t yuuki-api .

# Run container
docker run -p 7860:7860 yuuki-api
```

**Note:** The Docker build step downloads all three models (~1.5GB total) which takes 5-10 minutes on first build. Subsequent builds use Docker layer caching and are much faster.

<br>

---

<br>

<div align="center">

## Deploy to HuggingFace Spaces

</div>

<br>

The recommended deployment method for zero-cost hosting.

### Steps

1. **Create a new Space** at [huggingface.co/new-space](https://huggingface.co/new-space)
2. **Choose SDK:** Docker
3. **Upload files:**
   - `README.md` (with YAML header)
   - `Dockerfile`
   - `app.py`
   - `requirements.txt`
4. **Wait for build** (~10 minutes for model downloads)
5. **Access API** at `https://YOUR-USERNAME-SPACE-NAME.hf.space`

<br>

### README.md Header

```yaml
---
title: Yuuki API
emoji: πŸ€–
colorFrom: purple
colorTo: black
sdk: docker
pinned: false
---
```

<br>

### Environment Variables

None required. The API has zero external dependencies -- no API keys, no database, no auth services.

<br>

---

<br>

<div align="center">

## Usage Examples

</div>

<br>

### Python

```python
import requests

response = requests.post(
    "https://opceanai-yuuki-api.hf.space/generate",
    json={
        "prompt": "def hello_world():",
        "model": "yuuki-best",
        "max_new_tokens": 50,
        "temperature": 0.7
    }
)

data = response.json()
print(data["response"])
print(f"Generated {data['tokens_generated']} tokens in {data['time_ms']}ms")
```

<br>

### JavaScript / TypeScript

```typescript
const response = await fetch('https://opceanai-yuuki-api.hf.space/generate', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    prompt: 'def hello_world():',
    model: 'yuuki-best',
    max_new_tokens: 50,
    temperature: 0.7
  })
});

const data = await response.json();
console.log(data.response);
console.log(`Generated ${data.tokens_generated} tokens in ${data.time_ms}ms`);
```

<br>

### cURL

```bash
curl -X POST https://opceanai-yuuki-api.hf.space/generate \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "def hello_world():",
    "model": "yuuki-best",
    "max_new_tokens": 50,
    "temperature": 0.7
  }'
```

<br>

### Next.js API Route

```typescript
// app/api/generate/route.ts
import { NextRequest, NextResponse } from 'next/server';

export async function POST(req: NextRequest) {
  const { prompt, model = 'yuuki-best', max_new_tokens = 100 } = await req.json();

  const response = await fetch('https://opceanai-yuuki-api.hf.space/generate', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt, model, max_new_tokens })
  });

  const data = await response.json();
  return NextResponse.json(data);
}
```

<br>

---

<br>

<div align="center">

## Performance

</div>

<br>

### Inference Speed (CPU)

| Tokens | Yuuki Best | Yuuki 3.7 | Yuuki v0.1 |
|:-------|:-----------|:----------|:-----------|
| 50 | ~1.0s | ~1.0s | ~0.9s |
| 100 | ~2.0s | ~2.0s | ~1.8s |
| 250 | ~5.0s | ~4.8s | ~4.5s |
| 512 (max) | ~10.2s | ~10.0s | ~9.3s |

Benchmarked on HuggingFace Spaces Free tier (2-core CPU). Times are for first request after model load. Subsequent requests are ~10% faster due to PyTorch optimizations.

<br>

### Memory Usage

| State | RAM Usage |
|:------|:----------|
| Server idle (no models loaded) | ~250MB |
| + 1 model loaded | ~750MB |
| + 2 models loaded | ~1.2GB |
| + 3 models loaded | ~1.7GB |

HuggingFace Spaces Free tier provides 16GB RAM, so all three models can be loaded simultaneously with plenty of headroom.

<br>

### Cold Start Time

| Operation | Duration |
|:----------|:---------|
| Server startup (no models) | <1s |
| First request (model download + load) | 8-12s |
| Subsequent requests (cached) | <100ms overhead |

Docker build pre-downloads models, so cold start on HuggingFace Spaces is instant.

<br>

---

<br>

<div align="center">

## Configuration

</div>

<br>

### Model Limits

Adjust `max_new_tokens` limit in `app.py`:

```python
class GenerateRequest(BaseModel):
    prompt: str = Field(..., min_length=1, max_length=4000)
    max_new_tokens: int = Field(default=120, ge=1, le=512)  # Change 512 to your limit
    temperature: float = Field(default=0.7, ge=0.1, le=2.0)
    top_p: float = Field(default=0.95, ge=0.0, le=1.0)
```

Higher limits increase memory usage and inference time. 512 tokens (~2KB text) balances quality and speed on CPU.

<br>

### Adding More Models

Add new models to the `MODELS` dict in `app.py`:

```python
MODELS = {
    "yuuki-best": "OpceanAI/Yuuki-best",
    "yuuki-3.7": "OpceanAI/Yuuki-3.7",
    "yuuki-v0.1": "OpceanAI/Yuuki-v0.1",
    "my-model": "username/my-model-checkpoint",  # Add here
}
```

<br>

### CORS Configuration

Modify CORS settings in `app.py`:

```python
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # Change to specific domains: ["https://myapp.com"]
    allow_methods=["*"],
    allow_headers=["*"],
)
```

<br>

---

<br>

<div align="center">

## Troubleshooting

</div>

<br>

### Server returns 500 error

**Check logs for:**
- `Out of memory` β†’ Model too large for available RAM. Try `yuuki-v0.1` or reduce `max_new_tokens`.
- `Connection timeout` β†’ Model loading takes >30s. This is normal on first load.

<br>

### Models not loading

**Verify:**
- HuggingFace Transformers is installed: `pip show transformers`
- Model IDs are correct in `MODELS` dict
- Internet connection available for model downloads
- `~/.cache/huggingface/` has write permissions

<br>

### Slow inference

**Optimizations:**
- Use `yuuki-v0.1` instead of `yuuki-best` for 10-15% speedup
- Reduce `max_new_tokens` to minimum needed
- Lower `temperature` to 0.3-0.5 for faster sampling
- Ensure no other processes are using CPU

<br>

### Docker build fails

**Common issues:**
- Out of disk space β†’ Model downloads need 2GB+ free
- Network timeout β†’ Retry build, HuggingFace servers may be busy
- Python version mismatch β†’ Use Python 3.10 base image

<br>

---

<br>

<div align="center">

## Roadmap

</div>

<br>

### v1.0 -- Current (Complete)

- [x] Three Yuuki model variants
- [x] Lazy loading with memory caching
- [x] FastAPI with OpenAPI docs
- [x] Docker deployment
- [x] Health check endpoint
- [x] CORS enabled
- [x] Request validation
- [x] Response timing metrics

### v1.1 -- Enhancements (Planned)

- [ ] Streaming responses (Server-Sent Events)
- [ ] Token usage statistics endpoint
- [ ] Model warm-up on server start
- [ ] Request queuing for concurrent requests
- [ ] Prometheus metrics export
- [ ] Rate limiting per IP

### v2.0 -- Advanced Features (Future)

- [ ] GPU support with CUDA
- [ ] Batch inference
- [ ] Model quantization (4-bit/8-bit)
- [ ] Multi-turn conversation context
- [ ] Fine-tuning API
- [ ] WebSocket support

<br>

---

<br>

<div align="center">

## Contributing

</div>

<br>

### Development Setup

```bash
git clone https://github.com/YuuKi-OS/Yuuki-api
cd Yuuki-api

python3.10 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Run with hot reload
uvicorn app:app --reload --host 0.0.0.0 --port 7860
```

<br>

### Commit Convention

```
<type>(<scope>): <subject>
```

Types: `feat` | `fix` | `docs` | `perf` | `refactor` | `chore`

```
feat(api): add streaming response support

- Implement SSE endpoint at /generate/stream
- Add async generator for token-by-token streaming
- Update docs with streaming examples

Closes #12
```

<br>

### Pull Request Checklist

- [ ] Code follows PEP 8 style guidelines
- [ ] All endpoints tested with valid/invalid inputs
- [ ] No breaking changes to existing API
- [ ] Documentation updated (README + docstrings)
- [ ] Dockerfile builds successfully
- [ ] Commits follow the convention above

<br>

---

<br>

<div align="center">

## About the Yuuki Project

</div>

<br>

Yuuki-API is part of the [Yuuki project](https://huggingface.co/OpceanAI/Yuuki-best) -- a code-generation LLM being trained entirely on a smartphone (Redmi 12, Snapdragon 685, CPU only) with zero cloud budget.

<table>
<tr>
<td width="50%" valign="top">

**Training Details**

| | |
|:--|:--|
| Base model | GPT-2 (124M parameters) |
| Training type | Continued pre-training |
| Hardware | Snapdragon 685, CPU only |
| Training time | 50+ hours |
| Progress | 2,000 / 37,500 steps (5.3%) |
| Cost | $0.00 |

</td>
<td width="50%" valign="top">

**Quality Scores (Checkpoint 2000)**

| Language | Score |
|:---------|:------|
| Agda | 55 / 100 |
| C | 20 / 100 |
| Assembly | 15 / 100 |
| Python | 8 / 100 |

</td>
</tr>
</table>

Created by **agua_omg** -- a young independent developer who started the project in January 2026 because paying for Claude was no longer an option. The name Yuuki combines the Japanese word for snow (Yuki) with the character Yuu from Girls' Last Tour.

<br>

---

<br>

<div align="center">

## Related Projects

</div>

<br>

| Project | Description |
|:--------|:------------|
| [Yuuki Chat](https://github.com/YuuKi-OS/Yuuki-chat) | macOS-inspired chat interface with web research and YouTube search |
| [Yuuki Web](https://github.com/YuuKi-OS/yuuki-web) | Official landing page for the Yuuki project |
| [yuy](https://github.com/YuuKi-OS/yuy) | CLI for downloading, managing, and running Yuuki models |
| [yuy-chat](https://github.com/YuuKi-OS/yuy-chat) | TUI chat interface for local AI conversations |
| [Yuuki-best](https://huggingface.co/OpceanAI/Yuuki-best) | Best checkpoint model weights |
| [Yuuki Space](https://huggingface.co/spaces/OpceanAI/Yuuki) | Web-based interactive demo |
| [yuuki-training](https://github.com/YuuKi-OS/yuuki-training) | Training code and scripts |

<br>

---

<br>

<div align="center">

## Links

</div>

<br>

<div align="center">

[![Live API](https://img.shields.io/badge/Live_API-HuggingFace_Spaces-ffd21e?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/spaces/OpceanAI/Yuuki-api)
&nbsp;
[![Model Weights](https://img.shields.io/badge/Model_Weights-Hugging_Face-ffd21e?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/OpceanAI/Yuuki-best)
&nbsp;
[![Yuuki Chat](https://img.shields.io/badge/Yuuki_Chat-Vercel-000000?style=for-the-badge&logo=vercel&logoColor=white)](https://yuuki-chat.vercel.app)

<br>

[![YUY CLI](https://img.shields.io/badge/Yuy_CLI-GitHub-181717?style=for-the-badge&logo=github&logoColor=white)](https://github.com/YuuKi-OS/yuy)
&nbsp;
[![YUY Chat](https://img.shields.io/badge/Yuy_Chat-GitHub-181717?style=for-the-badge&logo=github&logoColor=white)](https://github.com/YuuKi-OS/yuy-chat)
&nbsp;
[![Sponsor](https://img.shields.io/badge/Sponsor-GitHub_Sponsors-ea4aaa?style=for-the-badge&logo=githubsponsors&logoColor=white)](https://github.com/sponsors/aguitauwu)

</div>

<br>

---

<br>

<div align="center">

## License

</div>

<br>

```
MIT License

Copyright (c) 2026 Yuuki Project

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
```

<br>

---

<br>

<div align="center">

**Built with patience, a phone, and zero budget.**

<br>

[![Yuuki Project](https://img.shields.io/badge/Yuuki_Project-2026-000000?style=for-the-badge)](https://huggingface.co/OpceanAI)

<br>

</div>