Models for local deployment - a yjernite Collection

yjernite 's Collections

Text embedding models I use

super-smol to fine-tune

Models for local deployment

Inference-supported production models

Document processing

Privacy

Models for local deployment

updated 4 days ago

Qwen/Qwen3-4B-Instruct-2507

Text Generation • 4B • Updated Sep 17 • 6.21M • • 528
unsloth/Qwen3-4B-Instruct-2507-GGUF

4B • Updated Aug 20 • 54.1k • 102
HuggingFaceTB/SmolLM3-3B

Text Generation • 3B • Updated Sep 10 • 90.6k • • 832
unsloth/SmolLM3-3B-GGUF

3B • Updated Jul 8 • 2.15k • 17
Qwen/Qwen3-Embedding-4B-GGUF

4B • Updated Aug 3 • 59.4k • 74

Note ./build/bin/llama-server -hf Qwen/Qwen3-Embedding-4B-GGUF:Q6_K --port 8080 --pooling last --verbose-prompt --parallel 4 --batch-size 8 --embedding --ctx-size 8000 --cache-type-k q4_0 --gpu-layers 36
Qwen/Qwen3-Embedding-0.6B

Feature Extraction • 0.6B • Updated Jun 20 • 4.69M • • 767
google/gemma-3-270m

Text Generation • 0.3B • Updated Aug 14 • 73.9k • 919
ByteDance/Ouro-1.4B

Text Generation • Updated 22 days ago • 10.8k • 56