Hugging Face's logo

noctrex
/
Huihui-Qwen3-VL-4B-Instruct-abliterated-i1-GGUF

Image-Text-to-Text
GGUF
imatrix
conversational
Model card Files
xet
Community

These are quantizations of the model Huihui-Qwen3-VL-4B-Instruct-abliterated.

These quantizations were created using an imatrix merged from combined_all_large and harmful.txt to leverage the abliterated nature of the model.

For higher precision versions (which are not in an need for an imatrix), please refer to the main repository: noctrex/Huihui-Qwen3-VL-4B-Instruct-abliterated-GGUF

Usage Notes:

  • Download the latest llama.cpp to use these quantizations.
  • Try to use the best quality you can run.
  • For the mmproj file, the F32 version is recommended for best results (F32 > BF16 > F16).
Downloads last month
754
GGUF
Model size
4B params
Architecture
qwen3vl
Hardware compatibility
Log In to view the estimation

3-bit

4-bit

5-bit

6-bit

View +1 variant
Inference Providers NEW
Image-Text-to-Text
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for noctrex/Huihui-Qwen3-VL-4B-Instruct-abliterated-i1-GGUF

Base model

Qwen/Qwen3-VL-4B-Instruct
Quantized
huihui-ai/Huihui-Qwen3-VL-4B-Instruct-abliterated
Quantized
(5)
this model

Collections including noctrex/Huihui-Qwen3-VL-4B-Instruct-abliterated-i1-GGUF