huihui-ai

/

Huihui-Qwen3-VL-4B-Thinking-abliterated

like 19

llama.cpp or one of the forks needs to first support it. and I just checked, and literal minutes ago: https://github.com/Thireus/llama.cpp/releases it's maybe still buggy, but that should be it.

I will test with tr-qwen3-vl-6-b7106-495c611 to see if it can satisfy the requirements.

I got it to compile after a few issues. it produced a completely broken model with all output garbled. this seems half cooked. his normal builds compile fine. and I never had broken models. if you get it running, it'd be great. but this does not look good. -- by the way, is this a not yet refusal loaded model? https://huggingface.co/inclusionAI/LLaDA2.0-flash-preview

https://huggingface.co/huihui-ai/Huihui-Qwen3-VL-4B-Thinking-abliterated/tree/main/GGUF

thank you!!! I am a noob, so no wonder I didn't get it to work...