Eval Request: GLM-4.5-Air-Derestricted
Hmm. I can't get this model to load with vllm. I get:
An error occurred during the testing of ArliAI/GLM-4.5-Air-Derestricted: 'dict' object has no attribute 'model_type'
no matter what I try.
GGUF user here, i don't know how to help, maybe the 8bit official versions might not have that problem?
FP8: https://huggingface.co/ArliAI/GLM-4.5-Air-Derestricted-FP8
INT8: https://huggingface.co/ArliAI/GLM-4.5-Air-Derestricted-W8A8-INT8
Idk , just wondering, maybe the full precision model has been uploaded with some missing meta tags..
Nice call! Yeah the FP8 version worked for me. Strange.
I thought you where using GGUF versions of models.
If you use vLLM, models not yet supported by llama.cpp, but supported by vLLM could be evaluated too!
Have you considered testing linear attention (or 3:1 delta Net) modules?
Like:
Not sure I'm able to test Nemotron-Elastic-12B since you need permission to download it, and it seems they are not accepting people's requests to do so. And no one else has uploaded like a gguf or anything.