How good is it?
Since this is 2bit, was wondering if you tested and if it's stable for long context (eg. 64K or more). Trying to fit in 96GB Ultra and there don't seem to be good models in mlx.
For long context, no. And honestly would not recommend this as it thinks for a long time, outputs something, and seemingly starts thinking again and goes into loops.
GLM 4.6 gguf from unsloth actually worked better and fits into 96GB but at a smaller context, still waiting for GLM 4.6 Air.
Also check out mradermacher/MiniMax-M2-THRIFT-i1-GGUF i1-IQ3_M, hope it helps.
GLM 4.6 gguf from unsloth actually worked better and fits into 96GB but at a smaller context, still waiting for GLM 4.6 Air.
GGUF seems slower than MLX and trying to stick to MLX if possible. But I'll check out MiniMax-M2 from mradermacher. Agree GLM 4.6 Air would be good middle ground for quality and speed once it's released.
Have you tried any other REAP models?