llmcompressor settings
#1
by
hf-100
- opened
Hi, thank you for providing this quant: I was wondering how it was created using llmcompressor
So far I've only been able to generate text-only quants with llmcompressor which wouldn't load on vllm nightly, but it looks like this one does work correctly due to including the vision functionality in the resulting model
Have you comped against the quant.dynamic yet?