Smaller GGUF without the vision weights?
#19
by
rtzurtz
- opened
I wonder if it's possible to provide a smaller GGUF, without the vision weights, for those of us who don't need vision capabilities.
Just don't download mmproj files, that is where the vision encoder is injected. If you are not loading mmproj the memory usage is the same as a model without vision.