New issue: naive REAM not supporting MTP?
#5
by
TomLucidor
- opened
Started collab with the following issue, seems like the REAM version stripped out the MTP function? In that case, the 2x speedup would be lost. https://github.com/waybarrios/vllm-mlx/pull/82#issuecomment-3903706729
Yes, indeed, MTP weights were ignored in REAM because we used hf transformers which doesn't load mtp weights. I will try to look into this next week.