Possible to run on six RTX Pro 6000 Blackwell with vLLM oder SGLang?
#2
by
FabianHeller
- opened
Is it possible to run this model with 6 RTX Pro 6000 Blackwell GPUs? I think generally it should work with a combination of --tensor-parallel-size 2 and --pipeline-parallel-size 3, but I am not sure.
would be interesting to see if it does
Not on the latest vllm:
NotImplementedError: Pipeline parallelism is not supported for this model. Supported models implement the SupportsPP interface.