What are my options to run it on multiple GPUs?
#32
by
BigDeeper
- opened
With different layers loaded into multiple GPUs' VRAM?
I have 4 of them with 12.2GB each.
Nice gpus :D you can set device map to balanced but note that it will load the model on all of the GPUS so you dont have 4x the VRAM