how is this a "lightweight 5B model" ? each part is 5gb and theres 10...
#8
by
realrebelai
- opened
im confused. 5B parameters should not be over 50gb of data. Z-Image-Base is an 8B parameter model with only 12gb of data.
fp64?
What is fp64? Lol
Float precision 64, mostly used in scientific simulation
The final ZIP contains the pretrain, SFT, RL models and the optimizers states to continue the training.
I extracted just the RL and it is about 6GB. You can also check out the collection to test the SFT model
Alex11556666
changed discussion status to
closed