Training Details on using several videos

#1
by rs3d - opened

Would you please share how you trained these AD MotionLoRAs with several videos (70 sounds impressive)?

The AD MotionDirector implementation for Comfy allows use of multiple image batches via MakeBatchList, but I could not get them to work in the training process.
The resulting LoRa with 1 batch or 1 + addition batches looks the same. I think there might be something missing that you potentially have fixed.

Thank you for all these interesting MotionLoRAs, hope you can give a bit of an insight.

Owner

HI! I gladly help. I attach a png of one of my training runs, you can get idea from here. You have to make an image batch list like you mentioned. It should work like this. Just drop the png into your comfyui it should contain the workflow. If you have a bigger dataset like I used 155 videos for the 32 frame lora, then this method could be time consuming. So when I did that, I wrote a custom node that basically do this "image batch list" from all videos in a folder. Unfortunately I don't have that node anymore but I think an LLM like Claude can help you making one. This example workflow contains the 'manual method'.
20240716_9213319_camera-lateral_left-10vid_mv2_0.00031_56_w768_h512_fr16_400_00001.png

Thank you very much for sharing the workflow, that's really helpful!

Owner

no problem and good luck!

Sign up or log in to comment