Create README.md

README.md

@@ -0,0 +1,37 @@
+# VIRES model card
+
+**Model Page**: [VIRES](https://hjzheng.net/projects/VIRES/)
+
+## Model Information
+
+Summary description and brief definition of inputs and outputs.
+
+### Description
+
+VIRES is a video instance repainting method with sketch and text guidance, enabling video instance repainting, replacement, generation, and removal. Existing approaches struggle with temporal consistency and accurate alignment with the provided sketch sequence. VIRES leverages the generative priors of text-to-video models to maintain temporal consistency and produce visually pleasing results. We propose the Sequential ControlNet with the standardized self-scaling, which effectively extracts structure layouts and adaptively captures high-contrast sketch details. We further augment the diffusion transformer backbone with the sketch attention to interpret and inject fine-grained sketch semantics. A sketch-aware encoder ensures that repainted results are aligned with the provided sketch sequence.
+
+### Inputs and outputs
+
+-   **Input:**
+    -  Text string
+    -  Mask Sequence, normalized to 51 x 512 x 512 resolution
+    -  Sketch Sequence, normalized to 51 x 512 x 512 resolution
+
+-   **Output:**
+    -   A repainting result video
+
+### Usage
+
+Ref to our GitHub page: https://github.com/suimuc/VIRES/
+
+## Citation
+
+```BibTeX
+@article{vires,
+      title={VIRES: Video Instance Repainting via Sketch and Text Guided Generation},
+      author={Weng, Shuchen and Zheng, Haojie and Zhang, Peixuan and Hong, Yuchen and Jiang, Han and Li, Si and Shi, Boxin},
+      journal={arXiv preprint arXiv:2411.16199},
+      year={2024}
+}
+```
+