HikariDawn commited on
Commit
b3980c8
·
1 Parent(s): 13a5cf7

docs: update app page

Browse files
Files changed (2) hide show
  1. .gitignore +1 -3
  2. app.py +25 -13
.gitignore CHANGED
@@ -28,6 +28,4 @@ preprocess/sam2_code
28
  !preprocess/oneformer_code/oneformer/data/bpe_simple_vocab_16e6.txt
29
  !config/*.json
30
  !requirements.txt
31
- !requirements/*
32
- !__assets__/*
33
- !__assets__/page/*
 
28
  !preprocess/oneformer_code/oneformer/data/bpe_simple_vocab_16e6.txt
29
  !config/*.json
30
  !requirements.txt
31
+ !requirements/*
 
 
app.py CHANGED
@@ -72,25 +72,37 @@ MARKDOWN = \
72
 
73
  </div>
74
 
75
- Frame In-N-Out extends the first-frame conditioning to a larger spatial canvas by specifying top-left and bottom-right expansion offsets.
76
- Further, it allows users to assign motion trajectories to existing objects, introduce new identities that enter the scene with their own trajectories, or both.<br>
77
  The model we used here is [<b>Wan2.2-5B</b> V1.6](https://huggingface.co/uva-cv-lab/FrameINO_Wan2.2_5B_Stage2_MotionINO_v1.6) trained on our Frame In-N-Out control mechanism.
78
 
79
 
80
  <br>
81
- <b>Easiest way:</b>
82
- Choose one example and then simply click <b>Generate</b>.
 
 
83
 
 
 
 
 
 
 
 
84
  <br>
 
 
 
 
 
 
 
 
 
 
 
 
85
  <br>
86
- ❗️❗️❗️Instruction Steps:<br>
87
- 1️⃣ Upload your first frame image. Set the size you want to resize to for <b>Resized Height for Input Image</b> and <b>Resized Width for Input Image</b>. <br>
88
- 2️⃣ Set your <b>canvas top left</b> and <b>bottom right expansion</b>. The combined height and width should be the multiplier of 32. <br>
89
- Recommend <b>Canvas HEIGHT = 704</b> and <b>Canvas WIDTH = 1280</b> for the best performance (Pre-trained training Resolution). <br>
90
- 3️⃣ Click <b>Build the Canvas</b>. <br>
91
- 4️⃣ Provide the trajectory of the main object in the canvas by clicking on the <b>Expanded Canvas</b>. <br>
92
- 5️⃣ Provide the ID reference image and its trajectory (optional). Also, write a detailed <b>text prompt</b>. <br>
93
- Click the <b>Generate</b> button to start the Video Generation. <br>
94
 
95
 
96
  If **Frame In-N-Out** is helpful, please help star the [GitHub Repo](https://github.com/UVA-Computer-Vision-Lab/FrameINO?tab=readme-ov-file). Thanks!
@@ -804,7 +816,7 @@ if __name__ == '__main__':
804
  with gr.Row():
805
 
806
  # Button
807
- generation_btn = gr.Button(value="Generate")
808
 
809
 
810
  with gr.Row():
 
72
 
73
  </div>
74
 
75
+ Frame In-N-Out expands the first-frame to a larger canvas, where it allows users to assign motion trajectories to existing objects and introduce new identities that enter the scene with specified trajectories.<br>
 
76
  The model we used here is [<b>Wan2.2-5B</b> V1.6](https://huggingface.co/uva-cv-lab/FrameINO_Wan2.2_5B_Stage2_MotionINO_v1.6) trained on our Frame In-N-Out control mechanism.
77
 
78
 
79
  <br>
80
+ <p style="color: red;">
81
+ <b>Easiest way:</b> Choose one from <b>Examples</b> below and then simply click <b>Generate</b>.
82
+ </p>
83
+ <br>
84
 
85
+ ❗️❗️❗️Instruction Steps:<br>
86
+ 1️⃣ Upload your <b>Input Image 🖼️ </b>.
87
+ Next, set <b>Resized Height for Input Image</b> and <b>Resized Width for Input Image</b> for the size you want.
88
+ <br>
89
+ 2️⃣ Set <b>Top-Left Expand Height</b>, <b>Top-Left Expand Width</b>, <b>Bottom-Right Expand Height</b>, and <b>Bottom-Right Expand Width</b> for the expansion amount.
90
+ <br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; The Canvas Height (Resized Height + Top-Left Expand Height + Bottom-Right Expand Height) and Canvas Width (Resized Width + Top-Left Expand Width + Bottom-Right Expand Width) should be the multiplier of 32.
91
+ <br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Recommend <b>Canvas Height = 704</b> and <b>Canvas Width = 1280</b> for the best performance (pre-trained model default resolution).
92
  <br>
93
+ 3️⃣ Click <b>Build the Canvas</b>.
94
+ 4️⃣ Provide the motion trajectory of the object by clicking on the <b>Expanded Canvas 🖼️ </b>.
95
+ You can make additional trajectory for the same object by clicking <b>Add New Traj Line (Same Obj)</b>.
96
+ Reset by <b>Clear All Traj</b>.
97
+ <br>
98
+ 5️⃣ Provide the <b>Identity Reference</b> image and its trajectory (optional).
99
+ Since image is segmented by SAM first (providng center point as query), it will be nice for the inputs to be center cropped.
100
+ <br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; New instance trajectory can be done by clicking <b>Add New Instance (New Obj, including new ID)</b>.
101
+ <br>
102
+ 6️⃣ Write a detailed <b>text prompt</b>.
103
+ <br>
104
+ 7️⃣ Click the <b>Generate!</b> button to start the Video Generation.
105
  <br>
 
 
 
 
 
 
 
 
106
 
107
 
108
  If **Frame In-N-Out** is helpful, please help star the [GitHub Repo](https://github.com/UVA-Computer-Vision-Lab/FrameINO?tab=readme-ov-file). Thanks!
 
816
  with gr.Row():
817
 
818
  # Button
819
+ generation_btn = gr.Button(value="Generate!")
820
 
821
 
822
  with gr.Row():