Hugging Face's logo

midbee
/
Janus-Pro-R1-7B

Safetensors
multi_modality
Model card Files
xet
Community
1
wendell0218 commited on
Commit
9dd8828
·
1 Parent(s): 832597d

Update ckpt

Browse files
Files changed (8) hide show
  1. LEGAL.md +7 -0
  2. Readme.md +2 -0
  3. model-00001-of-00003.safetensors +1 -1
  4. model-00002-of-00003.safetensors +1 -1
  5. model-00003-of-00003.safetensors +1 -1
  6. tokenizer.json +0 -0
  7. trainer_state.json +1434 -0
  8. training_args.bin +3 -0
LEGAL.md ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ Legal Disclaimer
2
+
3
+ Within this source code, the comments in Chinese shall be the original, governing version. Any comment in other languages are for reference only. In the event of any conflict between the Chinese language version comments and other language version comments, the Chinese language version shall prevail.
4
+
5
+ 法律免责声明
6
+
7
+ 关于代码注释部分，中文注释为官方版本，其它语言注释仅做参考。中文注释可能与其它语言注释存在不一致，当中文注释与其它语言注释存在不一致时，请以中文注释为准。
Readme.md ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ # Readme
2
+
model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:58aa8bb3766483f358cb063d61c82ed10909e2d8f9c965258a59ce3ac53947ef
3
  size 4916851534
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5906d7caf823c47f36c35788361be2ebd15cd52fd8d2877675ce6d31a6dae7a5
3
  size 4916851534
model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:83b4b98e3b7901f05491a301081fb71f75b3cd77ad3c7d508f4788ffaac59b11
3
  size 4947392496
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eda8d16618070afb26cee0b20979cc6dae8912f3d33b002f0cbb2debb2d6367e
3
  size 4947392496
model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:045d87d83c8646a0c49a0773f24327f6d5be5a37b80ed78063308d49fb102b01
3
  size 4976742608
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:93cabf9a2cf52906ab742b382dd95535d2c8232ed619de8e9cc28e69a6f0b082
3
  size 4976742608
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
trainer_state.json ADDED
@@ -0,0 +1,1434 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 0.5235602094240838,
6
+ "eval_steps": 100000,
7
+ "global_step": 50,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "completion_length_0": 576.0,
14
+ "completion_length_1": 59.8125,
15
+ "completion_length_2": 576.0,
16
+ "compre": 0.7549091279506683,
17
+ "compre_std": 0.14603755250573158,
18
+ "epoch": 0.010471204188481676,
19
+ "finalgen": 0.7989666610956192,
20
+ "first_gen": 0.7695323675870895,
21
+ "first_gen_std": 0.09897536411881447,
22
+ "geneval_final": 0.8333281576633453,
23
+ "geneval_first": 0.833327904343605,
24
+ "geneval_second": 0.7552048228681087,
25
+ "grad_norm": 1.0952143669128418,
26
+ "internvl_final": 0.7692298889160156,
27
+ "internvl_first": 0.7423409670591354,
28
+ "internvl_second": 0.7017457336187363,
29
+ "kl_0": 0.0,
30
+ "kl_1": 0.0,
31
+ "kl_2": 0.0,
32
+ "learning_rate": 1.0000000000000002e-06,
33
+ "loss": 0.0,
34
+ "rewards/selfcheck": 0.640625,
35
+ "second_gen": 0.7350995540618896,
36
+ "second_gen_std": 0.16677509993314743,
37
+ "step": 1,
38
+ "tot_reward": 2.2595410495996475
39
+ },
40
+ {
41
+ "completion_length_0": 576.0,
42
+ "completion_length_1": 39.84375,
43
+ "completion_length_2": 576.0,
44
+ "compre": 0.737465888261795,
45
+ "compre_std": 0.23401770368218422,
46
+ "epoch": 0.020942408376963352,
47
+ "finalgen": 0.7441405802965164,
48
+ "first_gen": 0.7343152761459351,
49
+ "first_gen_std": 0.17313585616648197,
50
+ "geneval_final": 0.6437513083219528,
51
+ "geneval_first": 0.653126060962677,
52
+ "geneval_second": 0.5812521427869797,
53
+ "grad_norm": 1.0801606178283691,
54
+ "internvl_final": 0.8145325481891632,
55
+ "internvl_first": 0.8075367659330368,
56
+ "internvl_second": 0.7767415642738342,
57
+ "kl_0": 0.0,
58
+ "kl_1": 0.0,
59
+ "kl_2": 0.0,
60
+ "learning_rate": 2.0000000000000003e-06,
61
+ "loss": 0.0,
62
+ "rewards/selfcheck": 0.66796875,
63
+ "second_gen": 0.6918127685785294,
64
+ "second_gen_std": 0.1750396341085434,
65
+ "step": 2,
66
+ "tot_reward": 2.1635939329862595
67
+ },
68
+ {
69
+ "completion_length_0": 576.0,
70
+ "completion_length_1": 48.5625,
71
+ "completion_length_2": 576.0,
72
+ "compre": 0.6974785923957825,
73
+ "compre_std": 0.2092800009995699,
74
+ "epoch": 0.031413612565445025,
75
+ "finalgen": 0.7925812155008316,
76
+ "first_gen": 0.8119042217731476,
77
+ "first_gen_std": 0.16565837524831295,
78
+ "geneval_final": 0.8124982565641403,
79
+ "geneval_first": 0.8255188465118408,
80
+ "geneval_second": 0.7421867400407791,
81
+ "grad_norm": 1.3857570886611938,
82
+ "internvl_final": 0.7827608734369278,
83
+ "internvl_first": 0.8039457201957703,
84
+ "internvl_second": 0.741925835609436,
85
+ "kl_0": 0.004405975341796875,
86
+ "kl_1": 0.00030541419982910156,
87
+ "kl_2": 0.00640106201171875,
88
+ "learning_rate": 3.0000000000000005e-06,
89
+ "loss": 0.0001,
90
+ "rewards/selfcheck": 0.66015625,
91
+ "second_gen": 0.7375303208827972,
92
+ "second_gen_std": 0.244943518191576,
93
+ "step": 3,
94
+ "tot_reward": 2.2469131350517273
95
+ },
96
+ {
97
+ "completion_length_0": 576.0,
98
+ "completion_length_1": 63.875,
99
+ "completion_length_2": 576.0,
100
+ "compre": 0.6304676532745361,
101
+ "compre_std": 0.2166498824954033,
102
+ "epoch": 0.041884816753926704,
103
+ "finalgen": 0.706999346613884,
104
+ "first_gen": 0.7176250070333481,
105
+ "first_gen_std": 0.1319098360836506,
106
+ "geneval_final": 0.5312501322478056,
107
+ "geneval_first": 0.5833326280117035,
108
+ "geneval_second": 0.42708506839153415,
109
+ "grad_norm": 0.9866518378257751,
110
+ "internvl_final": 0.7840945720672607,
111
+ "internvl_first": 0.7837744653224945,
112
+ "internvl_second": 0.7741784453392029,
113
+ "kl_0": 0.00339508056640625,
114
+ "kl_1": 0.00030803680419921875,
115
+ "kl_2": 0.005641937255859375,
116
+ "learning_rate": 4.000000000000001e-06,
117
+ "loss": 0.0001,
118
+ "rewards/selfcheck": 0.59765625,
119
+ "second_gen": 0.6680488586425781,
120
+ "second_gen_std": 0.13900070078670979,
121
+ "step": 4,
122
+ "tot_reward": 2.0161415189504623
123
+ },
124
+ {
125
+ "completion_length_0": 576.0,
126
+ "completion_length_1": 56.875,
127
+ "completion_length_2": 576.0,
128
+ "compre": 0.7337857931852341,
129
+ "compre_std": 0.17267515510320663,
130
+ "epoch": 0.05235602094240838,
131
+ "finalgen": 0.8179268836975098,
132
+ "first_gen": 0.8474402278661728,
133
+ "first_gen_std": 0.10339293256402016,
134
+ "geneval_final": 0.4921884387731552,
135
+ "geneval_first": 0.5651040226221085,
136
+ "geneval_second": 0.4505223035812378,
137
+ "grad_norm": 0.9777835607528687,
138
+ "internvl_final": 0.88887158036232,
139
+ "internvl_first": 0.8897245526313782,
140
+ "internvl_second": 0.8588467538356781,
141
+ "kl_0": 0.00335693359375,
142
+ "kl_1": 0.0003905296325683594,
143
+ "kl_2": 0.006153106689453125,
144
+ "learning_rate": 5e-06,
145
+ "loss": 0.0001,
146
+ "rewards/selfcheck": 0.6484375,
147
+ "second_gen": 0.768099382519722,
148
+ "second_gen_std": 0.19330058619379997,
149
+ "step": 5,
150
+ "tot_reward": 2.349325403571129
151
+ },
152
+ {
153
+ "completion_length_0": 576.0,
154
+ "completion_length_1": 70.28125,
155
+ "completion_length_2": 576.0,
156
+ "compre": 0.8213787376880646,
157
+ "compre_std": 0.17125927656888962,
158
+ "epoch": 0.06282722513089005,
159
+ "finalgen": 0.8025798797607422,
160
+ "first_gen": 0.792873278260231,
161
+ "first_gen_std": 0.1552301160991192,
162
+ "geneval_final": 0.7744787484407425,
163
+ "geneval_first": 0.7614581435918808,
164
+ "geneval_second": 0.638022243976593,
165
+ "grad_norm": 0.926077663898468,
166
+ "internvl_final": 0.8014426976442337,
167
+ "internvl_first": 0.7876186370849609,
168
+ "internvl_second": 0.7742805629968643,
169
+ "kl_0": 0.003208160400390625,
170
+ "kl_1": 0.004992485046386719,
171
+ "kl_2": 0.00876617431640625,
172
+ "learning_rate": 4.84e-06,
173
+ "loss": 0.0005,
174
+ "rewards/selfcheck": 0.72265625,
175
+ "second_gen": 0.7216387689113617,
176
+ "second_gen_std": 0.16806087456643581,
177
+ "step": 6,
178
+ "tot_reward": 2.3358907848596573
179
+ },
180
+ {
181
+ "completion_length_0": 576.0,
182
+ "completion_length_1": 52.0,
183
+ "completion_length_2": 576.0,
184
+ "compre": 0.8553071022033691,
185
+ "compre_std": 0.11782849393785,
186
+ "epoch": 0.07329842931937172,
187
+ "finalgen": 0.8177923709154129,
188
+ "first_gen": 0.784370481967926,
189
+ "first_gen_std": 0.12809692788869143,
190
+ "geneval_final": 0.8593727946281433,
191
+ "geneval_first": 0.7786447405815125,
192
+ "geneval_second": 0.7109374031424522,
193
+ "grad_norm": 0.9277043342590332,
194
+ "internvl_final": 0.7972260862588882,
195
+ "internvl_first": 0.7938382625579834,
196
+ "internvl_second": 0.7694690227508545,
197
+ "kl_0": 0.004093170166015625,
198
+ "kl_1": 0.003676891326904297,
199
+ "kl_2": 0.0273284912109375,
200
+ "learning_rate": 4.68e-06,
201
+ "loss": 0.0013,
202
+ "rewards/selfcheck": 0.72265625,
203
+ "second_gen": 0.7431355491280556,
204
+ "second_gen_std": 0.19814559444785118,
205
+ "step": 7,
206
+ "tot_reward": 2.3828131332993507
207
+ },
208
+ {
209
+ "completion_length_0": 576.0,
210
+ "completion_length_1": 62.15625,
211
+ "completion_length_2": 576.0,
212
+ "compre": 0.7571111023426056,
213
+ "compre_std": 0.18298335000872612,
214
+ "epoch": 0.08376963350785341,
215
+ "finalgen": 0.7291563600301743,
216
+ "first_gen": 0.6889523267745972,
217
+ "first_gen_std": 0.18111810460686684,
218
+ "geneval_final": 0.5364595651626587,
219
+ "geneval_first": 0.4895849898457527,
220
+ "geneval_second": 0.39583662897348404,
221
+ "grad_norm": 3.1168320178985596,
222
+ "internvl_final": 0.8005142956972122,
223
+ "internvl_first": 0.7751448452472687,
224
+ "internvl_second": 0.7655676752328873,
225
+ "kl_0": 0.00445556640625,
226
+ "kl_1": 0.3070030212402344,
227
+ "kl_2": 0.050140380859375,
228
+ "learning_rate": 4.520000000000001e-06,
229
+ "loss": 0.0085,
230
+ "rewards/selfcheck": 0.640625,
231
+ "second_gen": 0.6585755944252014,
232
+ "second_gen_std": 0.1962592490017414,
233
+ "step": 8,
234
+ "tot_reward": 2.104639023542404
235
+ },
236
+ {
237
+ "completion_length_0": 576.0,
238
+ "completion_length_1": 59.03125,
239
+ "completion_length_2": 576.0,
240
+ "compre": 0.8145061582326889,
241
+ "compre_std": 0.20007650181651115,
242
+ "epoch": 0.09424083769633508,
243
+ "finalgen": 0.7829647809267044,
244
+ "first_gen": 0.7806855589151382,
245
+ "first_gen_std": 0.1751751657575369,
246
+ "geneval_final": 0.6697908341884613,
247
+ "geneval_first": 0.6677073389291763,
248
+ "geneval_second": 0.5239605121314526,
249
+ "grad_norm": 1.870141625404358,
250
+ "internvl_final": 0.8357957899570465,
251
+ "internvl_first": 0.842738002538681,
252
+ "internvl_second": 0.8030338883399963,
253
+ "kl_0": 0.0045166015625,
254
+ "kl_1": 0.00498199462890625,
255
+ "kl_2": 0.26806640625,
256
+ "learning_rate": 4.360000000000001e-06,
257
+ "loss": 0.005,
258
+ "rewards/selfcheck": 0.8671875,
259
+ "second_gen": 0.7249370217323303,
260
+ "second_gen_std": 0.2135489135980606,
261
+ "step": 9,
262
+ "tot_reward": 2.3201287388801575
263
+ },
264
+ {
265
+ "completion_length_0": 576.0,
266
+ "completion_length_1": 51.4375,
267
+ "completion_length_2": 576.0,
268
+ "compre": 0.8112478852272034,
269
+ "compre_std": 0.11972623318433762,
270
+ "epoch": 0.10471204188481675,
271
+ "finalgen": 0.8595622777938843,
272
+ "first_gen": 0.8183109611272812,
273
+ "first_gen_std": 0.1156176682561636,
274
+ "geneval_final": 0.8749973326921463,
275
+ "geneval_first": 0.8124981969594955,
276
+ "geneval_second": 0.8645808100700378,
277
+ "grad_norm": 1.774263620376587,
278
+ "internvl_final": 0.8654665052890778,
279
+ "internvl_first": 0.83393794298172,
280
+ "internvl_second": 0.7860487401485443,
281
+ "kl_0": 0.00567626953125,
282
+ "kl_1": 0.03774833679199219,
283
+ "kl_2": 0.168212890625,
284
+ "learning_rate": 4.2000000000000004e-06,
285
+ "loss": 0.0067,
286
+ "rewards/selfcheck": 0.83984375,
287
+ "second_gen": 0.812427893280983,
288
+ "second_gen_std": 0.1961305681616068,
289
+ "step": 10,
290
+ "tot_reward": 2.4419867396354675
291
+ },
292
+ {
293
+ "completion_length_0": 576.0,
294
+ "completion_length_1": 59.4375,
295
+ "completion_length_2": 576.0,
296
+ "compre": 0.7816697210073471,
297
+ "compre_std": 0.16708647273480892,
298
+ "epoch": 0.11518324607329843,
299
+ "finalgen": 0.7165499031543732,
300
+ "first_gen": 0.7182296440005302,
301
+ "first_gen_std": 0.16919826716184616,
302
+ "geneval_final": 0.68958330899477,
303
+ "geneval_first": 0.6770834550261497,
304
+ "geneval_second": 0.5447937361896038,
305
+ "grad_norm": 1.3713111877441406,
306
+ "internvl_final": 0.7387097775936127,
307
+ "internvl_first": 0.7484088242053986,
308
+ "internvl_second": 0.6730778962373734,
309
+ "kl_0": 0.00601959228515625,
310
+ "kl_1": 0.01692962646484375,
311
+ "kl_2": 0.1885986328125,
312
+ "learning_rate": 4.04e-06,
313
+ "loss": 0.0052,
314
+ "rewards/selfcheck": 0.70703125,
315
+ "second_gen": 0.6220707595348358,
316
+ "second_gen_std": 0.23473705351352692,
317
+ "step": 11,
318
+ "tot_reward": 2.121970124542713
319
+ },
320
+ {
321
+ "completion_length_0": 576.0,
322
+ "completion_length_1": 56.96875,
323
+ "completion_length_2": 576.0,
324
+ "compre": 0.7651939988136292,
325
+ "compre_std": 0.16647255420684814,
326
+ "epoch": 0.1256544502617801,
327
+ "finalgen": 0.737395167350769,
328
+ "first_gen": 0.7266053706407547,
329
+ "first_gen_std": 0.16155973821878433,
330
+ "geneval_final": 0.5595262910433121,
331
+ "geneval_first": 0.6250007450580597,
332
+ "geneval_second": 0.48363446846678926,
333
+ "grad_norm": 1.548578143119812,
334
+ "internvl_final": 0.8127259463071823,
335
+ "internvl_first": 0.8052111566066742,
336
+ "internvl_second": 0.7006373405456543,
337
+ "kl_0": 0.0052490234375,
338
+ "kl_1": 0.105865478515625,
339
+ "kl_2": 0.1307373046875,
340
+ "learning_rate": 3.88e-06,
341
+ "loss": 0.0033,
342
+ "rewards/selfcheck": 0.7265625,
343
+ "second_gen": 0.6569458693265915,
344
+ "second_gen_std": 0.19354603812098503,
345
+ "step": 12,
346
+ "tot_reward": 2.1487452387809753
347
+ },
348
+ {
349
+ "completion_length_0": 576.0,
350
+ "completion_length_1": 50.96875,
351
+ "completion_length_2": 576.0,
352
+ "compre": 0.7275254130363464,
353
+ "compre_std": 0.22794683650135994,
354
+ "epoch": 0.13612565445026178,
355
+ "finalgen": 0.771732971072197,
356
+ "first_gen": 0.7685244232416153,
357
+ "first_gen_std": 0.18027238175272942,
358
+ "geneval_final": 0.7343721091747284,
359
+ "geneval_first": 0.7031228244304657,
360
+ "geneval_second": 0.5703130066394806,
361
+ "grad_norm": 0.9769389629364014,
362
+ "internvl_final": 0.8102993816137314,
363
+ "internvl_first": 0.8013420253992081,
364
+ "internvl_second": 0.7744639068841934,
365
+ "kl_0": 0.005084991455078125,
366
+ "kl_1": 0.18712615966796875,
367
+ "kl_2": 0.159912109375,
368
+ "learning_rate": 3.7200000000000004e-06,
369
+ "loss": 0.0035,
370
+ "rewards/selfcheck": 0.66796875,
371
+ "second_gen": 0.7124785929918289,
372
+ "second_gen_std": 0.20800333097577095,
373
+ "step": 13,
374
+ "tot_reward": 2.2085284292697906
375
+ },
376
+ {
377
+ "completion_length_0": 576.0,
378
+ "completion_length_1": 47.96875,
379
+ "completion_length_2": 576.0,
380
+ "compre": 0.7608457058668137,
381
+ "compre_std": 0.20356886088848114,
382
+ "epoch": 0.14659685863874344,
383
+ "finalgen": 0.7777686715126038,
384
+ "first_gen": 0.788392037153244,
385
+ "first_gen_std": 0.19788672029972076,
386
+ "geneval_final": 0.721354216337204,
387
+ "geneval_first": 0.7369789481163025,
388
+ "geneval_second": 0.611980676651001,
389
+ "grad_norm": 1.6281715631484985,
390
+ "internvl_final": 0.8137162774801254,
391
+ "internvl_first": 0.8182865977287292,
392
+ "internvl_second": 0.749033197760582,
393
+ "kl_0": 0.005817413330078125,
394
+ "kl_1": 0.04917144775390625,
395
+ "kl_2": 0.12237548828125,
396
+ "learning_rate": 3.5600000000000002e-06,
397
+ "loss": 0.0028,
398
+ "rewards/selfcheck": 0.72265625,
399
+ "second_gen": 0.6939488351345062,
400
+ "second_gen_std": 0.26941393315792084,
401
+ "step": 14,
402
+ "tot_reward": 2.243186578154564
403
+ },
404
+ {
405
+ "completion_length_0": 576.0,
406
+ "completion_length_1": 55.375,
407
+ "completion_length_2": 576.0,
408
+ "compre": 0.7917969673871994,
409
+ "compre_std": 0.21661141142249107,
410
+ "epoch": 0.15706806282722513,
411
+ "finalgen": 0.7536004930734634,
412
+ "first_gen": 0.7422216385602951,
413
+ "first_gen_std": 0.1565408930182457,
414
+ "geneval_final": 0.6354171559214592,
415
+ "geneval_first": 0.6197923645377159,
416
+ "geneval_second": 0.567709818482399,
417
+ "grad_norm": 1.154263973236084,
418
+ "internvl_final": 0.8050427883863449,
419
+ "internvl_first": 0.7937590032815933,
420
+ "internvl_second": 0.7572074085474014,
421
+ "kl_0": 0.005199432373046875,
422
+ "kl_1": 0.057464599609375,
423
+ "kl_2": 0.116455078125,
424
+ "learning_rate": 3.4000000000000005e-06,
425
+ "loss": 0.0028,
426
+ "rewards/selfcheck": 0.74609375,
427
+ "second_gen": 0.6974593102931976,
428
+ "second_gen_std": 0.16282034665346146,
429
+ "step": 15,
430
+ "tot_reward": 2.231477916240692
431
+ },
432
+ {
433
+ "completion_length_0": 576.0,
434
+ "completion_length_1": 51.90625,
435
+ "completion_length_2": 576.0,
436
+ "compre": 0.7714394629001617,
437
+ "compre_std": 0.15307269245386124,
438
+ "epoch": 0.16753926701570682,
439
+ "finalgen": 0.8282599300146103,
440
+ "first_gen": 0.8087948560714722,
441
+ "first_gen_std": 0.1717423852533102,
442
+ "geneval_final": 0.8437473922967911,
443
+ "geneval_first": 0.794269010424614,
444
+ "geneval_second": 0.7343740463256836,
445
+ "grad_norm": 0.7912507057189941,
446
+ "internvl_final": 0.8215305358171463,
447
+ "internvl_first": 0.8133489638566971,
448
+ "internvl_second": 0.8031008988618851,
449
+ "kl_0": 0.00527191162109375,
450
+ "kl_1": 0.067535400390625,
451
+ "kl_2": 0.11920166015625,
452
+ "learning_rate": 3.2400000000000003e-06,
453
+ "loss": 0.0026,
454
+ "rewards/selfcheck": 0.76171875,
455
+ "second_gen": 0.7794294506311417,
456
+ "second_gen_std": 0.21150891855359077,
457
+ "step": 16,
458
+ "tot_reward": 2.3596637696027756
459
+ },
460
+ {
461
+ "completion_length_0": 576.0,
462
+ "completion_length_1": 50.84375,
463
+ "completion_length_2": 576.0,
464
+ "compre": 0.8622442930936813,
465
+ "compre_std": 0.1704554744064808,
466
+ "epoch": 0.17801047120418848,
467
+ "finalgen": 0.8196248710155487,
468
+ "first_gen": 0.7956789284944534,
469
+ "first_gen_std": 0.1943669207394123,
470
+ "geneval_final": 0.7427082657814026,
471
+ "geneval_first": 0.7041671276092529,
472
+ "geneval_second": 0.6416679620742798,
473
+ "grad_norm": 0.9625406265258789,
474
+ "internvl_final": 0.8516624420881271,
475
+ "internvl_first": 0.8354064524173737,
476
+ "internvl_second": 0.8153494000434875,
477
+ "kl_0": 0.00511932373046875,
478
+ "kl_1": 0.0963134765625,
479
+ "kl_2": 0.14788818359375,
480
+ "learning_rate": 3.08e-06,
481
+ "loss": 0.0023,
482
+ "rewards/selfcheck": 0.84375,
483
+ "second_gen": 0.7515811622142792,
484
+ "second_gen_std": 0.22218142822384834,
485
+ "step": 17,
486
+ "tot_reward": 2.409504383802414
487
+ },
488
+ {
489
+ "completion_length_0": 576.0,
490
+ "completion_length_1": 48.40625,
491
+ "completion_length_2": 576.0,
492
+ "compre": 0.861387774348259,
493
+ "compre_std": 0.13461659289896488,
494
+ "epoch": 0.18848167539267016,
495
+ "finalgen": 0.8284620344638824,
496
+ "first_gen": 0.8211266100406647,
497
+ "first_gen_std": 0.17254696041345596,
498
+ "geneval_final": 0.7864573150873184,
499
+ "geneval_first": 0.7812490016222,
500
+ "geneval_second": 0.6484384983778,
501
+ "grad_norm": 0.7912412881851196,
502
+ "internvl_final": 0.8521965593099594,
503
+ "internvl_first": 0.8812606334686279,
504
+ "internvl_second": 0.7947878241539001,
505
+ "kl_0": 0.004608154296875,
506
+ "kl_1": 0.0240936279296875,
507
+ "kl_2": 0.09210205078125,
508
+ "learning_rate": 2.92e-06,
509
+ "loss": 0.0021,
510
+ "rewards/selfcheck": 0.78125,
511
+ "second_gen": 0.7385903745889664,
512
+ "second_gen_std": 0.20756110176444054,
513
+ "step": 18,
514
+ "tot_reward": 2.42110475897789
515
+ },
516
+ {
517
+ "completion_length_0": 576.0,
518
+ "completion_length_1": 49.28125,
519
+ "completion_length_2": 576.0,
520
+ "compre": 0.7792719453573227,
521
+ "compre_std": 0.18045332841575146,
522
+ "epoch": 0.19895287958115182,
523
+ "finalgen": 0.7645432502031326,
524
+ "first_gen": 0.7664026916027069,
525
+ "first_gen_std": 0.1389799863100052,
526
+ "geneval_final": 0.7083334475755692,
527
+ "geneval_first": 0.6979169696569443,
528
+ "geneval_second": 0.674479752779007,
529
+ "grad_norm": 1.4261727333068848,
530
+ "internvl_final": 0.7876286506652832,
531
+ "internvl_first": 0.7962436228990555,
532
+ "internvl_second": 0.7371761798858643,
533
+ "kl_0": 0.004669189453125,
534
+ "kl_1": 0.01157379150390625,
535
+ "kl_2": 0.12786865234375,
536
+ "learning_rate": 2.7600000000000003e-06,
537
+ "loss": 0.0022,
538
+ "rewards/selfcheck": 0.6796875,
539
+ "second_gen": 0.7210484892129898,
540
+ "second_gen_std": 0.16337783448398113,
541
+ "step": 19,
542
+ "tot_reward": 2.2667231261730194
543
+ },
544
+ {
545
+ "completion_length_0": 576.0,
546
+ "completion_length_1": 43.25,
547
+ "completion_length_2": 576.0,
548
+ "compre": 0.8075016289949417,
549
+ "compre_std": 0.1704008486121893,
550
+ "epoch": 0.2094240837696335,
551
+ "finalgen": 0.7825709283351898,
552
+ "first_gen": 0.7692948579788208,
553
+ "first_gen_std": 0.13403410837054253,
554
+ "geneval_final": 0.7145832553505898,
555
+ "geneval_first": 0.6958335563540459,
556
+ "geneval_second": 0.6552091091871262,
557
+ "grad_norm": 0.9123572111129761,
558
+ "internvl_final": 0.8629064708948135,
559
+ "internvl_first": 0.8535181134939194,
560
+ "internvl_second": 0.8180961459875107,
561
+ "kl_0": 0.00579833984375,
562
+ "kl_1": 0.050815582275390625,
563
+ "kl_2": 0.064666748046875,
564
+ "learning_rate": 2.6e-06,
565
+ "loss": 0.0024,
566
+ "rewards/selfcheck": 0.7109375,
567
+ "second_gen": 0.7336102575063705,
568
+ "second_gen_std": 0.17554061487317085,
569
+ "step": 20,
570
+ "tot_reward": 2.310406744480133
571
+ },
572
+ {
573
+ "completion_length_0": 576.0,
574
+ "completion_length_1": 63.96875,
575
+ "completion_length_2": 576.0,
576
+ "compre": 0.776437446475029,
577
+ "compre_std": 0.18083850666880608,
578
+ "epoch": 0.2198952879581152,
579
+ "finalgen": 0.7892505973577499,
580
+ "first_gen": 0.7880778908729553,
581
+ "first_gen_std": 0.12110341340303421,
582
+ "geneval_final": 0.5729151666164398,
583
+ "geneval_first": 0.578123465180397,
584
+ "geneval_second": 0.5781233906745911,
585
+ "grad_norm": 1.147928237915039,
586
+ "internvl_final": 0.8042012304067612,
587
+ "internvl_first": 0.7973868250846863,
588
+ "internvl_second": 0.7700908780097961,
589
+ "kl_0": 0.00434112548828125,
590
+ "kl_1": 0.070892333984375,
591
+ "kl_2": 0.0592803955078125,
592
+ "learning_rate": 2.4400000000000004e-06,
593
+ "loss": 0.0019,
594
+ "rewards/selfcheck": 0.7890625,
595
+ "second_gen": 0.7677064388990402,
596
+ "second_gen_std": 0.14560945704579353,
597
+ "step": 21,
598
+ "tot_reward": 2.3322217762470245
599
+ },
600
+ {
601
+ "completion_length_0": 576.0,
602
+ "completion_length_1": 46.78125,
603
+ "completion_length_2": 576.0,
604
+ "compre": 0.7662104368209839,
605
+ "compre_std": 0.18649493157863617,
606
+ "epoch": 0.23036649214659685,
607
+ "finalgen": 0.7516999542713165,
608
+ "first_gen": 0.7476924508810043,
609
+ "first_gen_std": 0.16294222325086594,
610
+ "geneval_final": 0.49739643931388855,
611
+ "geneval_first": 0.5156253203749657,
612
+ "geneval_second": 0.437501423060894,
613
+ "grad_norm": 0.7344913482666016,
614
+ "internvl_final": 0.8158693909645081,
615
+ "internvl_first": 0.8004629462957382,
616
+ "internvl_second": 0.7996131181716919,
617
+ "kl_0": 0.00626373291015625,
618
+ "kl_1": 0.011867523193359375,
619
+ "kl_2": 0.0703125,
620
+ "learning_rate": 2.2800000000000006e-06,
621
+ "loss": 0.0031,
622
+ "rewards/selfcheck": 0.67578125,
623
+ "second_gen": 0.7156740128993988,
624
+ "second_gen_std": 0.1773982010781765,
625
+ "step": 22,
626
+ "tot_reward": 2.229576900601387
627
+ },
628
+ {
629
+ "completion_length_0": 576.0,
630
+ "completion_length_1": 38.59375,
631
+ "completion_length_2": 576.0,
632
+ "compre": 0.8990298211574554,
633
+ "compre_std": 0.08603513706475496,
634
+ "epoch": 0.24083769633507854,
635
+ "finalgen": 0.8516775518655777,
636
+ "first_gen": 0.8254719525575638,
637
+ "first_gen_std": 0.09153617918491364,
638
+ "geneval_final": 0.8333310335874557,
639
+ "geneval_first": 0.7708318829536438,
640
+ "geneval_second": 0.8229145109653473,
641
+ "grad_norm": 0.6684104800224304,
642
+ "internvl_final": 0.8759130835533142,
643
+ "internvl_first": 0.8701326102018356,
644
+ "internvl_second": 0.8153904676437378,
645
+ "kl_0": 0.00778961181640625,
646
+ "kl_1": 0.01245880126953125,
647
+ "kl_2": 0.0904541015625,
648
+ "learning_rate": 2.1200000000000005e-06,
649
+ "loss": 0.002,
650
+ "rewards/selfcheck": 0.81640625,
651
+ "second_gen": 0.8093397468328476,
652
+ "second_gen_std": 0.13964065350592136,
653
+ "step": 23,
654
+ "tot_reward": 2.533841520547867
655
+ },
656
+ {
657
+ "completion_length_0": 576.0,
658
+ "completion_length_1": 50.21875,
659
+ "completion_length_2": 576.0,
660
+ "compre": 0.8330367654561996,
661
+ "compre_std": 0.16215411643497646,
662
+ "epoch": 0.2513089005235602,
663
+ "finalgen": 0.7843883782625198,
664
+ "first_gen": 0.7797036319971085,
665
+ "first_gen_std": 0.12760567921213806,
666
+ "geneval_final": 0.6000005006790161,
667
+ "geneval_first": 0.5979171544313431,
668
+ "geneval_second": 0.4541696608066559,
669
+ "grad_norm": 0.6334791779518127,
670
+ "internvl_final": 0.8782133162021637,
671
+ "internvl_first": 0.8767320364713669,
672
+ "internvl_second": 0.8230772018432617,
673
+ "kl_0": 0.0048828125,
674
+ "kl_1": 0.0131378173828125,
675
+ "kl_2": 0.048858642578125,
676
+ "learning_rate": 1.9600000000000003e-06,
677
+ "loss": 0.0018,
678
+ "rewards/selfcheck": 0.828125,
679
+ "second_gen": 0.7158899009227753,
680
+ "second_gen_std": 0.15695794858038425,
681
+ "step": 24,
682
+ "tot_reward": 2.3286302983760834
683
+ },
684
+ {
685
+ "completion_length_0": 576.0,
686
+ "completion_length_1": 51.78125,
687
+ "completion_length_2": 576.0,
688
+ "compre": 0.8843115568161011,
689
+ "compre_std": 0.1233286838978529,
690
+ "epoch": 0.2617801047120419,
691
+ "finalgen": 0.8016263395547867,
692
+ "first_gen": 0.7955255508422852,
693
+ "first_gen_std": 0.13115915283560753,
694
+ "geneval_final": 0.7031246274709702,
695
+ "geneval_first": 0.7057287618517876,
696
+ "geneval_second": 0.5781264305114746,
697
+ "grad_norm": 1.0080742835998535,
698
+ "internvl_final": 0.8972594141960144,
699
+ "internvl_first": 0.8881365358829498,
700
+ "internvl_second": 0.8625568002462387,
701
+ "kl_0": 0.00562286376953125,
702
+ "kl_1": 0.161407470703125,
703
+ "kl_2": 0.068145751953125,
704
+ "learning_rate": 1.8000000000000001e-06,
705
+ "loss": 0.0022,
706
+ "rewards/selfcheck": 0.8046875,
707
+ "second_gen": 0.737279087305069,
708
+ "second_gen_std": 0.15692505612969398,
709
+ "step": 25,
710
+ "tot_reward": 2.417116194963455
711
+ },
712
+ {
713
+ "completion_length_0": 576.0,
714
+ "completion_length_1": 62.65625,
715
+ "completion_length_2": 576.0,
716
+ "compre": 0.7978219985961914,
717
+ "compre_std": 0.17816441133618355,
718
+ "epoch": 0.27225130890052357,
719
+ "finalgen": 0.7744689732789993,
720
+ "first_gen": 0.7429674714803696,
721
+ "first_gen_std": 0.16385456919670105,
722
+ "geneval_final": 0.4270826429128647,
723
+ "geneval_first": 0.40624966472387314,
724
+ "geneval_second": 0.4010413736104965,
725
+ "grad_norm": 0.7011401057243347,
726
+ "internvl_final": 0.8419179618358612,
727
+ "internvl_first": 0.8042358160018921,
728
+ "internvl_second": 0.7991660982370377,
729
+ "kl_0": 0.0066375732421875,
730
+ "kl_1": 0.021697998046875,
731
+ "kl_2": 0.07000732421875,
732
+ "learning_rate": 1.64e-06,
733
+ "loss": 0.0022,
734
+ "rewards/selfcheck": 0.76953125,
735
+ "second_gen": 0.7295635938644409,
736
+ "second_gen_std": 0.17908411100506783,
737
+ "step": 26,
738
+ "tot_reward": 2.270353063941002
739
+ },
740
+ {
741
+ "completion_length_0": 576.0,
742
+ "completion_length_1": 52.21875,
743
+ "completion_length_2": 576.0,
744
+ "compre": 0.8099090754985809,
745
+ "compre_std": 0.174162395298481,
746
+ "epoch": 0.28272251308900526,
747
+ "finalgen": 0.7575195133686066,
748
+ "first_gen": 0.7415773570537567,
749
+ "first_gen_std": 0.1676672399044037,
750
+ "geneval_final": 0.7812484055757523,
751
+ "geneval_first": 0.7718735635280609,
752
+ "geneval_second": 0.7218743413686752,
753
+ "grad_norm": 0.8027878999710083,
754
+ "internvl_final": 0.8015442937612534,
755
+ "internvl_first": 0.7773027122020721,
756
+ "internvl_second": 0.7680691182613373,
757
+ "kl_0": 0.006519317626953125,
758
+ "kl_1": 0.00907135009765625,
759
+ "kl_2": 0.057464599609375,
760
+ "learning_rate": 1.4800000000000002e-06,
761
+ "loss": 0.0028,
762
+ "rewards/selfcheck": 0.7578125,
763
+ "second_gen": 0.7214943021535873,
764
+ "second_gen_std": 0.19232826307415962,
765
+ "step": 27,
766
+ "tot_reward": 2.272980734705925
767
+ },
768
+ {
769
+ "completion_length_0": 576.0,
770
+ "completion_length_1": 66.59375,
771
+ "completion_length_2": 576.0,
772
+ "compre": 0.806842252612114,
773
+ "compre_std": 0.19021093845367432,
774
+ "epoch": 0.2931937172774869,
775
+ "finalgen": 0.847057968378067,
776
+ "first_gen": 0.8385341465473175,
777
+ "first_gen_std": 0.16478701680898666,
778
+ "geneval_final": 0.859372615814209,
779
+ "geneval_first": 0.8567684590816498,
780
+ "geneval_second": 0.7317702323198318,
781
+ "grad_norm": 0.6873025298118591,
782
+ "internvl_final": 0.8440103977918625,
783
+ "internvl_first": 0.8305246829986572,
784
+ "internvl_second": 0.8086360841989517,
785
+ "kl_0": 0.00698089599609375,
786
+ "kl_1": 0.014835357666015625,
787
+ "kl_2": 0.06842041015625,
788
+ "learning_rate": 1.32e-06,
789
+ "loss": 0.0019,
790
+ "rewards/selfcheck": 0.859375,
791
+ "second_gen": 0.7720397561788559,
792
+ "second_gen_std": 0.1886446215212345,
793
+ "step": 28,
794
+ "tot_reward": 2.4174161553382874
795
+ },
796
+ {
797
+ "completion_length_0": 576.0,
798
+ "completion_length_1": 53.53125,
799
+ "completion_length_2": 576.0,
800
+ "compre": 0.8857230991125107,
801
+ "compre_std": 0.10681997612118721,
802
+ "epoch": 0.3036649214659686,
803
+ "finalgen": 0.7842017859220505,
804
+ "first_gen": 0.7742256820201874,
805
+ "first_gen_std": 0.09851318411529064,
806
+ "geneval_final": 0.6927078068256378,
807
+ "geneval_first": 0.6718747466802597,
808
+ "geneval_second": 0.6770829856395721,
809
+ "grad_norm": 0.8448681235313416,
810
+ "internvl_final": 0.8277648985385895,
811
+ "internvl_first": 0.823953315615654,
812
+ "internvl_second": 0.7563973963260651,
813
+ "kl_0": 0.00862884521484375,
814
+ "kl_1": 0.043426513671875,
815
+ "kl_2": 0.080963134765625,
816
+ "learning_rate": 1.1600000000000001e-06,
817
+ "loss": 0.0026,
818
+ "rewards/selfcheck": 0.7890625,
819
+ "second_gen": 0.7272432595491409,
820
+ "second_gen_std": 0.1504376232624054,
821
+ "step": 29,
822
+ "tot_reward": 2.387192040681839
823
+ },
824
+ {
825
+ "completion_length_0": 576.0,
826
+ "completion_length_1": 62.21875,
827
+ "completion_length_2": 576.0,
828
+ "compre": 0.8224151283502579,
829
+ "compre_std": 0.1981170941144228,
830
+ "epoch": 0.31413612565445026,
831
+ "finalgen": 0.8049910217523575,
832
+ "first_gen": 0.7610630840063095,
833
+ "first_gen_std": 0.21358660981059074,
834
+ "geneval_final": 0.7473958879709244,
835
+ "geneval_first": 0.7005215138196945,
836
+ "geneval_second": 0.6614595353603363,
837
+ "grad_norm": 0.8343987464904785,
838
+ "internvl_final": 0.8584465086460114,
839
+ "internvl_first": 0.8124430626630783,
840
+ "internvl_second": 0.825078934431076,
841
+ "kl_0": 0.008777618408203125,
842
+ "kl_1": 0.09954833984375,
843
+ "kl_2": 0.03533935546875,
844
+ "learning_rate": 1e-06,
845
+ "loss": 0.0016,
846
+ "rewards/selfcheck": 0.7421875,
847
+ "second_gen": 0.7491757720708847,
848
+ "second_gen_std": 0.1921403631567955,
849
+ "step": 30,
850
+ "tot_reward": 2.332653984427452
851
+ },
852
+ {
853
+ "completion_length_0": 576.0,
854
+ "completion_length_1": 44.71875,
855
+ "completion_length_2": 576.0,
856
+ "compre": 0.8410721868276596,
857
+ "compre_std": 0.13702194951474667,
858
+ "epoch": 0.32460732984293195,
859
+ "finalgen": 0.7979970574378967,
860
+ "first_gen": 0.7568379044532776,
861
+ "first_gen_std": 0.15108785405755043,
862
+ "geneval_final": 0.747393012046814,
863
+ "geneval_first": 0.6536446660757065,
864
+ "geneval_second": 0.7421848177909851,
865
+ "grad_norm": 0.6864271759986877,
866
+ "internvl_final": 0.846704438328743,
867
+ "internvl_first": 0.8158496171236038,
868
+ "internvl_second": 0.7937302589416504,
869
+ "kl_0": 0.0094451904296875,
870
+ "kl_1": 0.01679229736328125,
871
+ "kl_2": 0.0709228515625,
872
+ "learning_rate": 9.998996832101372e-07,
873
+ "loss": 0.0021,
874
+ "rewards/selfcheck": 0.796875,
875
+ "second_gen": 0.7617728412151337,
876
+ "second_gen_std": 0.17904167994856834,
877
+ "step": 31,
878
+ "tot_reward": 2.359682932496071
879
+ },
880
+ {
881
+ "completion_length_0": 576.0,
882
+ "completion_length_1": 50.53125,
883
+ "completion_length_2": 576.0,
884
+ "compre": 0.8558274954557419,
885
+ "compre_std": 0.17379233241081238,
886
+ "epoch": 0.33507853403141363,
887
+ "finalgen": 0.7417566031217575,
888
+ "first_gen": 0.6925482824444771,
889
+ "first_gen_std": 0.18836861476302147,
890
+ "geneval_final": 0.6807302311062813,
891
+ "geneval_first": 0.6281267628073692,
892
+ "geneval_second": 0.5859397873282433,
893
+ "grad_norm": 0.6784985661506653,
894
+ "internvl_final": 0.818776473402977,
895
+ "internvl_first": 0.7752449959516525,
896
+ "internvl_second": 0.8047575652599335,
897
+ "kl_0": 0.0068511962890625,
898
+ "kl_1": 0.0993804931640625,
899
+ "kl_2": 0.06658935546875,
900
+ "learning_rate": 9.997993664202746e-07,
901
+ "loss": 0.0015,
902
+ "rewards/selfcheck": 0.75,
903
+ "second_gen": 0.6777446419000626,
904
+ "second_gen_std": 0.20427290350198746,
905
+ "step": 32,
906
+ "tot_reward": 2.2261204198002815
907
+ },
908
+ {
909
+ "completion_length_0": 576.0,
910
+ "completion_length_1": 59.84375,
911
+ "completion_length_2": 576.0,
912
+ "compre": 0.7981031835079193,
913
+ "compre_std": 0.1447254717350006,
914
+ "epoch": 0.34554973821989526,
915
+ "finalgen": 0.7495086044073105,
916
+ "first_gen": 0.7078296095132828,
917
+ "first_gen_std": 0.15199239365756512,
918
+ "geneval_final": 0.7072916179895401,
919
+ "geneval_first": 0.61770960688591,
920
+ "geneval_second": 0.6052097231149673,
921
+ "grad_norm": 1.1254621744155884,
922
+ "internvl_final": 0.7729150354862213,
923
+ "internvl_first": 0.7561691552400589,
924
+ "internvl_second": 0.7697331458330154,
925
+ "kl_0": 0.0098876953125,
926
+ "kl_1": 0.06658172607421875,
927
+ "kl_2": 0.0515594482421875,
928
+ "learning_rate": 9.996990496304119e-07,
929
+ "loss": 0.0018,
930
+ "rewards/selfcheck": 0.77734375,
931
+ "second_gen": 0.7034687846899033,
932
+ "second_gen_std": 0.20352383330464363,
933
+ "step": 33,
934
+ "tot_reward": 2.2094015777111053
935
+ },
936
+ {
937
+ "completion_length_0": 576.0,
938
+ "completion_length_1": 44.3125,
939
+ "completion_length_2": 576.0,
940
+ "compre": 0.8364836275577545,
941
+ "compre_std": 0.13604356348514557,
942
+ "epoch": 0.35602094240837695,
943
+ "finalgen": 0.8379629701375961,
944
+ "first_gen": 0.8319551795721054,
945
+ "first_gen_std": 0.13666080683469772,
946
+ "geneval_final": 0.7968707084655762,
947
+ "geneval_first": 0.7812459617853165,
948
+ "geneval_second": 0.5781252458691597,
949
+ "grad_norm": 0.7418293952941895,
950
+ "internvl_final": 0.8514181822538376,
951
+ "internvl_first": 0.8490835875272751,
952
+ "internvl_second": 0.8219228535890579,
953
+ "kl_0": 0.008148193359375,
954
+ "kl_1": 0.0732421875,
955
+ "kl_2": 0.0586700439453125,
956
+ "learning_rate": 9.99598732840549e-07,
957
+ "loss": 0.0017,
958
+ "rewards/selfcheck": 0.92578125,
959
+ "second_gen": 0.7799781709909439,
960
+ "second_gen_std": 0.16469088196754456,
961
+ "step": 34,
962
+ "tot_reward": 2.448416978120804
963
+ },
964
+ {
965
+ "completion_length_0": 576.0,
966
+ "completion_length_1": 51.0,
967
+ "completion_length_2": 576.0,
968
+ "compre": 0.8061879575252533,
969
+ "compre_std": 0.1319187581539154,
970
+ "epoch": 0.36649214659685864,
971
+ "finalgen": 0.8501281291246414,
972
+ "first_gen": 0.8229310363531113,
973
+ "first_gen_std": 0.11965293064713478,
974
+ "geneval_final": 0.7499970346689224,
975
+ "geneval_first": 0.7083309888839722,
976
+ "geneval_second": 0.6458321213722229,
977
+ "grad_norm": 0.6788915991783142,
978
+ "internvl_final": 0.8535157740116119,
979
+ "internvl_first": 0.8384850025177002,
980
+ "internvl_second": 0.8233160376548767,
981
+ "kl_0": 0.017181396484375,
982
+ "kl_1": 0.0913848876953125,
983
+ "kl_2": 0.07708740234375,
984
+ "learning_rate": 9.994984160506863e-07,
985
+ "loss": 0.0019,
986
+ "rewards/selfcheck": 0.7578125,
987
+ "second_gen": 0.7993690818548203,
988
+ "second_gen_std": 0.1662791483104229,
989
+ "step": 35,
990
+ "tot_reward": 2.428488075733185
991
+ },
992
+ {
993
+ "completion_length_0": 576.0,
994
+ "completion_length_1": 48.71875,
995
+ "completion_length_2": 576.0,
996
+ "compre": 0.8119166195392609,
997
+ "compre_std": 0.16001782193779945,
998
+ "epoch": 0.3769633507853403,
999
+ "finalgen": 0.7916422188282013,
1000
+ "first_gen": 0.7797348350286484,
1001
+ "first_gen_std": 0.12765273917466402,
1002
+ "geneval_final": 0.5937489829957485,
1003
+ "geneval_first": 0.5677079077072449,
1004
+ "geneval_second": 0.5937490127980709,
1005
+ "grad_norm": 0.9368501901626587,
1006
+ "internvl_final": 0.8316355049610138,
1007
+ "internvl_first": 0.820400059223175,
1008
+ "internvl_second": 0.7844327837228775,
1009
+ "kl_0": 0.010711669921875,
1010
+ "kl_1": 0.195526123046875,
1011
+ "kl_2": 0.07763671875,
1012
+ "learning_rate": 9.993980992608236e-07,
1013
+ "loss": 0.0024,
1014
+ "rewards/selfcheck": 0.8515625,
1015
+ "second_gen": 0.7547754347324371,
1016
+ "second_gen_std": 0.16838618367910385,
1017
+ "step": 36,
1018
+ "tot_reward": 2.3464268893003464
1019
+ },
1020
+ {
1021
+ "completion_length_0": 576.0,
1022
+ "completion_length_1": 53.625,
1023
+ "completion_length_2": 576.0,
1024
+ "compre": 0.8024957925081253,
1025
+ "compre_std": 0.18278253450989723,
1026
+ "epoch": 0.387434554973822,
1027
+ "finalgen": 0.7481552511453629,
1028
+ "first_gen": 0.7225710302591324,
1029
+ "first_gen_std": 0.1874546892940998,
1030
+ "geneval_final": 0.5546881854534149,
1031
+ "geneval_first": 0.5468755140900612,
1032
+ "geneval_second": 0.4765642583370209,
1033
+ "grad_norm": 0.8567041158676147,
1034
+ "internvl_final": 0.8841516524553299,
1035
+ "internvl_first": 0.8685643374919891,
1036
+ "internvl_second": 0.8400794863700867,
1037
+ "kl_0": 0.00566864013671875,
1038
+ "kl_1": 0.02951812744140625,
1039
+ "kl_2": 0.0916748046875,
1040
+ "learning_rate": 9.99297782470961e-07,
1041
+ "loss": 0.0018,
1042
+ "rewards/selfcheck": 0.7578125,
1043
+ "second_gen": 0.6853782534599304,
1044
+ "second_gen_std": 0.1954638883471489,
1045
+ "step": 37,
1046
+ "tot_reward": 2.210445076227188
1047
+ },
1048
+ {
1049
+ "completion_length_0": 576.0,
1050
+ "completion_length_1": 40.59375,
1051
+ "completion_length_2": 576.0,
1052
+ "compre": 0.7131433486938477,
1053
+ "compre_std": 0.19255046173930168,
1054
+ "epoch": 0.39790575916230364,
1055
+ "finalgen": 0.7346409112215042,
1056
+ "first_gen": 0.7273536771535873,
1057
+ "first_gen_std": 0.1450207568705082,
1058
+ "geneval_final": 0.6406233012676239,
1059
+ "geneval_first": 0.6927057802677155,
1060
+ "geneval_second": 0.453126210719347,
1061
+ "grad_norm": 0.6762747764587402,
1062
+ "internvl_final": 0.7879036962985992,
1063
+ "internvl_first": 0.7670356035232544,
1064
+ "internvl_second": 0.7651602327823639,
1065
+ "kl_0": 0.01068878173828125,
1066
+ "kl_1": 0.01580810546875,
1067
+ "kl_2": 0.060943603515625,
1068
+ "learning_rate": 9.991974656810983e-07,
1069
+ "loss": 0.002,
1070
+ "rewards/selfcheck": 0.6875,
1071
+ "second_gen": 0.6624156683683395,
1072
+ "second_gen_std": 0.18323800340294838,
1073
+ "step": 38,
1074
+ "tot_reward": 2.1029126942157745
1075
+ },
1076
+ {
1077
+ "completion_length_0": 576.0,
1078
+ "completion_length_1": 51.65625,
1079
+ "completion_length_2": 576.0,
1080
+ "compre": 0.8225916773080826,
1081
+ "compre_std": 0.09966968512162566,
1082
+ "epoch": 0.4083769633507853,
1083
+ "finalgen": 0.8234386146068573,
1084
+ "first_gen": 0.7953141033649445,
1085
+ "first_gen_std": 0.09945447416976094,
1086
+ "geneval_final": 0.9166632145643234,
1087
+ "geneval_first": 0.885413646697998,
1088
+ "geneval_second": 0.8124980479478836,
1089
+ "grad_norm": 0.7397149801254272,
1090
+ "internvl_final": 0.7746958285570145,
1091
+ "internvl_first": 0.7505313530564308,
1092
+ "internvl_second": 0.7618338316679001,
1093
+ "kl_0": 0.0141448974609375,
1094
+ "kl_1": 0.054218292236328125,
1095
+ "kl_2": 0.0968017578125,
1096
+ "learning_rate": 9.990971488912355e-07,
1097
+ "loss": 0.0016,
1098
+ "rewards/selfcheck": 0.74609375,
1099
+ "second_gen": 0.7801451981067657,
1100
+ "second_gen_std": 0.18415237590670586,
1101
+ "step": 39,
1102
+ "tot_reward": 2.398050978779793
1103
+ },
1104
+ {
1105
+ "completion_length_0": 576.0,
1106
+ "completion_length_1": 52.8125,
1107
+ "completion_length_2": 576.0,
1108
+ "compre": 0.8490500450134277,
1109
+ "compre_std": 0.14167104102671146,
1110
+ "epoch": 0.418848167539267,
1111
+ "finalgen": 0.7479293346405029,
1112
+ "first_gen": 0.6769865304231644,
1113
+ "first_gen_std": 0.16864520870149136,
1114
+ "geneval_final": 0.6250012814998627,
1115
+ "geneval_first": 0.5125028006732464,
1116
+ "geneval_second": 0.589585080742836,
1117
+ "grad_norm": 0.9414799213409424,
1118
+ "internvl_final": 0.8500373214483261,
1119
+ "internvl_first": 0.8148254454135895,
1120
+ "internvl_second": 0.7945325374603271,
1121
+ "kl_0": 0.01337432861328125,
1122
+ "kl_1": 0.018688201904296875,
1123
+ "kl_2": 0.084716796875,
1124
+ "learning_rate": 9.989968321013727e-07,
1125
+ "loss": 0.0015,
1126
+ "rewards/selfcheck": 0.69921875,
1127
+ "second_gen": 0.6964385211467743,
1128
+ "second_gen_std": 0.19724002853035927,
1129
+ "step": 40,
1130
+ "tot_reward": 2.2224750965833664
1131
+ },
1132
+ {
1133
+ "completion_length_0": 576.0,
1134
+ "completion_length_1": 52.0625,
1135
+ "completion_length_2": 576.0,
1136
+ "compre": 0.7664348036050797,
1137
+ "compre_std": 0.18268020451068878,
1138
+ "epoch": 0.4293193717277487,
1139
+ "finalgen": 0.7703444063663483,
1140
+ "first_gen": 0.7305110096931458,
1141
+ "first_gen_std": 0.1900181919336319,
1142
+ "geneval_final": 0.6979156285524368,
1143
+ "geneval_first": 0.6041674762964249,
1144
+ "geneval_second": 0.47396068274974823,
1145
+ "grad_norm": 0.8097413778305054,
1146
+ "internvl_final": 0.8117622584104538,
1147
+ "internvl_first": 0.77989761531353,
1148
+ "internvl_second": 0.7899433225393295,
1149
+ "kl_0": 0.01154327392578125,
1150
+ "kl_1": 0.02215576171875,
1151
+ "kl_2": 0.08843994140625,
1152
+ "learning_rate": 9.9889651531151e-07,
1153
+ "loss": 0.002,
1154
+ "rewards/selfcheck": 0.6796875,
1155
+ "second_gen": 0.6740073412656784,
1156
+ "second_gen_std": 0.2109435349702835,
1157
+ "step": 41,
1158
+ "tot_reward": 2.170953154563904
1159
+ },
1160
+ {
1161
+ "completion_length_0": 576.0,
1162
+ "completion_length_1": 51.65625,
1163
+ "completion_length_2": 576.0,
1164
+ "compre": 0.7402557134628296,
1165
+ "compre_std": 0.1825437918305397,
1166
+ "epoch": 0.4397905759162304,
1167
+ "finalgen": 0.7364934235811234,
1168
+ "first_gen": 0.716862678527832,
1169
+ "first_gen_std": 0.159286230802536,
1170
+ "geneval_final": 0.7604135125875473,
1171
+ "geneval_first": 0.7322889342904091,
1172
+ "geneval_second": 0.6989560350775719,
1173
+ "grad_norm": 0.8425426483154297,
1174
+ "internvl_final": 0.7959964275360107,
1175
+ "internvl_first": 0.7877412140369415,
1176
+ "internvl_second": 0.7823043167591095,
1177
+ "kl_0": 0.009307861328125,
1178
+ "kl_1": 0.07275390625,
1179
+ "kl_2": 0.05108642578125,
1180
+ "learning_rate": 9.987961985216472e-07,
1181
+ "loss": 0.0017,
1182
+ "rewards/selfcheck": 0.625,
1183
+ "second_gen": 0.7019034326076508,
1184
+ "second_gen_std": 0.15979842096567154,
1185
+ "step": 42,
1186
+ "tot_reward": 2.1590218245983124
1187
+ },
1188
+ {
1189
+ "completion_length_0": 576.0,
1190
+ "completion_length_1": 54.96875,
1191
+ "completion_length_2": 576.0,
1192
+ "compre": 0.832016795873642,
1193
+ "compre_std": 0.14732321724295616,
1194
+ "epoch": 0.450261780104712,
1195
+ "finalgen": 0.7879950702190399,
1196
+ "first_gen": 0.766228049993515,
1197
+ "first_gen_std": 0.14289268106222153,
1198
+ "geneval_final": 0.8234342187643051,
1199
+ "geneval_first": 0.7953095883131027,
1200
+ "geneval_second": 0.7864555567502975,
1201
+ "grad_norm": 0.708795964717865,
1202
+ "internvl_final": 0.8033061027526855,
1203
+ "internvl_first": 0.7949481457471848,
1204
+ "internvl_second": 0.7645547389984131,
1205
+ "kl_0": 0.00658416748046875,
1206
+ "kl_1": 0.047210693359375,
1207
+ "kl_2": 0.0685577392578125,
1208
+ "learning_rate": 9.986958817317846e-07,
1209
+ "loss": 0.0016,
1210
+ "rewards/selfcheck": 0.765625,
1211
+ "second_gen": 0.7446198761463165,
1212
+ "second_gen_std": 0.16749507747590542,
1213
+ "step": 43,
1214
+ "tot_reward": 2.3428647220134735
1215
+ },
1216
+ {
1217
+ "completion_length_0": 576.0,
1218
+ "completion_length_1": 35.59375,
1219
+ "completion_length_2": 576.0,
1220
+ "compre": 0.8220645636320114,
1221
+ "compre_std": 0.18921106681227684,
1222
+ "epoch": 0.4607329842931937,
1223
+ "finalgen": 0.8380867093801498,
1224
+ "first_gen": 0.8182298690080643,
1225
+ "first_gen_std": 0.1516233654692769,
1226
+ "geneval_final": 0.8619765937328339,
1227
+ "geneval_first": 0.8333311378955841,
1228
+ "geneval_second": 0.7604154795408249,
1229
+ "grad_norm": 0.8835089206695557,
1230
+ "internvl_final": 0.8661111295223236,
1231
+ "internvl_first": 0.8650543540716171,
1232
+ "internvl_second": 0.8211274743080139,
1233
+ "kl_0": 0.0194091796875,
1234
+ "kl_1": 0.0438995361328125,
1235
+ "kl_2": 0.09698486328125,
1236
+ "learning_rate": 9.985955649419219e-07,
1237
+ "loss": 0.0023,
1238
+ "rewards/selfcheck": 0.796875,
1239
+ "second_gen": 0.7649048268795013,
1240
+ "second_gen_std": 0.20057369023561478,
1241
+ "step": 44,
1242
+ "tot_reward": 2.405199259519577
1243
+ },
1244
+ {
1245
+ "completion_length_0": 576.0,
1246
+ "completion_length_1": 37.4375,
1247
+ "completion_length_2": 576.0,
1248
+ "compre": 0.7776415646076202,
1249
+ "compre_std": 0.16899557411670685,
1250
+ "epoch": 0.4712041884816754,
1251
+ "finalgen": 0.7281599640846252,
1252
+ "first_gen": 0.7270458787679672,
1253
+ "first_gen_std": 0.15472668409347534,
1254
+ "geneval_final": 0.6536467671394348,
1255
+ "geneval_first": 0.6666674166917801,
1256
+ "geneval_second": 0.48698242753744125,
1257
+ "grad_norm": 0.9925355911254883,
1258
+ "internvl_final": 0.7626287937164307,
1259
+ "internvl_first": 0.7545899301767349,
1260
+ "internvl_second": 0.7231644093990326,
1261
+ "kl_0": 0.0115509033203125,
1262
+ "kl_1": 0.087677001953125,
1263
+ "kl_2": 0.0679931640625,
1264
+ "learning_rate": 9.984952481520591e-07,
1265
+ "loss": 0.002,
1266
+ "rewards/selfcheck": 0.6953125,
1267
+ "second_gen": 0.6345587372779846,
1268
+ "second_gen_std": 0.18290312960743904,
1269
+ "step": 45,
1270
+ "tot_reward": 2.139246180653572
1271
+ },
1272
+ {
1273
+ "completion_length_0": 576.0,
1274
+ "completion_length_1": 64.84375,
1275
+ "completion_length_2": 576.0,
1276
+ "compre": 0.7952178418636322,
1277
+ "compre_std": 0.14555131271481514,
1278
+ "epoch": 0.4816753926701571,
1279
+ "finalgen": 0.7756165713071823,
1280
+ "first_gen": 0.7566163092851639,
1281
+ "first_gen_std": 0.12462454661726952,
1282
+ "geneval_final": 0.5625010244548321,
1283
+ "geneval_first": 0.505210455805809,
1284
+ "geneval_second": 0.5312514565885067,
1285
+ "grad_norm": 0.7020373344421387,
1286
+ "internvl_final": 0.8309992402791977,
1287
+ "internvl_first": 0.8211416453123093,
1288
+ "internvl_second": 0.8066484332084656,
1289
+ "kl_0": 0.01102447509765625,
1290
+ "kl_1": 0.05503082275390625,
1291
+ "kl_2": 0.0787353515625,
1292
+ "learning_rate": 9.983949313621963e-07,
1293
+ "loss": 0.0018,
1294
+ "rewards/selfcheck": 0.7890625,
1295
+ "second_gen": 0.7443623840808868,
1296
+ "second_gen_std": 0.15503870509564877,
1297
+ "step": 46,
1298
+ "tot_reward": 2.296196535229683
1299
+ },
1300
+ {
1301
+ "completion_length_0": 576.0,
1302
+ "completion_length_1": 59.375,
1303
+ "completion_length_2": 576.0,
1304
+ "compre": 0.8056609481573105,
1305
+ "compre_std": 0.15513703413307667,
1306
+ "epoch": 0.49214659685863876,
1307
+ "finalgen": 0.773963987827301,
1308
+ "first_gen": 0.7415499240159988,
1309
+ "first_gen_std": 0.16437550447881222,
1310
+ "geneval_final": 0.7083333283662796,
1311
+ "geneval_first": 0.6041681617498398,
1312
+ "geneval_second": 0.5729185864329338,
1313
+ "grad_norm": 0.9495850205421448,
1314
+ "internvl_final": 0.8133386075496674,
1315
+ "internvl_first": 0.8239754736423492,
1316
+ "internvl_second": 0.7794905602931976,
1317
+ "kl_0": 0.010894775390625,
1318
+ "kl_1": 0.01897430419921875,
1319
+ "kl_2": 0.09686279296875,
1320
+ "learning_rate": 9.982946145723336e-07,
1321
+ "loss": 0.0018,
1322
+ "rewards/selfcheck": 0.76171875,
1323
+ "second_gen": 0.7020281553268433,
1324
+ "second_gen_std": 0.2232751026749611,
1325
+ "step": 47,
1326
+ "tot_reward": 2.2492390275001526
1327
+ },
1328
+ {
1329
+ "completion_length_0": 576.0,
1330
+ "completion_length_1": 41.6875,
1331
+ "completion_length_2": 576.0,
1332
+ "compre": 0.7868409007787704,
1333
+ "compre_std": 0.1367402784526348,
1334
+ "epoch": 0.5026178010471204,
1335
+ "finalgen": 0.7759291082620621,
1336
+ "first_gen": 0.8014131784439087,
1337
+ "first_gen_std": 0.1358846165239811,
1338
+ "geneval_final": 0.7031254507601261,
1339
+ "geneval_first": 0.7968740910291672,
1340
+ "geneval_second": 0.645834494382143,
1341
+ "grad_norm": 0.7847272157669067,
1342
+ "internvl_final": 0.847522646188736,
1343
+ "internvl_first": 0.8382976651191711,
1344
+ "internvl_second": 0.8017169833183289,
1345
+ "kl_0": 0.011627197265625,
1346
+ "kl_1": 0.018829345703125,
1347
+ "kl_2": 0.07354736328125,
1348
+ "learning_rate": 9.98194297782471e-07,
1349
+ "loss": 0.0018,
1350
+ "rewards/selfcheck": 0.69921875,
1351
+ "second_gen": 0.7293520718812943,
1352
+ "second_gen_std": 0.1890699453651905,
1353
+ "step": 48,
1354
+ "tot_reward": 2.3176061511039734
1355
+ },
1356
+ {
1357
+ "completion_length_0": 576.0,
1358
+ "completion_length_1": 44.8125,
1359
+ "completion_length_2": 576.0,
1360
+ "compre": 0.8000631183385849,
1361
+ "compre_std": 0.1882825717329979,
1362
+ "epoch": 0.5130890052356021,
1363
+ "finalgen": 0.7556480169296265,
1364
+ "first_gen": 0.7593663930892944,
1365
+ "first_gen_std": 0.1389727033674717,
1366
+ "geneval_final": 0.6812484711408615,
1367
+ "geneval_first": 0.6895816922187805,
1368
+ "geneval_second": 0.618749350309372,
1369
+ "grad_norm": 1.2835190296173096,
1370
+ "internvl_final": 0.8502883911132812,
1371
+ "internvl_first": 0.8563270419836044,
1372
+ "internvl_second": 0.8191845417022705,
1373
+ "kl_0": 0.0147552490234375,
1374
+ "kl_1": 0.021228790283203125,
1375
+ "kl_2": 0.088623046875,
1376
+ "learning_rate": 9.980939809926083e-07,
1377
+ "loss": 0.0018,
1378
+ "rewards/selfcheck": 0.7421875,
1379
+ "second_gen": 0.714243933558464,
1380
+ "second_gen_std": 0.18192566186189651,
1381
+ "step": 49,
1382
+ "tot_reward": 2.2736734449863434
1383
+ },
1384
+ {
1385
+ "completion_length_0": 576.0,
1386
+ "completion_length_1": 55.125,
1387
+ "completion_length_2": 576.0,
1388
+ "compre": 0.8787847608327866,
1389
+ "compre_std": 0.1345262099057436,
1390
+ "epoch": 0.5235602094240838,
1391
+ "finalgen": 0.8423563987016678,
1392
+ "first_gen": 0.8243885934352875,
1393
+ "first_gen_std": 0.14151314459741116,
1394
+ "geneval_final": 0.8437488526105881,
1395
+ "geneval_first": 0.8124992549419403,
1396
+ "geneval_second": 0.7552084177732468,
1397
+ "grad_norm": 0.9143640995025635,
1398
+ "internvl_final": 0.8766902834177017,
1399
+ "internvl_first": 0.8708599954843521,
1400
+ "internvl_second": 0.8306637108325958,
1401
+ "kl_0": 0.01788330078125,
1402
+ "kl_1": 0.01178741455078125,
1403
+ "kl_2": 0.13427734375,
1404
+ "learning_rate": 9.979936642027455e-07,
1405
+ "loss": 0.0017,
1406
+ "rewards/selfcheck": 0.8359375,
1407
+ "second_gen": 0.7776670157909393,
1408
+ "second_gen_std": 0.1923159956932068,
1409
+ "step": 50,
1410
+ "tot_reward": 2.4808403700590134
1411
+ }
1412
+ ],
1413
+ "logging_steps": 1,
1414
+ "max_steps": 9500,
1415
+ "num_input_tokens_seen": 0,
1416
+ "num_train_epochs": 100,
1417
+ "save_steps": 25,
1418
+ "stateful_callbacks": {
1419
+ "TrainerControl": {
1420
+ "args": {
1421
+ "should_epoch_stop": false,
1422
+ "should_evaluate": false,
1423
+ "should_log": false,
1424
+ "should_save": true,
1425
+ "should_training_stop": false
1426
+ },
1427
+ "attributes": {}
1428
+ }
1429
+ },
1430
+ "total_flos": 0.0,
1431
+ "train_batch_size": 1,
1432
+ "trial_name": null,
1433
+ "trial_params": null
1434
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:caeb81b431498fc553411b77c5bac0f238a7cb96a6ffc3f09db9f99a3378b860
3
+ size 8824