Upload LoRA adapter from dpo_40k_abla_one_cat_one

.gitattributes

@@ -33,3 +33,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-100/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-200/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-300/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-400/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-600/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-621/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,88 @@
+---
+library_name: peft
+license: other
+base_model: Qwen/Qwen2.5-VL-7B-Instruct
+tags:
+- base_model:adapter:/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct
+- llama-factory
+- lora
+- transformers
+pipeline_tag: text-generation
+model-index:
+- name: dpo_40k_abla_one_cat_one
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# dpo_40k_abla_one_cat_one
+
+This model is a fine-tuned version of [/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct](https://huggingface.co//p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct) on the dpo_ablation_one_cat_one dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.5072
+- Rewards/chosen: -0.4765
+- Rewards/rejected: -1.1458
+- Rewards/accuracies: 0.7700
+- Rewards/margins: 0.6692
+- Logps/chosen: -35.2662
+- Logps/rejected: -46.7822
+- Logits/chosen: 0.2949
+- Logits/rejected: 0.2973
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 2
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 4
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 64
+- total_eval_batch_size: 4
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 1.0
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/chosen | Logps/rejected | Logits/chosen | Logits/rejected |
+|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:------------:|:--------------:|:-------------:|:---------------:|
+| 0.6882        | 0.0804 | 50   | 0.6898          | -0.0044        | -0.0115          | 0.5350             | 0.0071          | -30.5444     | -35.4393       | 0.5502        | 0.5581          |
+| 0.6603        | 0.1608 | 100  | 0.6593          | -0.0806        | -0.1570          | 0.6850             | 0.0763          | -31.3070     | -36.8941       | 0.5384        | 0.5446          |
+| 0.6387        | 0.2412 | 150  | 0.6298          | -0.1917        | -0.3455          | 0.7250             | 0.1538          | -32.4175     | -38.7793       | 0.5058        | 0.5080          |
+| 0.5986        | 0.3216 | 200  | 0.5988          | -0.2330        | -0.4814          | 0.7050             | 0.2485          | -32.8302     | -40.1388       | 0.4847        | 0.4844          |
+| 0.5368        | 0.4020 | 250  | 0.5667          | -0.2959        | -0.6688          | 0.7200             | 0.3728          | -33.4601     | -42.0120       | 0.4258        | 0.4350          |
+| 0.5416        | 0.4824 | 300  | 0.5450          | -0.3299        | -0.8038          | 0.7450             | 0.4739          | -33.8000     | -43.3626       | 0.3828        | 0.3894          |
+| 0.5141        | 0.5628 | 350  | 0.5301          | -0.3794        | -0.9226          | 0.7450             | 0.5432          | -34.2943     | -44.5501       | 0.3541        | 0.3622          |
+| 0.5122        | 0.6432 | 400  | 0.5206          | -0.4136        | -1.0123          | 0.7550             | 0.5987          | -34.6362     | -45.4474       | 0.3284        | 0.3337          |
+| 0.4817        | 0.7236 | 450  | 0.5165          | -0.4476        | -1.0766          | 0.7750             | 0.6290          | -34.9764     | -46.0903       | 0.3096        | 0.3177          |
+| 0.4709        | 0.8040 | 500  | 0.5102          | -0.4623        | -1.1173          | 0.7800             | 0.6550          | -35.1233     | -46.4975       | 0.3006        | 0.3063          |
+| 0.4759        | 0.8844 | 550  | 0.5098          | -0.4751        | -1.1359          | 0.7800             | 0.6609          | -35.2515     | -46.6838       | 0.2987        | 0.3002          |
+| 0.4342        | 0.9648 | 600  | 0.5086          | -0.4804        | -1.1453          | 0.7800             | 0.6649          | -35.3051     | -46.7775       | 0.2947        | 0.2991          |
+
+
+### Framework versions
+
+- PEFT 0.17.1
+- Transformers 4.49.0
+- Pytorch 2.5.1+cu124
+- Datasets 4.0.0
+- Tokenizers 0.21.0

README.md.bak

@@ -0,0 +1,88 @@
+---
+library_name: peft
+license: other
+base_model: /p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct
+tags:
+- base_model:adapter:/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct
+- llama-factory
+- lora
+- transformers
+pipeline_tag: text-generation
+model-index:
+- name: dpo_40k_abla_one_cat_one
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# dpo_40k_abla_one_cat_one
+
+This model is a fine-tuned version of [/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct](https://huggingface.co//p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct) on the dpo_ablation_one_cat_one dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.5072
+- Rewards/chosen: -0.4765
+- Rewards/rejected: -1.1458
+- Rewards/accuracies: 0.7700
+- Rewards/margins: 0.6692
+- Logps/chosen: -35.2662
+- Logps/rejected: -46.7822
+- Logits/chosen: 0.2949
+- Logits/rejected: 0.2973
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 2
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 4
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 64
+- total_eval_batch_size: 4
+- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 1.0
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/chosen | Logps/rejected | Logits/chosen | Logits/rejected |
+|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:------------:|:--------------:|:-------------:|:---------------:|
+| 0.6882        | 0.0804 | 50   | 0.6898          | -0.0044        | -0.0115          | 0.5350             | 0.0071          | -30.5444     | -35.4393       | 0.5502        | 0.5581          |
+| 0.6603        | 0.1608 | 100  | 0.6593          | -0.0806        | -0.1570          | 0.6850             | 0.0763          | -31.3070     | -36.8941       | 0.5384        | 0.5446          |
+| 0.6387        | 0.2412 | 150  | 0.6298          | -0.1917        | -0.3455          | 0.7250             | 0.1538          | -32.4175     | -38.7793       | 0.5058        | 0.5080          |
+| 0.5986        | 0.3216 | 200  | 0.5988          | -0.2330        | -0.4814          | 0.7050             | 0.2485          | -32.8302     | -40.1388       | 0.4847        | 0.4844          |
+| 0.5368        | 0.4020 | 250  | 0.5667          | -0.2959        | -0.6688          | 0.7200             | 0.3728          | -33.4601     | -42.0120       | 0.4258        | 0.4350          |
+| 0.5416        | 0.4824 | 300  | 0.5450          | -0.3299        | -0.8038          | 0.7450             | 0.4739          | -33.8000     | -43.3626       | 0.3828        | 0.3894          |
+| 0.5141        | 0.5628 | 350  | 0.5301          | -0.3794        | -0.9226          | 0.7450             | 0.5432          | -34.2943     | -44.5501       | 0.3541        | 0.3622          |
+| 0.5122        | 0.6432 | 400  | 0.5206          | -0.4136        | -1.0123          | 0.7550             | 0.5987          | -34.6362     | -45.4474       | 0.3284        | 0.3337          |
+| 0.4817        | 0.7236 | 450  | 0.5165          | -0.4476        | -1.0766          | 0.7750             | 0.6290          | -34.9764     | -46.0903       | 0.3096        | 0.3177          |
+| 0.4709        | 0.8040 | 500  | 0.5102          | -0.4623        | -1.1173          | 0.7800             | 0.6550          | -35.1233     | -46.4975       | 0.3006        | 0.3063          |
+| 0.4759        | 0.8844 | 550  | 0.5098          | -0.4751        | -1.1359          | 0.7800             | 0.6609          | -35.2515     | -46.6838       | 0.2987        | 0.3002          |
+| 0.4342        | 0.9648 | 600  | 0.5086          | -0.4804        | -1.1453          | 0.7800             | 0.6649          | -35.3051     | -46.7775       | 0.2947        | 0.2991          |
+
+
+### Framework versions
+
+- PEFT 0.17.1
+- Transformers 4.49.0
+- Pytorch 2.5.1+cu124
+- Datasets 4.0.0
+- Tokenizers 0.21.0

adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25e363363cf61168751ff5e9f947016b10f167fd728689648f705ccd98039bbd
+size 40385208

added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

all_results.json

@@ -0,0 +1,20 @@
+{
+    "epoch": 0.9985929648241206,
+    "eval_logits/chosen": 0.2949379086494446,
+    "eval_logits/rejected": 0.2972874939441681,
+    "eval_logps/chosen": -35.26618194580078,
+    "eval_logps/rejected": -46.7822380065918,
+    "eval_loss": 0.5072444677352905,
+    "eval_rewards/accuracies": 0.7699999809265137,
+    "eval_rewards/chosen": -0.4765482246875763,
+    "eval_rewards/margins": 0.669231653213501,
+    "eval_rewards/rejected": -1.1457798480987549,
+    "eval_runtime": 48.1974,
+    "eval_samples_per_second": 4.15,
+    "eval_steps_per_second": 1.037,
+    "total_flos": 1.595144247000957e+18,
+    "train_loss": 0.5550001341363658,
+    "train_runtime": 15024.8122,
+    "train_samples_per_second": 2.649,
+    "train_steps_per_second": 0.041
+}

chat_template.json

@@ -0,0 +1,3 @@
+{
+  "chat_template": "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}"
+}

checkpoint-100/README.md

@@ -0,0 +1,208 @@
+---
+base_model: /p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct
+- llama-factory
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-100/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5eb092e07b0837ef2c3510e82ad21ec89eb026ab6c39f7a9e23c8392d893d45a
+size 40385208

checkpoint-100/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-100/chat_template.json

@@ -0,0 +1,3 @@
+{
+  "chat_template": "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}"
+}

checkpoint-100/preprocessor_config.json

@@ -0,0 +1,32 @@
+{
+  "crop_size": null,
+  "default_to_square": true,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_processor_type": "Qwen2VLImageProcessorFast",
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "max_pixels": 12845056,
+  "merge_size": 2,
+  "min_pixels": 3136,
+  "patch_size": 14,
+  "processor_class": "Qwen2_5_VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "longest_edge": 1003520,
+    "shortest_edge": 3136
+  },
+  "temporal_patch_size": 2
+}

checkpoint-100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:308f94f9a5c24e1bad5c393d56ae7af7782600f4e791d9c6ac35b22fff2105b6
+size 15024

checkpoint-100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b056f3c23cb32dc77a2ec9e7651e0b64e4440e21f0fdf969b86bfc56a1cbdf06
+size 15024

checkpoint-100/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f3f8a05714bc528f4885a2816181652f2303b3e8150f89b56aaee6bec56aa520
+size 15024

checkpoint-100/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f755bd3c330281961e5c03af9d10ce8c1e1678619d384f6f1fd5fd7dce2ff50
+size 15024

checkpoint-100/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-100/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

checkpoint-100/tokenizer_config.json

@@ -0,0 +1,210 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "processor_class": "Qwen2_5_VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-100/trainer_state.json

@@ -0,0 +1,815 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.16080402010050251,
+  "eval_steps": 50,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0032160804020100503,
+      "grad_norm": 0.398336261510849,
+      "learning_rate": 1.5873015873015874e-07,
+      "logits/chosen": 0.9754317998886108,
+      "logits/rejected": 1.027288556098938,
+      "logps/chosen": -31.835588455200195,
+      "logps/rejected": -34.915592193603516,
+      "loss": 0.6931,
+      "rewards/accuracies": 0.0,
+      "rewards/chosen": 0.0,
+      "rewards/margins": 0.0,
+      "rewards/rejected": 0.0,
+      "step": 2
+    },
+    {
+      "epoch": 0.006432160804020101,
+      "grad_norm": 0.4530814290046692,
+      "learning_rate": 3.174603174603175e-07,
+      "logits/chosen": 1.4162323474884033,
+      "logits/rejected": 1.4583587646484375,
+      "logps/chosen": -30.352081298828125,
+      "logps/rejected": -33.31166076660156,
+      "loss": 0.6938,
+      "rewards/accuracies": 0.5078125,
+      "rewards/chosen": -0.003089881967753172,
+      "rewards/margins": -0.0008391638984903693,
+      "rewards/rejected": -0.0022507174871861935,
+      "step": 4
+    },
+    {
+      "epoch": 0.009648241206030151,
+      "grad_norm": 0.46197977662086487,
+      "learning_rate": 4.7619047619047623e-07,
+      "logits/chosen": 1.2524590492248535,
+      "logits/rejected": 1.3010387420654297,
+      "logps/chosen": -29.417587280273438,
+      "logps/rejected": -33.4231071472168,
+      "loss": 0.6936,
+      "rewards/accuracies": 0.4609375,
+      "rewards/chosen": 0.0004551531164906919,
+      "rewards/margins": -0.00048057991079986095,
+      "rewards/rejected": 0.0009357327362522483,
+      "step": 6
+    },
+    {
+      "epoch": 0.012864321608040201,
+      "grad_norm": 0.4546354115009308,
+      "learning_rate": 6.34920634920635e-07,
+      "logits/chosen": 1.4723472595214844,
+      "logits/rejected": 1.5091776847839355,
+      "logps/chosen": -32.08555221557617,
+      "logps/rejected": -34.23310089111328,
+      "loss": 0.6917,
+      "rewards/accuracies": 0.484375,
+      "rewards/chosen": 0.004024952184408903,
+      "rewards/margins": 0.0033740843646228313,
+      "rewards/rejected": 0.0006508685764856637,
+      "step": 8
+    },
+    {
+      "epoch": 0.016080402010050253,
+      "grad_norm": 0.4574313759803772,
+      "learning_rate": 7.936507936507937e-07,
+      "logits/chosen": 1.278916597366333,
+      "logits/rejected": 1.3133082389831543,
+      "logps/chosen": -31.22161102294922,
+      "logps/rejected": -34.931907653808594,
+      "loss": 0.6929,
+      "rewards/accuracies": 0.4453125,
+      "rewards/chosen": 0.0015166704542934895,
+      "rewards/margins": 0.0008749825647100806,
+      "rewards/rejected": 0.0006416881224140525,
+      "step": 10
+    },
+    {
+      "epoch": 0.019296482412060303,
+      "grad_norm": 0.44448181986808777,
+      "learning_rate": 9.523809523809525e-07,
+      "logits/chosen": 1.3098528385162354,
+      "logits/rejected": 1.3612464666366577,
+      "logps/chosen": -30.1417179107666,
+      "logps/rejected": -35.877037048339844,
+      "loss": 0.6958,
+      "rewards/accuracies": 0.4296875,
+      "rewards/chosen": -0.00687691755592823,
+      "rewards/margins": -0.004914135672152042,
+      "rewards/rejected": -0.001962781883776188,
+      "step": 12
+    },
+    {
+      "epoch": 0.022512562814070353,
+      "grad_norm": 0.4568133056163788,
+      "learning_rate": 1.111111111111111e-06,
+      "logits/chosen": 1.4617085456848145,
+      "logits/rejected": 1.4705244302749634,
+      "logps/chosen": -30.95242691040039,
+      "logps/rejected": -36.34254455566406,
+      "loss": 0.6939,
+      "rewards/accuracies": 0.4921875,
+      "rewards/chosen": -0.0029178508557379246,
+      "rewards/margins": -0.0011518384562805295,
+      "rewards/rejected": -0.001766012515872717,
+      "step": 14
+    },
+    {
+      "epoch": 0.025728643216080403,
+      "grad_norm": 0.4778600037097931,
+      "learning_rate": 1.26984126984127e-06,
+      "logits/chosen": 1.3174049854278564,
+      "logits/rejected": 1.3094675540924072,
+      "logps/chosen": -28.937427520751953,
+      "logps/rejected": -36.366676330566406,
+      "loss": 0.6932,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.0009857461554929614,
+      "rewards/margins": 0.00013500778004527092,
+      "rewards/rejected": -0.001120753469876945,
+      "step": 16
+    },
+    {
+      "epoch": 0.028944723618090452,
+      "grad_norm": 0.421551913022995,
+      "learning_rate": 1.4285714285714286e-06,
+      "logits/chosen": 1.3308428525924683,
+      "logits/rejected": 1.4079976081848145,
+      "logps/chosen": -33.79199981689453,
+      "logps/rejected": -36.55875778198242,
+      "loss": 0.6941,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.002126973122358322,
+      "rewards/margins": -0.001552498433738947,
+      "rewards/rejected": -0.0005744747468270361,
+      "step": 18
+    },
+    {
+      "epoch": 0.032160804020100506,
+      "grad_norm": 0.4264390468597412,
+      "learning_rate": 1.5873015873015873e-06,
+      "logits/chosen": 1.4721007347106934,
+      "logits/rejected": 1.485215663909912,
+      "logps/chosen": -31.09832000732422,
+      "logps/rejected": -36.288822174072266,
+      "loss": 0.6951,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.005364666692912579,
+      "rewards/margins": -0.003565119579434395,
+      "rewards/rejected": -0.0017995464149862528,
+      "step": 20
+    },
+    {
+      "epoch": 0.03537688442211055,
+      "grad_norm": 0.40373197197914124,
+      "learning_rate": 1.746031746031746e-06,
+      "logits/chosen": 1.3501726388931274,
+      "logits/rejected": 1.370957374572754,
+      "logps/chosen": -31.461055755615234,
+      "logps/rejected": -34.38249588012695,
+      "loss": 0.6938,
+      "rewards/accuracies": 0.46875,
+      "rewards/chosen": -0.003749656490981579,
+      "rewards/margins": -0.0010161390528082848,
+      "rewards/rejected": -0.0027335172053426504,
+      "step": 22
+    },
+    {
+      "epoch": 0.038592964824120606,
+      "grad_norm": 0.4503360390663147,
+      "learning_rate": 1.904761904761905e-06,
+      "logits/chosen": 1.1474275588989258,
+      "logits/rejected": 1.136375904083252,
+      "logps/chosen": -32.1020393371582,
+      "logps/rejected": -35.39215087890625,
+      "loss": 0.695,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.0007312107481993735,
+      "rewards/margins": -0.003288470208644867,
+      "rewards/rejected": 0.002557259751483798,
+      "step": 24
+    },
+    {
+      "epoch": 0.04180904522613065,
+      "grad_norm": 0.39122092723846436,
+      "learning_rate": 2.0634920634920634e-06,
+      "logits/chosen": 1.0779863595962524,
+      "logits/rejected": 1.0993211269378662,
+      "logps/chosen": -31.845603942871094,
+      "logps/rejected": -35.04288864135742,
+      "loss": 0.6949,
+      "rewards/accuracies": 0.46875,
+      "rewards/chosen": -0.0012633285950869322,
+      "rewards/margins": -0.0032839984633028507,
+      "rewards/rejected": 0.002020669635385275,
+      "step": 26
+    },
+    {
+      "epoch": 0.045025125628140705,
+      "grad_norm": 0.4362376630306244,
+      "learning_rate": 2.222222222222222e-06,
+      "logits/chosen": 1.4343101978302002,
+      "logits/rejected": 1.4324826002120972,
+      "logps/chosen": -29.318994522094727,
+      "logps/rejected": -31.928974151611328,
+      "loss": 0.6941,
+      "rewards/accuracies": 0.53125,
+      "rewards/chosen": -0.00553980004042387,
+      "rewards/margins": -0.0015490761725232005,
+      "rewards/rejected": -0.003990724217146635,
+      "step": 28
+    },
+    {
+      "epoch": 0.04824120603015075,
+      "grad_norm": 0.4269290864467621,
+      "learning_rate": 2.380952380952381e-06,
+      "logits/chosen": 1.29887056350708,
+      "logits/rejected": 1.3346788883209229,
+      "logps/chosen": -32.30498123168945,
+      "logps/rejected": -35.8641357421875,
+      "loss": 0.694,
+      "rewards/accuracies": 0.46875,
+      "rewards/chosen": -0.00222906400449574,
+      "rewards/margins": -0.0013898832257837057,
+      "rewards/rejected": -0.0008391811279579997,
+      "step": 30
+    },
+    {
+      "epoch": 0.051457286432160805,
+      "grad_norm": 0.4749397337436676,
+      "learning_rate": 2.53968253968254e-06,
+      "logits/chosen": 1.2239577770233154,
+      "logits/rejected": 1.2360297441482544,
+      "logps/chosen": -30.756057739257812,
+      "logps/rejected": -35.22294616699219,
+      "loss": 0.6937,
+      "rewards/accuracies": 0.46875,
+      "rewards/chosen": 0.0008076138328760862,
+      "rewards/margins": -0.0006099215243011713,
+      "rewards/rejected": 0.0014175355900079012,
+      "step": 32
+    },
+    {
+      "epoch": 0.05467336683417085,
+      "grad_norm": 0.41970404982566833,
+      "learning_rate": 2.6984126984126986e-06,
+      "logits/chosen": 1.4691964387893677,
+      "logits/rejected": 1.5019806623458862,
+      "logps/chosen": -30.574474334716797,
+      "logps/rejected": -33.60447311401367,
+      "loss": 0.695,
+      "rewards/accuracies": 0.453125,
+      "rewards/chosen": -0.004354957491159439,
+      "rewards/margins": -0.00334002822637558,
+      "rewards/rejected": -0.001014929497614503,
+      "step": 34
+    },
+    {
+      "epoch": 0.057889447236180905,
+      "grad_norm": 0.4544912278652191,
+      "learning_rate": 2.8571428571428573e-06,
+      "logits/chosen": 1.2973356246948242,
+      "logits/rejected": 1.3056020736694336,
+      "logps/chosen": -29.477462768554688,
+      "logps/rejected": -33.272056579589844,
+      "loss": 0.6925,
+      "rewards/accuracies": 0.5390625,
+      "rewards/chosen": -0.0026571941561996937,
+      "rewards/margins": 0.001574229565449059,
+      "rewards/rejected": -0.0042314231395721436,
+      "step": 36
+    },
+    {
+      "epoch": 0.06110552763819096,
+      "grad_norm": 0.4139344096183777,
+      "learning_rate": 3.015873015873016e-06,
+      "logits/chosen": 1.3676331043243408,
+      "logits/rejected": 1.4585086107254028,
+      "logps/chosen": -31.87004280090332,
+      "logps/rejected": -33.22168731689453,
+      "loss": 0.6944,
+      "rewards/accuracies": 0.4453125,
+      "rewards/chosen": -0.006321355234831572,
+      "rewards/margins": -0.0021584928035736084,
+      "rewards/rejected": -0.004162862431257963,
+      "step": 38
+    },
+    {
+      "epoch": 0.06432160804020101,
+      "grad_norm": 0.4309309422969818,
+      "learning_rate": 3.1746031746031746e-06,
+      "logits/chosen": 1.4414952993392944,
+      "logits/rejected": 1.4826974868774414,
+      "logps/chosen": -29.79780387878418,
+      "logps/rejected": -33.88084030151367,
+      "loss": 0.6933,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.001039503258652985,
+      "rewards/margins": -7.016677409410477e-05,
+      "rewards/rejected": -0.0009693363681435585,
+      "step": 40
+    },
+    {
+      "epoch": 0.06753768844221106,
+      "grad_norm": 0.4449164569377899,
+      "learning_rate": 3.3333333333333333e-06,
+      "logits/chosen": 1.2832252979278564,
+      "logits/rejected": 1.2548108100891113,
+      "logps/chosen": -30.628887176513672,
+      "logps/rejected": -36.61481475830078,
+      "loss": 0.6917,
+      "rewards/accuracies": 0.53125,
+      "rewards/chosen": -0.0037453575059771538,
+      "rewards/margins": 0.003167451359331608,
+      "rewards/rejected": -0.006912808865308762,
+      "step": 42
+    },
+    {
+      "epoch": 0.0707537688442211,
+      "grad_norm": 0.45570507645606995,
+      "learning_rate": 3.492063492063492e-06,
+      "logits/chosen": 1.5801219940185547,
+      "logits/rejected": 1.5999751091003418,
+      "logps/chosen": -31.53429412841797,
+      "logps/rejected": -36.12282943725586,
+      "loss": 0.693,
+      "rewards/accuracies": 0.453125,
+      "rewards/chosen": -0.0012303784023970366,
+      "rewards/margins": 0.000611663912422955,
+      "rewards/rejected": -0.0018420428968966007,
+      "step": 44
+    },
+    {
+      "epoch": 0.07396984924623115,
+      "grad_norm": 0.44526174664497375,
+      "learning_rate": 3.6507936507936507e-06,
+      "logits/chosen": 1.2573561668395996,
+      "logits/rejected": 1.2938108444213867,
+      "logps/chosen": -30.280303955078125,
+      "logps/rejected": -34.33146667480469,
+      "loss": 0.6913,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": -0.0019697281531989574,
+      "rewards/margins": 0.004040538799017668,
+      "rewards/rejected": -0.006010266952216625,
+      "step": 46
+    },
+    {
+      "epoch": 0.07718592964824121,
+      "grad_norm": 1.2515525817871094,
+      "learning_rate": 3.80952380952381e-06,
+      "logits/chosen": 0.8721216917037964,
+      "logits/rejected": 0.8969432711601257,
+      "logps/chosen": -32.89268493652344,
+      "logps/rejected": -34.27225875854492,
+      "loss": 0.6897,
+      "rewards/accuracies": 0.59375,
+      "rewards/chosen": -0.0003448700299486518,
+      "rewards/margins": 0.0074737779796123505,
+      "rewards/rejected": -0.007818647660315037,
+      "step": 48
+    },
+    {
+      "epoch": 0.08040201005025126,
+      "grad_norm": 0.6490676999092102,
+      "learning_rate": 3.968253968253968e-06,
+      "logits/chosen": 1.3292715549468994,
+      "logits/rejected": 1.3583744764328003,
+      "logps/chosen": -27.783782958984375,
+      "logps/rejected": -34.58745574951172,
+      "loss": 0.6882,
+      "rewards/accuracies": 0.625,
+      "rewards/chosen": 0.0020002706442028284,
+      "rewards/margins": 0.010215929709374905,
+      "rewards/rejected": -0.00821565929800272,
+      "step": 50
+    },
+    {
+      "epoch": 0.08040201005025126,
+      "eval_logits/chosen": 0.5501546263694763,
+      "eval_logits/rejected": 0.5580804944038391,
+      "eval_logps/chosen": -30.54437255859375,
+      "eval_logps/rejected": -35.43931198120117,
+      "eval_loss": 0.6898252964019775,
+      "eval_rewards/accuracies": 0.5349999666213989,
+      "eval_rewards/chosen": -0.0043669044971466064,
+      "eval_rewards/margins": 0.0071200719103217125,
+      "eval_rewards/rejected": -0.011486975476145744,
+      "eval_runtime": 48.2016,
+      "eval_samples_per_second": 4.149,
+      "eval_steps_per_second": 1.037,
+      "step": 50
+    },
+    {
+      "epoch": 0.0836180904522613,
+      "grad_norm": 0.4490942358970642,
+      "learning_rate": 4.126984126984127e-06,
+      "logits/chosen": 1.4922609329223633,
+      "logits/rejected": 1.5168870687484741,
+      "logps/chosen": -30.933542251586914,
+      "logps/rejected": -32.83856964111328,
+      "loss": 0.6887,
+      "rewards/accuracies": 0.5859375,
+      "rewards/chosen": -0.0028348369523882866,
+      "rewards/margins": 0.009395316243171692,
+      "rewards/rejected": -0.012230154126882553,
+      "step": 52
+    },
+    {
+      "epoch": 0.08683417085427135,
+      "grad_norm": 0.4455580413341522,
+      "learning_rate": 4.2857142857142855e-06,
+      "logits/chosen": 1.2956047058105469,
+      "logits/rejected": 1.3195202350616455,
+      "logps/chosen": -33.25441360473633,
+      "logps/rejected": -34.96155548095703,
+      "loss": 0.6888,
+      "rewards/accuracies": 0.609375,
+      "rewards/chosen": -0.011137772351503372,
+      "rewards/margins": 0.009241417050361633,
+      "rewards/rejected": -0.020379189401865005,
+      "step": 54
+    },
+    {
+      "epoch": 0.09005025125628141,
+      "grad_norm": 0.46649816632270813,
+      "learning_rate": 4.444444444444444e-06,
+      "logits/chosen": 1.442166805267334,
+      "logits/rejected": 1.4832069873809814,
+      "logps/chosen": -30.720643997192383,
+      "logps/rejected": -34.70309829711914,
+      "loss": 0.6936,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": -0.010124221444129944,
+      "rewards/margins": -0.0002470978070050478,
+      "rewards/rejected": -0.009877122938632965,
+      "step": 56
+    },
+    {
+      "epoch": 0.09326633165829146,
+      "grad_norm": 0.467673122882843,
+      "learning_rate": 4.603174603174604e-06,
+      "logits/chosen": 1.3763530254364014,
+      "logits/rejected": 1.4356203079223633,
+      "logps/chosen": -31.76504135131836,
+      "logps/rejected": -33.25387954711914,
+      "loss": 0.6874,
+      "rewards/accuracies": 0.5546875,
+      "rewards/chosen": -0.008322305977344513,
+      "rewards/margins": 0.012220000848174095,
+      "rewards/rejected": -0.020542306825518608,
+      "step": 58
+    },
+    {
+      "epoch": 0.0964824120603015,
+      "grad_norm": 0.5383568406105042,
+      "learning_rate": 4.761904761904762e-06,
+      "logits/chosen": 1.2813664674758911,
+      "logits/rejected": 1.3347243070602417,
+      "logps/chosen": -33.40138626098633,
+      "logps/rejected": -34.50080871582031,
+      "loss": 0.6869,
+      "rewards/accuracies": 0.5859375,
+      "rewards/chosen": -0.014261961914598942,
+      "rewards/margins": 0.013282392174005508,
+      "rewards/rejected": -0.027544351294636726,
+      "step": 60
+    },
+    {
+      "epoch": 0.09969849246231156,
+      "grad_norm": 0.4995841383934021,
+      "learning_rate": 4.920634920634921e-06,
+      "logits/chosen": 1.3010649681091309,
+      "logits/rejected": 1.3390882015228271,
+      "logps/chosen": -29.836774826049805,
+      "logps/rejected": -34.70252990722656,
+      "loss": 0.6843,
+      "rewards/accuracies": 0.625,
+      "rewards/chosen": -0.010679999366402626,
+      "rewards/margins": 0.018656719475984573,
+      "rewards/rejected": -0.0293367151170969,
+      "step": 62
+    },
+    {
+      "epoch": 0.10291457286432161,
+      "grad_norm": 0.4841148555278778,
+      "learning_rate": 4.999960377651517e-06,
+      "logits/chosen": 1.422184705734253,
+      "logits/rejected": 1.4505563974380493,
+      "logps/chosen": -34.65579605102539,
+      "logps/rejected": -33.47809600830078,
+      "loss": 0.6897,
+      "rewards/accuracies": 0.5390625,
+      "rewards/chosen": -0.01736070215702057,
+      "rewards/margins": 0.007918944582343102,
+      "rewards/rejected": -0.02527964487671852,
+      "step": 64
+    },
+    {
+      "epoch": 0.10613065326633166,
+      "grad_norm": 0.5981670022010803,
+      "learning_rate": 4.999643406399275e-06,
+      "logits/chosen": 1.1232147216796875,
+      "logits/rejected": 1.145438313484192,
+      "logps/chosen": -29.947956085205078,
+      "logps/rejected": -33.95368194580078,
+      "loss": 0.6838,
+      "rewards/accuracies": 0.5703125,
+      "rewards/chosen": -0.018678227439522743,
+      "rewards/margins": 0.019824998453259468,
+      "rewards/rejected": -0.03850322589278221,
+      "step": 66
+    },
+    {
+      "epoch": 0.1093467336683417,
+      "grad_norm": 0.571692705154419,
+      "learning_rate": 4.999009504083738e-06,
+      "logits/chosen": 1.3420298099517822,
+      "logits/rejected": 1.338175654411316,
+      "logps/chosen": -30.924522399902344,
+      "logps/rejected": -37.51594161987305,
+      "loss": 0.6828,
+      "rewards/accuracies": 0.609375,
+      "rewards/chosen": -0.020221160724759102,
+      "rewards/margins": 0.02198150008916855,
+      "rewards/rejected": -0.0422026626765728,
+      "step": 68
+    },
+    {
+      "epoch": 0.11256281407035176,
+      "grad_norm": 0.5207106471061707,
+      "learning_rate": 4.998058751077705e-06,
+      "logits/chosen": 1.3332023620605469,
+      "logits/rejected": 1.3814537525177002,
+      "logps/chosen": -31.7532958984375,
+      "logps/rejected": -34.753013610839844,
+      "loss": 0.6822,
+      "rewards/accuracies": 0.59375,
+      "rewards/chosen": -0.031598228961229324,
+      "rewards/margins": 0.023511648178100586,
+      "rewards/rejected": -0.05510988086462021,
+      "step": 70
+    },
+    {
+      "epoch": 0.11577889447236181,
+      "grad_norm": 0.47247663140296936,
+      "learning_rate": 4.996791267927632e-06,
+      "logits/chosen": 1.3693702220916748,
+      "logits/rejected": 1.3828941583633423,
+      "logps/chosen": -30.8132266998291,
+      "logps/rejected": -35.77416229248047,
+      "loss": 0.6827,
+      "rewards/accuracies": 0.6015625,
+      "rewards/chosen": -0.030194712802767754,
+      "rewards/margins": 0.022484488785266876,
+      "rewards/rejected": -0.05267919972538948,
+      "step": 72
+    },
+    {
+      "epoch": 0.11899497487437186,
+      "grad_norm": 0.48132631182670593,
+      "learning_rate": 4.995207215338358e-06,
+      "logits/chosen": 1.490431785583496,
+      "logits/rejected": 1.5298147201538086,
+      "logps/chosen": -34.107643127441406,
+      "logps/rejected": -34.63951110839844,
+      "loss": 0.687,
+      "rewards/accuracies": 0.515625,
+      "rewards/chosen": -0.036115553230047226,
+      "rewards/margins": 0.014621476642787457,
+      "rewards/rejected": -0.05073703080415726,
+      "step": 74
+    },
+    {
+      "epoch": 0.12221105527638192,
+      "grad_norm": 0.5402019619941711,
+      "learning_rate": 4.9933067941527175e-06,
+      "logits/chosen": 1.3514180183410645,
+      "logits/rejected": 1.3422389030456543,
+      "logps/chosen": -36.069549560546875,
+      "logps/rejected": -33.26152038574219,
+      "loss": 0.6841,
+      "rewards/accuracies": 0.5625,
+      "rewards/chosen": -0.04465916007757187,
+      "rewards/margins": 0.02099396474659443,
+      "rewards/rejected": -0.06565312296152115,
+      "step": 76
+    },
+    {
+      "epoch": 0.12542713567839195,
+      "grad_norm": 0.5146540999412537,
+      "learning_rate": 4.991090245326083e-06,
+      "logits/chosen": 1.0872302055358887,
+      "logits/rejected": 1.1015247106552124,
+      "logps/chosen": -29.906904220581055,
+      "logps/rejected": -38.96195983886719,
+      "loss": 0.6758,
+      "rewards/accuracies": 0.609375,
+      "rewards/chosen": -0.038489095866680145,
+      "rewards/margins": 0.037541136145591736,
+      "rewards/rejected": -0.07603023201227188,
+      "step": 78
+    },
+    {
+      "epoch": 0.12864321608040202,
+      "grad_norm": 0.5551427602767944,
+      "learning_rate": 4.988557849895811e-06,
+      "logits/chosen": 1.3899340629577637,
+      "logits/rejected": 1.4685105085372925,
+      "logps/chosen": -34.21710205078125,
+      "logps/rejected": -38.152099609375,
+      "loss": 0.6695,
+      "rewards/accuracies": 0.65625,
+      "rewards/chosen": -0.03813210129737854,
+      "rewards/margins": 0.05099986493587494,
+      "rewards/rejected": -0.08913196623325348,
+      "step": 80
+    },
+    {
+      "epoch": 0.13185929648241207,
+      "grad_norm": 0.5468294620513916,
+      "learning_rate": 4.9857099289456115e-06,
+      "logits/chosen": 1.4150495529174805,
+      "logits/rejected": 1.4033807516098022,
+      "logps/chosen": -30.979236602783203,
+      "logps/rejected": -36.2646484375,
+      "loss": 0.6759,
+      "rewards/accuracies": 0.6484375,
+      "rewards/chosen": -0.04814925417304039,
+      "rewards/margins": 0.0380784347653389,
+      "rewards/rejected": -0.08622768521308899,
+      "step": 82
+    },
+    {
+      "epoch": 0.13507537688442212,
+      "grad_norm": 0.47936275601387024,
+      "learning_rate": 4.9825468435648345e-06,
+      "logits/chosen": 1.1383363008499146,
+      "logits/rejected": 1.1823409795761108,
+      "logps/chosen": -29.871156692504883,
+      "logps/rejected": -32.21099090576172,
+      "loss": 0.6789,
+      "rewards/accuracies": 0.578125,
+      "rewards/chosen": -0.04363500326871872,
+      "rewards/margins": 0.03290008008480072,
+      "rewards/rejected": -0.07653507590293884,
+      "step": 84
+    },
+    {
+      "epoch": 0.13829145728643216,
+      "grad_norm": 0.48245131969451904,
+      "learning_rate": 4.97906899480269e-06,
+      "logits/chosen": 1.4419957399368286,
+      "logits/rejected": 1.475970983505249,
+      "logps/chosen": -30.77682113647461,
+      "logps/rejected": -35.76774215698242,
+      "loss": 0.6698,
+      "rewards/accuracies": 0.6484375,
+      "rewards/chosen": -0.04197033494710922,
+      "rewards/margins": 0.05154535919427872,
+      "rewards/rejected": -0.09351569414138794,
+      "step": 86
+    },
+    {
+      "epoch": 0.1415075376884422,
+      "grad_norm": 0.5745317935943604,
+      "learning_rate": 4.975276823617399e-06,
+      "logits/chosen": 1.4063524007797241,
+      "logits/rejected": 1.439096450805664,
+      "logps/chosen": -29.64651870727539,
+      "logps/rejected": -35.58737564086914,
+      "loss": 0.6644,
+      "rewards/accuracies": 0.6484375,
+      "rewards/chosen": -0.045151449739933014,
+      "rewards/margins": 0.06304246932268143,
+      "rewards/rejected": -0.10819391906261444,
+      "step": 88
+    },
+    {
+      "epoch": 0.14472361809045226,
+      "grad_norm": 0.5324569344520569,
+      "learning_rate": 4.971170810820279e-06,
+      "logits/chosen": 1.3591543436050415,
+      "logits/rejected": 1.3323087692260742,
+      "logps/chosen": -34.168128967285156,
+      "logps/rejected": -37.837242126464844,
+      "loss": 0.6722,
+      "rewards/accuracies": 0.640625,
+      "rewards/chosen": -0.05879779905080795,
+      "rewards/margins": 0.04740719497203827,
+      "rewards/rejected": -0.10620499402284622,
+      "step": 90
+    },
+    {
+      "epoch": 0.1479396984924623,
+      "grad_norm": 0.6153348684310913,
+      "learning_rate": 4.96675147701479e-06,
+      "logits/chosen": 1.3236520290374756,
+      "logits/rejected": 1.3638286590576172,
+      "logps/chosen": -33.26496887207031,
+      "logps/rejected": -35.09466552734375,
+      "loss": 0.678,
+      "rewards/accuracies": 0.609375,
+      "rewards/chosen": -0.07028309255838394,
+      "rewards/margins": 0.037059590220451355,
+      "rewards/rejected": -0.1073426827788353,
+      "step": 92
+    },
+    {
+      "epoch": 0.15115577889447235,
+      "grad_norm": 0.6804412603378296,
+      "learning_rate": 4.962019382530521e-06,
+      "logits/chosen": 1.1772613525390625,
+      "logits/rejected": 1.2550945281982422,
+      "logps/chosen": -34.816246032714844,
+      "logps/rejected": -34.04191970825195,
+      "loss": 0.676,
+      "rewards/accuracies": 0.5859375,
+      "rewards/chosen": -0.0760345533490181,
+      "rewards/margins": 0.041960567235946655,
+      "rewards/rejected": -0.11799512058496475,
+      "step": 94
+    },
+    {
+      "epoch": 0.15437185929648242,
+      "grad_norm": 0.5287646055221558,
+      "learning_rate": 4.956975127352146e-06,
+      "logits/chosen": 1.1335078477859497,
+      "logits/rejected": 1.0898680686950684,
+      "logps/chosen": -32.298492431640625,
+      "logps/rejected": -38.93742370605469,
+      "loss": 0.6661,
+      "rewards/accuracies": 0.6328125,
+      "rewards/chosen": -0.07041005790233612,
+      "rewards/margins": 0.06344542652368546,
+      "rewards/rejected": -0.13385546207427979,
+      "step": 96
+    },
+    {
+      "epoch": 0.15758793969849247,
+      "grad_norm": 0.5636870265007019,
+      "learning_rate": 4.951619351043353e-06,
+      "logits/chosen": 1.3233567476272583,
+      "logits/rejected": 1.356798768043518,
+      "logps/chosen": -31.532367706298828,
+      "logps/rejected": -33.19044494628906,
+      "loss": 0.6698,
+      "rewards/accuracies": 0.6015625,
+      "rewards/chosen": -0.08084239810705185,
+      "rewards/margins": 0.05414910987019539,
+      "rewards/rejected": -0.13499149680137634,
+      "step": 98
+    },
+    {
+      "epoch": 0.16080402010050251,
+      "grad_norm": 0.5570967197418213,
+      "learning_rate": 4.945952732665755e-06,
+      "logits/chosen": 1.301795244216919,
+      "logits/rejected": 1.3351643085479736,
+      "logps/chosen": -32.771270751953125,
+      "logps/rejected": -41.387725830078125,
+      "loss": 0.6603,
+      "rewards/accuracies": 0.6015625,
+      "rewards/chosen": -0.07904825359582901,
+      "rewards/margins": 0.07616038620471954,
+      "rewards/rejected": -0.15520863234996796,
+      "step": 100
+    },
+    {
+      "epoch": 0.16080402010050251,
+      "eval_logits/chosen": 0.5383610725402832,
+      "eval_logits/rejected": 0.544625997543335,
+      "eval_logps/chosen": -31.30695343017578,
+      "eval_logps/rejected": -36.894073486328125,
+      "eval_loss": 0.6592575311660767,
+      "eval_rewards/accuracies": 0.6850000023841858,
+      "eval_rewards/chosen": -0.08062513917684555,
+      "eval_rewards/margins": 0.07633791118860245,
+      "eval_rewards/rejected": -0.156963050365448,
+      "eval_runtime": 48.1277,
+      "eval_samples_per_second": 4.156,
+      "eval_steps_per_second": 1.039,
+      "step": 100
+    }
+  ],
+  "logging_steps": 2,
+  "max_steps": 621,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.5711860103774208e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcf7d186e3568d77f55778a26c607e4da9c0a96dd9d37a4828aaf8e3780e13fb
+size 5752

checkpoint-200/README.md

@@ -0,0 +1,208 @@
+---
+base_model: /p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct
+- llama-factory
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-200/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9dac64d83cb235284c26a460f27c061eed2600489ff67edf5842e43eea76ab7
+size 40385208

checkpoint-200/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-200/chat_template.json

@@ -0,0 +1,3 @@
+{
+  "chat_template": "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}"
+}

checkpoint-200/preprocessor_config.json

@@ -0,0 +1,32 @@
+{
+  "crop_size": null,
+  "default_to_square": true,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_processor_type": "Qwen2VLImageProcessorFast",
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "max_pixels": 12845056,
+  "merge_size": 2,
+  "min_pixels": 3136,
+  "patch_size": 14,
+  "processor_class": "Qwen2_5_VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "longest_edge": 1003520,
+    "shortest_edge": 3136
+  },
+  "temporal_patch_size": 2
+}

checkpoint-200/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5c5e18f922d0af74d820247ae97bee506ab412554a58345ddf2558abc94ee3e3
+size 15024

checkpoint-200/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a2dcca6d9741f46592359768ea2212b9321da6408d1fd7d3a80b017bf37f434
+size 15024

checkpoint-200/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69420ece2c255923c5cbb3c6c9c4a6b9cb38fb57e5d3033c8b7d436a1faf6f13
+size 15024

checkpoint-200/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:66f278b40a1e23b88a657c4e5d03afa8dbbbe14dfeb16f6b4beedaece6cdd0b9
+size 15024

checkpoint-200/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-200/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

checkpoint-200/tokenizer_config.json

@@ -0,0 +1,210 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "processor_class": "Qwen2_5_VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-200/trainer_state.json

@@ -0,0 +1,1597 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.32160804020100503,
+  "eval_steps": 50,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0032160804020100503,
+      "grad_norm": 0.398336261510849,
+      "learning_rate": 1.5873015873015874e-07,
+      "logits/chosen": 0.9754317998886108,
+      "logits/rejected": 1.027288556098938,
+      "logps/chosen": -31.835588455200195,
+      "logps/rejected": -34.915592193603516,
+      "loss": 0.6931,
+      "rewards/accuracies": 0.0,
+      "rewards/chosen": 0.0,
+      "rewards/margins": 0.0,
+      "rewards/rejected": 0.0,
+      "step": 2
+    },
+    {
+      "epoch": 0.006432160804020101,
+      "grad_norm": 0.4530814290046692,
+      "learning_rate": 3.174603174603175e-07,
+      "logits/chosen": 1.4162323474884033,
+      "logits/rejected": 1.4583587646484375,
+      "logps/chosen": -30.352081298828125,
+      "logps/rejected": -33.31166076660156,
+      "loss": 0.6938,
+      "rewards/accuracies": 0.5078125,
+      "rewards/chosen": -0.003089881967753172,
+      "rewards/margins": -0.0008391638984903693,
+      "rewards/rejected": -0.0022507174871861935,
+      "step": 4
+    },
+    {
+      "epoch": 0.009648241206030151,
+      "grad_norm": 0.46197977662086487,
+      "learning_rate": 4.7619047619047623e-07,
+      "logits/chosen": 1.2524590492248535,
+      "logits/rejected": 1.3010387420654297,
+      "logps/chosen": -29.417587280273438,
+      "logps/rejected": -33.4231071472168,
+      "loss": 0.6936,
+      "rewards/accuracies": 0.4609375,
+      "rewards/chosen": 0.0004551531164906919,
+      "rewards/margins": -0.00048057991079986095,
+      "rewards/rejected": 0.0009357327362522483,
+      "step": 6
+    },
+    {
+      "epoch": 0.012864321608040201,
+      "grad_norm": 0.4546354115009308,
+      "learning_rate": 6.34920634920635e-07,
+      "logits/chosen": 1.4723472595214844,
+      "logits/rejected": 1.5091776847839355,
+      "logps/chosen": -32.08555221557617,
+      "logps/rejected": -34.23310089111328,
+      "loss": 0.6917,
+      "rewards/accuracies": 0.484375,
+      "rewards/chosen": 0.004024952184408903,
+      "rewards/margins": 0.0033740843646228313,
+      "rewards/rejected": 0.0006508685764856637,
+      "step": 8
+    },
+    {
+      "epoch": 0.016080402010050253,
+      "grad_norm": 0.4574313759803772,
+      "learning_rate": 7.936507936507937e-07,
+      "logits/chosen": 1.278916597366333,
+      "logits/rejected": 1.3133082389831543,
+      "logps/chosen": -31.22161102294922,
+      "logps/rejected": -34.931907653808594,
+      "loss": 0.6929,
+      "rewards/accuracies": 0.4453125,
+      "rewards/chosen": 0.0015166704542934895,
+      "rewards/margins": 0.0008749825647100806,
+      "rewards/rejected": 0.0006416881224140525,
+      "step": 10
+    },
+    {
+      "epoch": 0.019296482412060303,
+      "grad_norm": 0.44448181986808777,
+      "learning_rate": 9.523809523809525e-07,
+      "logits/chosen": 1.3098528385162354,
+      "logits/rejected": 1.3612464666366577,
+      "logps/chosen": -30.1417179107666,
+      "logps/rejected": -35.877037048339844,
+      "loss": 0.6958,
+      "rewards/accuracies": 0.4296875,
+      "rewards/chosen": -0.00687691755592823,
+      "rewards/margins": -0.004914135672152042,
+      "rewards/rejected": -0.001962781883776188,
+      "step": 12
+    },
+    {
+      "epoch": 0.022512562814070353,
+      "grad_norm": 0.4568133056163788,
+      "learning_rate": 1.111111111111111e-06,
+      "logits/chosen": 1.4617085456848145,
+      "logits/rejected": 1.4705244302749634,
+      "logps/chosen": -30.95242691040039,
+      "logps/rejected": -36.34254455566406,
+      "loss": 0.6939,
+      "rewards/accuracies": 0.4921875,
+      "rewards/chosen": -0.0029178508557379246,
+      "rewards/margins": -0.0011518384562805295,
+      "rewards/rejected": -0.001766012515872717,
+      "step": 14
+    },
+    {
+      "epoch": 0.025728643216080403,
+      "grad_norm": 0.4778600037097931,
+      "learning_rate": 1.26984126984127e-06,
+      "logits/chosen": 1.3174049854278564,
+      "logits/rejected": 1.3094675540924072,
+      "logps/chosen": -28.937427520751953,
+      "logps/rejected": -36.366676330566406,
+      "loss": 0.6932,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.0009857461554929614,
+      "rewards/margins": 0.00013500778004527092,
+      "rewards/rejected": -0.001120753469876945,
+      "step": 16
+    },
+    {
+      "epoch": 0.028944723618090452,
+      "grad_norm": 0.421551913022995,
+      "learning_rate": 1.4285714285714286e-06,
+      "logits/chosen": 1.3308428525924683,
+      "logits/rejected": 1.4079976081848145,
+      "logps/chosen": -33.79199981689453,
+      "logps/rejected": -36.55875778198242,
+      "loss": 0.6941,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.002126973122358322,
+      "rewards/margins": -0.001552498433738947,
+      "rewards/rejected": -0.0005744747468270361,
+      "step": 18
+    },
+    {
+      "epoch": 0.032160804020100506,
+      "grad_norm": 0.4264390468597412,
+      "learning_rate": 1.5873015873015873e-06,
+      "logits/chosen": 1.4721007347106934,
+      "logits/rejected": 1.485215663909912,
+      "logps/chosen": -31.09832000732422,
+      "logps/rejected": -36.288822174072266,
+      "loss": 0.6951,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.005364666692912579,
+      "rewards/margins": -0.003565119579434395,
+      "rewards/rejected": -0.0017995464149862528,
+      "step": 20
+    },
+    {
+      "epoch": 0.03537688442211055,
+      "grad_norm": 0.40373197197914124,
+      "learning_rate": 1.746031746031746e-06,
+      "logits/chosen": 1.3501726388931274,
+      "logits/rejected": 1.370957374572754,
+      "logps/chosen": -31.461055755615234,
+      "logps/rejected": -34.38249588012695,
+      "loss": 0.6938,
+      "rewards/accuracies": 0.46875,
+      "rewards/chosen": -0.003749656490981579,
+      "rewards/margins": -0.0010161390528082848,
+      "rewards/rejected": -0.0027335172053426504,
+      "step": 22
+    },
+    {
+      "epoch": 0.038592964824120606,
+      "grad_norm": 0.4503360390663147,
+      "learning_rate": 1.904761904761905e-06,
+      "logits/chosen": 1.1474275588989258,
+      "logits/rejected": 1.136375904083252,
+      "logps/chosen": -32.1020393371582,
+      "logps/rejected": -35.39215087890625,
+      "loss": 0.695,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.0007312107481993735,
+      "rewards/margins": -0.003288470208644867,
+      "rewards/rejected": 0.002557259751483798,
+      "step": 24
+    },
+    {
+      "epoch": 0.04180904522613065,
+      "grad_norm": 0.39122092723846436,
+      "learning_rate": 2.0634920634920634e-06,
+      "logits/chosen": 1.0779863595962524,
+      "logits/rejected": 1.0993211269378662,
+      "logps/chosen": -31.845603942871094,
+      "logps/rejected": -35.04288864135742,
+      "loss": 0.6949,
+      "rewards/accuracies": 0.46875,
+      "rewards/chosen": -0.0012633285950869322,
+      "rewards/margins": -0.0032839984633028507,
+      "rewards/rejected": 0.002020669635385275,
+      "step": 26
+    },
+    {
+      "epoch": 0.045025125628140705,
+      "grad_norm": 0.4362376630306244,
+      "learning_rate": 2.222222222222222e-06,
+      "logits/chosen": 1.4343101978302002,
+      "logits/rejected": 1.4324826002120972,
+      "logps/chosen": -29.318994522094727,
+      "logps/rejected": -31.928974151611328,
+      "loss": 0.6941,
+      "rewards/accuracies": 0.53125,
+      "rewards/chosen": -0.00553980004042387,
+      "rewards/margins": -0.0015490761725232005,
+      "rewards/rejected": -0.003990724217146635,
+      "step": 28
+    },
+    {
+      "epoch": 0.04824120603015075,
+      "grad_norm": 0.4269290864467621,
+      "learning_rate": 2.380952380952381e-06,
+      "logits/chosen": 1.29887056350708,
+      "logits/rejected": 1.3346788883209229,
+      "logps/chosen": -32.30498123168945,
+      "logps/rejected": -35.8641357421875,
+      "loss": 0.694,
+      "rewards/accuracies": 0.46875,
+      "rewards/chosen": -0.00222906400449574,
+      "rewards/margins": -0.0013898832257837057,
+      "rewards/rejected": -0.0008391811279579997,
+      "step": 30
+    },
+    {
+      "epoch": 0.051457286432160805,
+      "grad_norm": 0.4749397337436676,
+      "learning_rate": 2.53968253968254e-06,
+      "logits/chosen": 1.2239577770233154,
+      "logits/rejected": 1.2360297441482544,
+      "logps/chosen": -30.756057739257812,
+      "logps/rejected": -35.22294616699219,
+      "loss": 0.6937,
+      "rewards/accuracies": 0.46875,
+      "rewards/chosen": 0.0008076138328760862,
+      "rewards/margins": -0.0006099215243011713,
+      "rewards/rejected": 0.0014175355900079012,
+      "step": 32
+    },
+    {
+      "epoch": 0.05467336683417085,
+      "grad_norm": 0.41970404982566833,
+      "learning_rate": 2.6984126984126986e-06,
+      "logits/chosen": 1.4691964387893677,
+      "logits/rejected": 1.5019806623458862,
+      "logps/chosen": -30.574474334716797,
+      "logps/rejected": -33.60447311401367,
+      "loss": 0.695,
+      "rewards/accuracies": 0.453125,
+      "rewards/chosen": -0.004354957491159439,
+      "rewards/margins": -0.00334002822637558,
+      "rewards/rejected": -0.001014929497614503,
+      "step": 34
+    },
+    {
+      "epoch": 0.057889447236180905,
+      "grad_norm": 0.4544912278652191,
+      "learning_rate": 2.8571428571428573e-06,
+      "logits/chosen": 1.2973356246948242,
+      "logits/rejected": 1.3056020736694336,
+      "logps/chosen": -29.477462768554688,
+      "logps/rejected": -33.272056579589844,
+      "loss": 0.6925,
+      "rewards/accuracies": 0.5390625,
+      "rewards/chosen": -0.0026571941561996937,
+      "rewards/margins": 0.001574229565449059,
+      "rewards/rejected": -0.0042314231395721436,
+      "step": 36
+    },
+    {
+      "epoch": 0.06110552763819096,
+      "grad_norm": 0.4139344096183777,
+      "learning_rate": 3.015873015873016e-06,
+      "logits/chosen": 1.3676331043243408,
+      "logits/rejected": 1.4585086107254028,
+      "logps/chosen": -31.87004280090332,
+      "logps/rejected": -33.22168731689453,
+      "loss": 0.6944,
+      "rewards/accuracies": 0.4453125,
+      "rewards/chosen": -0.006321355234831572,
+      "rewards/margins": -0.0021584928035736084,
+      "rewards/rejected": -0.004162862431257963,
+      "step": 38
+    },
+    {
+      "epoch": 0.06432160804020101,
+      "grad_norm": 0.4309309422969818,
+      "learning_rate": 3.1746031746031746e-06,
+      "logits/chosen": 1.4414952993392944,
+      "logits/rejected": 1.4826974868774414,
+      "logps/chosen": -29.79780387878418,
+      "logps/rejected": -33.88084030151367,
+      "loss": 0.6933,
+      "rewards/accuracies": 0.4765625,
+      "rewards/chosen": -0.001039503258652985,
+      "rewards/margins": -7.016677409410477e-05,
+      "rewards/rejected": -0.0009693363681435585,
+      "step": 40
+    },
+    {
+      "epoch": 0.06753768844221106,
+      "grad_norm": 0.4449164569377899,
+      "learning_rate": 3.3333333333333333e-06,
+      "logits/chosen": 1.2832252979278564,
+      "logits/rejected": 1.2548108100891113,
+      "logps/chosen": -30.628887176513672,
+      "logps/rejected": -36.61481475830078,
+      "loss": 0.6917,
+      "rewards/accuracies": 0.53125,
+      "rewards/chosen": -0.0037453575059771538,
+      "rewards/margins": 0.003167451359331608,
+      "rewards/rejected": -0.006912808865308762,
+      "step": 42
+    },
+    {
+      "epoch": 0.0707537688442211,
+      "grad_norm": 0.45570507645606995,
+      "learning_rate": 3.492063492063492e-06,
+      "logits/chosen": 1.5801219940185547,
+      "logits/rejected": 1.5999751091003418,
+      "logps/chosen": -31.53429412841797,
+      "logps/rejected": -36.12282943725586,
+      "loss": 0.693,
+      "rewards/accuracies": 0.453125,
+      "rewards/chosen": -0.0012303784023970366,
+      "rewards/margins": 0.000611663912422955,
+      "rewards/rejected": -0.0018420428968966007,
+      "step": 44
+    },
+    {
+      "epoch": 0.07396984924623115,
+      "grad_norm": 0.44526174664497375,
+      "learning_rate": 3.6507936507936507e-06,
+      "logits/chosen": 1.2573561668395996,
+      "logits/rejected": 1.2938108444213867,
+      "logps/chosen": -30.280303955078125,
+      "logps/rejected": -34.33146667480469,
+      "loss": 0.6913,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": -0.0019697281531989574,
+      "rewards/margins": 0.004040538799017668,
+      "rewards/rejected": -0.006010266952216625,
+      "step": 46
+    },
+    {
+      "epoch": 0.07718592964824121,
+      "grad_norm": 1.2515525817871094,
+      "learning_rate": 3.80952380952381e-06,
+      "logits/chosen": 0.8721216917037964,
+      "logits/rejected": 0.8969432711601257,
+      "logps/chosen": -32.89268493652344,
+      "logps/rejected": -34.27225875854492,
+      "loss": 0.6897,
+      "rewards/accuracies": 0.59375,
+      "rewards/chosen": -0.0003448700299486518,
+      "rewards/margins": 0.0074737779796123505,
+      "rewards/rejected": -0.007818647660315037,
+      "step": 48
+    },
+    {
+      "epoch": 0.08040201005025126,
+      "grad_norm": 0.6490676999092102,
+      "learning_rate": 3.968253968253968e-06,
+      "logits/chosen": 1.3292715549468994,
+      "logits/rejected": 1.3583744764328003,
+      "logps/chosen": -27.783782958984375,
+      "logps/rejected": -34.58745574951172,
+      "loss": 0.6882,
+      "rewards/accuracies": 0.625,
+      "rewards/chosen": 0.0020002706442028284,
+      "rewards/margins": 0.010215929709374905,
+      "rewards/rejected": -0.00821565929800272,
+      "step": 50
+    },
+    {
+      "epoch": 0.08040201005025126,
+      "eval_logits/chosen": 0.5501546263694763,
+      "eval_logits/rejected": 0.5580804944038391,
+      "eval_logps/chosen": -30.54437255859375,
+      "eval_logps/rejected": -35.43931198120117,
+      "eval_loss": 0.6898252964019775,
+      "eval_rewards/accuracies": 0.5349999666213989,
+      "eval_rewards/chosen": -0.0043669044971466064,
+      "eval_rewards/margins": 0.0071200719103217125,
+      "eval_rewards/rejected": -0.011486975476145744,
+      "eval_runtime": 48.2016,
+      "eval_samples_per_second": 4.149,
+      "eval_steps_per_second": 1.037,
+      "step": 50
+    },
+    {
+      "epoch": 0.0836180904522613,
+      "grad_norm": 0.4490942358970642,
+      "learning_rate": 4.126984126984127e-06,
+      "logits/chosen": 1.4922609329223633,
+      "logits/rejected": 1.5168870687484741,
+      "logps/chosen": -30.933542251586914,
+      "logps/rejected": -32.83856964111328,
+      "loss": 0.6887,
+      "rewards/accuracies": 0.5859375,
+      "rewards/chosen": -0.0028348369523882866,
+      "rewards/margins": 0.009395316243171692,
+      "rewards/rejected": -0.012230154126882553,
+      "step": 52
+    },
+    {
+      "epoch": 0.08683417085427135,
+      "grad_norm": 0.4455580413341522,
+      "learning_rate": 4.2857142857142855e-06,
+      "logits/chosen": 1.2956047058105469,
+      "logits/rejected": 1.3195202350616455,
+      "logps/chosen": -33.25441360473633,
+      "logps/rejected": -34.96155548095703,
+      "loss": 0.6888,
+      "rewards/accuracies": 0.609375,
+      "rewards/chosen": -0.011137772351503372,
+      "rewards/margins": 0.009241417050361633,
+      "rewards/rejected": -0.020379189401865005,
+      "step": 54
+    },
+    {
+      "epoch": 0.09005025125628141,
+      "grad_norm": 0.46649816632270813,
+      "learning_rate": 4.444444444444444e-06,
+      "logits/chosen": 1.442166805267334,
+      "logits/rejected": 1.4832069873809814,
+      "logps/chosen": -30.720643997192383,
+      "logps/rejected": -34.70309829711914,
+      "loss": 0.6936,
+      "rewards/accuracies": 0.5,
+      "rewards/chosen": -0.010124221444129944,
+      "rewards/margins": -0.0002470978070050478,
+      "rewards/rejected": -0.009877122938632965,
+      "step": 56
+    },
+    {
+      "epoch": 0.09326633165829146,
+      "grad_norm": 0.467673122882843,
+      "learning_rate": 4.603174603174604e-06,
+      "logits/chosen": 1.3763530254364014,
+      "logits/rejected": 1.4356203079223633,
+      "logps/chosen": -31.76504135131836,
+      "logps/rejected": -33.25387954711914,
+      "loss": 0.6874,
+      "rewards/accuracies": 0.5546875,
+      "rewards/chosen": -0.008322305977344513,
+      "rewards/margins": 0.012220000848174095,
+      "rewards/rejected": -0.020542306825518608,
+      "step": 58
+    },
+    {
+      "epoch": 0.0964824120603015,
+      "grad_norm": 0.5383568406105042,
+      "learning_rate": 4.761904761904762e-06,
+      "logits/chosen": 1.2813664674758911,
+      "logits/rejected": 1.3347243070602417,
+      "logps/chosen": -33.40138626098633,
+      "logps/rejected": -34.50080871582031,
+      "loss": 0.6869,
+      "rewards/accuracies": 0.5859375,
+      "rewards/chosen": -0.014261961914598942,
+      "rewards/margins": 0.013282392174005508,
+      "rewards/rejected": -0.027544351294636726,
+      "step": 60
+    },
+    {
+      "epoch": 0.09969849246231156,
+      "grad_norm": 0.4995841383934021,
+      "learning_rate": 4.920634920634921e-06,
+      "logits/chosen": 1.3010649681091309,
+      "logits/rejected": 1.3390882015228271,
+      "logps/chosen": -29.836774826049805,
+      "logps/rejected": -34.70252990722656,
+      "loss": 0.6843,
+      "rewards/accuracies": 0.625,
+      "rewards/chosen": -0.010679999366402626,
+      "rewards/margins": 0.018656719475984573,
+      "rewards/rejected": -0.0293367151170969,
+      "step": 62
+    },
+    {
+      "epoch": 0.10291457286432161,
+      "grad_norm": 0.4841148555278778,
+      "learning_rate": 4.999960377651517e-06,
+      "logits/chosen": 1.422184705734253,
+      "logits/rejected": 1.4505563974380493,
+      "logps/chosen": -34.65579605102539,
+      "logps/rejected": -33.47809600830078,
+      "loss": 0.6897,
+      "rewards/accuracies": 0.5390625,
+      "rewards/chosen": -0.01736070215702057,
+      "rewards/margins": 0.007918944582343102,
+      "rewards/rejected": -0.02527964487671852,
+      "step": 64
+    },
+    {
+      "epoch": 0.10613065326633166,
+      "grad_norm": 0.5981670022010803,
+      "learning_rate": 4.999643406399275e-06,
+      "logits/chosen": 1.1232147216796875,
+      "logits/rejected": 1.145438313484192,
+      "logps/chosen": -29.947956085205078,
+      "logps/rejected": -33.95368194580078,
+      "loss": 0.6838,
+      "rewards/accuracies": 0.5703125,
+      "rewards/chosen": -0.018678227439522743,
+      "rewards/margins": 0.019824998453259468,
+      "rewards/rejected": -0.03850322589278221,
+      "step": 66
+    },
+    {
+      "epoch": 0.1093467336683417,
+      "grad_norm": 0.571692705154419,
+      "learning_rate": 4.999009504083738e-06,
+      "logits/chosen": 1.3420298099517822,
+      "logits/rejected": 1.338175654411316,
+      "logps/chosen": -30.924522399902344,
+      "logps/rejected": -37.51594161987305,
+      "loss": 0.6828,
+      "rewards/accuracies": 0.609375,
+      "rewards/chosen": -0.020221160724759102,
+      "rewards/margins": 0.02198150008916855,
+      "rewards/rejected": -0.0422026626765728,
+      "step": 68
+    },
+    {
+      "epoch": 0.11256281407035176,
+      "grad_norm": 0.5207106471061707,
+      "learning_rate": 4.998058751077705e-06,
+      "logits/chosen": 1.3332023620605469,
+      "logits/rejected": 1.3814537525177002,
+      "logps/chosen": -31.7532958984375,
+      "logps/rejected": -34.753013610839844,
+      "loss": 0.6822,
+      "rewards/accuracies": 0.59375,
+      "rewards/chosen": -0.031598228961229324,
+      "rewards/margins": 0.023511648178100586,
+      "rewards/rejected": -0.05510988086462021,
+      "step": 70
+    },
+    {
+      "epoch": 0.11577889447236181,
+      "grad_norm": 0.47247663140296936,
+      "learning_rate": 4.996791267927632e-06,
+      "logits/chosen": 1.3693702220916748,
+      "logits/rejected": 1.3828941583633423,
+      "logps/chosen": -30.8132266998291,
+      "logps/rejected": -35.77416229248047,
+      "loss": 0.6827,
+      "rewards/accuracies": 0.6015625,
+      "rewards/chosen": -0.030194712802767754,
+      "rewards/margins": 0.022484488785266876,
+      "rewards/rejected": -0.05267919972538948,
+      "step": 72
+    },
+    {
+      "epoch": 0.11899497487437186,
+      "grad_norm": 0.48132631182670593,
+      "learning_rate": 4.995207215338358e-06,
+      "logits/chosen": 1.490431785583496,
+      "logits/rejected": 1.5298147201538086,
+      "logps/chosen": -34.107643127441406,
+      "logps/rejected": -34.63951110839844,
+      "loss": 0.687,
+      "rewards/accuracies": 0.515625,
+      "rewards/chosen": -0.036115553230047226,
+      "rewards/margins": 0.014621476642787457,
+      "rewards/rejected": -0.05073703080415726,
+      "step": 74
+    },
+    {
+      "epoch": 0.12221105527638192,
+      "grad_norm": 0.5402019619941711,
+      "learning_rate": 4.9933067941527175e-06,
+      "logits/chosen": 1.3514180183410645,
+      "logits/rejected": 1.3422389030456543,
+      "logps/chosen": -36.069549560546875,
+      "logps/rejected": -33.26152038574219,
+      "loss": 0.6841,
+      "rewards/accuracies": 0.5625,
+      "rewards/chosen": -0.04465916007757187,
+      "rewards/margins": 0.02099396474659443,
+      "rewards/rejected": -0.06565312296152115,
+      "step": 76
+    },
+    {
+      "epoch": 0.12542713567839195,
+      "grad_norm": 0.5146540999412537,
+      "learning_rate": 4.991090245326083e-06,
+      "logits/chosen": 1.0872302055358887,
+      "logits/rejected": 1.1015247106552124,
+      "logps/chosen": -29.906904220581055,
+      "logps/rejected": -38.96195983886719,
+      "loss": 0.6758,
+      "rewards/accuracies": 0.609375,
+      "rewards/chosen": -0.038489095866680145,
+      "rewards/margins": 0.037541136145591736,
+      "rewards/rejected": -0.07603023201227188,
+      "step": 78
+    },
+    {
+      "epoch": 0.12864321608040202,
+      "grad_norm": 0.5551427602767944,
+      "learning_rate": 4.988557849895811e-06,
+      "logits/chosen": 1.3899340629577637,
+      "logits/rejected": 1.4685105085372925,
+      "logps/chosen": -34.21710205078125,
+      "logps/rejected": -38.152099609375,
+      "loss": 0.6695,
+      "rewards/accuracies": 0.65625,
+      "rewards/chosen": -0.03813210129737854,
+      "rewards/margins": 0.05099986493587494,
+      "rewards/rejected": -0.08913196623325348,
+      "step": 80
+    },
+    {
+      "epoch": 0.13185929648241207,
+      "grad_norm": 0.5468294620513916,
+      "learning_rate": 4.9857099289456115e-06,
+      "logits/chosen": 1.4150495529174805,
+      "logits/rejected": 1.4033807516098022,
+      "logps/chosen": -30.979236602783203,
+      "logps/rejected": -36.2646484375,
+      "loss": 0.6759,
+      "rewards/accuracies": 0.6484375,
+      "rewards/chosen": -0.04814925417304039,
+      "rewards/margins": 0.0380784347653389,
+      "rewards/rejected": -0.08622768521308899,
+      "step": 82
+    },
+    {
+      "epoch": 0.13507537688442212,
+      "grad_norm": 0.47936275601387024,
+      "learning_rate": 4.9825468435648345e-06,
+      "logits/chosen": 1.1383363008499146,
+      "logits/rejected": 1.1823409795761108,
+      "logps/chosen": -29.871156692504883,
+      "logps/rejected": -32.21099090576172,
+      "loss": 0.6789,
+      "rewards/accuracies": 0.578125,
+      "rewards/chosen": -0.04363500326871872,
+      "rewards/margins": 0.03290008008480072,
+      "rewards/rejected": -0.07653507590293884,
+      "step": 84
+    },
+    {
+      "epoch": 0.13829145728643216,
+      "grad_norm": 0.48245131969451904,
+      "learning_rate": 4.97906899480269e-06,
+      "logits/chosen": 1.4419957399368286,
+      "logits/rejected": 1.475970983505249,
+      "logps/chosen": -30.77682113647461,
+      "logps/rejected": -35.76774215698242,
+      "loss": 0.6698,
+      "rewards/accuracies": 0.6484375,
+      "rewards/chosen": -0.04197033494710922,
+      "rewards/margins": 0.05154535919427872,
+      "rewards/rejected": -0.09351569414138794,
+      "step": 86
+    },
+    {
+      "epoch": 0.1415075376884422,
+      "grad_norm": 0.5745317935943604,
+      "learning_rate": 4.975276823617399e-06,
+      "logits/chosen": 1.4063524007797241,
+      "logits/rejected": 1.439096450805664,
+      "logps/chosen": -29.64651870727539,
+      "logps/rejected": -35.58737564086914,
+      "loss": 0.6644,
+      "rewards/accuracies": 0.6484375,
+      "rewards/chosen": -0.045151449739933014,
+      "rewards/margins": 0.06304246932268143,
+      "rewards/rejected": -0.10819391906261444,
+      "step": 88
+    },
+    {
+      "epoch": 0.14472361809045226,
+      "grad_norm": 0.5324569344520569,
+      "learning_rate": 4.971170810820279e-06,
+      "logits/chosen": 1.3591543436050415,
+      "logits/rejected": 1.3323087692260742,
+      "logps/chosen": -34.168128967285156,
+      "logps/rejected": -37.837242126464844,
+      "loss": 0.6722,
+      "rewards/accuracies": 0.640625,
+      "rewards/chosen": -0.05879779905080795,
+      "rewards/margins": 0.04740719497203827,
+      "rewards/rejected": -0.10620499402284622,
+      "step": 90
+    },
+    {
+      "epoch": 0.1479396984924623,
+      "grad_norm": 0.6153348684310913,
+      "learning_rate": 4.96675147701479e-06,
+      "logits/chosen": 1.3236520290374756,
+      "logits/rejected": 1.3638286590576172,
+      "logps/chosen": -33.26496887207031,
+      "logps/rejected": -35.09466552734375,
+      "loss": 0.678,
+      "rewards/accuracies": 0.609375,
+      "rewards/chosen": -0.07028309255838394,
+      "rewards/margins": 0.037059590220451355,
+      "rewards/rejected": -0.1073426827788353,
+      "step": 92
+    },
+    {
+      "epoch": 0.15115577889447235,
+      "grad_norm": 0.6804412603378296,
+      "learning_rate": 4.962019382530521e-06,
+      "logits/chosen": 1.1772613525390625,
+      "logits/rejected": 1.2550945281982422,
+      "logps/chosen": -34.816246032714844,
+      "logps/rejected": -34.04191970825195,
+      "loss": 0.676,
+      "rewards/accuracies": 0.5859375,
+      "rewards/chosen": -0.0760345533490181,
+      "rewards/margins": 0.041960567235946655,
+      "rewards/rejected": -0.11799512058496475,
+      "step": 94
+    },
+    {
+      "epoch": 0.15437185929648242,
+      "grad_norm": 0.5287646055221558,
+      "learning_rate": 4.956975127352146e-06,
+      "logits/chosen": 1.1335078477859497,
+      "logits/rejected": 1.0898680686950684,
+      "logps/chosen": -32.298492431640625,
+      "logps/rejected": -38.93742370605469,
+      "loss": 0.6661,
+      "rewards/accuracies": 0.6328125,
+      "rewards/chosen": -0.07041005790233612,
+      "rewards/margins": 0.06344542652368546,
+      "rewards/rejected": -0.13385546207427979,
+      "step": 96
+    },
+    {
+      "epoch": 0.15758793969849247,
+      "grad_norm": 0.5636870265007019,
+      "learning_rate": 4.951619351043353e-06,
+      "logits/chosen": 1.3233567476272583,
+      "logits/rejected": 1.356798768043518,
+      "logps/chosen": -31.532367706298828,
+      "logps/rejected": -33.19044494628906,
+      "loss": 0.6698,
+      "rewards/accuracies": 0.6015625,
+      "rewards/chosen": -0.08084239810705185,
+      "rewards/margins": 0.05414910987019539,
+      "rewards/rejected": -0.13499149680137634,
+      "step": 98
+    },
+    {
+      "epoch": 0.16080402010050251,
+      "grad_norm": 0.5570967197418213,
+      "learning_rate": 4.945952732665755e-06,
+      "logits/chosen": 1.301795244216919,
+      "logits/rejected": 1.3351643085479736,
+      "logps/chosen": -32.771270751953125,
+      "logps/rejected": -41.387725830078125,
+      "loss": 0.6603,
+      "rewards/accuracies": 0.6015625,
+      "rewards/chosen": -0.07904825359582901,
+      "rewards/margins": 0.07616038620471954,
+      "rewards/rejected": -0.15520863234996796,
+      "step": 100
+    },
+    {
+      "epoch": 0.16080402010050251,
+      "eval_logits/chosen": 0.5383610725402832,
+      "eval_logits/rejected": 0.544625997543335,
+      "eval_logps/chosen": -31.30695343017578,
+      "eval_logps/rejected": -36.894073486328125,
+      "eval_loss": 0.6592575311660767,
+      "eval_rewards/accuracies": 0.6850000023841858,
+      "eval_rewards/chosen": -0.08062513917684555,
+      "eval_rewards/margins": 0.07633791118860245,
+      "eval_rewards/rejected": -0.156963050365448,
+      "eval_runtime": 48.1277,
+      "eval_samples_per_second": 4.156,
+      "eval_steps_per_second": 1.039,
+      "step": 100
+    },
+    {
+      "epoch": 0.16402010050251256,
+      "grad_norm": 0.56045001745224,
+      "learning_rate": 4.939975990692789e-06,
+      "logits/chosen": 1.2697747945785522,
+      "logits/rejected": 1.2618439197540283,
+      "logps/chosen": -32.927146911621094,
+      "logps/rejected": -35.013206481933594,
+      "loss": 0.6671,
+      "rewards/accuracies": 0.65625,
+      "rewards/chosen": -0.08042748272418976,
+      "rewards/margins": 0.060588739812374115,
+      "rewards/rejected": -0.14101621508598328,
+      "step": 102
+    },
+    {
+      "epoch": 0.1672361809045226,
+      "grad_norm": 0.5901036858558655,
+      "learning_rate": 4.9336898829186185e-06,
+      "logits/chosen": 1.0983755588531494,
+      "logits/rejected": 1.1659495830535889,
+      "logps/chosen": -34.55509567260742,
+      "logps/rejected": -37.56676483154297,
+      "loss": 0.6575,
+      "rewards/accuracies": 0.671875,
+      "rewards/chosen": -0.08760575950145721,
+      "rewards/margins": 0.08133453875780106,
+      "rewards/rejected": -0.16894027590751648,
+      "step": 104
+    },
+    {
+      "epoch": 0.17045226130653265,
+      "grad_norm": 0.7394624352455139,
+      "learning_rate": 4.927095206362057e-06,
+      "logits/chosen": 1.1626837253570557,
+      "logits/rejected": 1.1353670358657837,
+      "logps/chosen": -33.66203308105469,
+      "logps/rejected": -35.955352783203125,
+      "loss": 0.6693,
+      "rewards/accuracies": 0.6328125,
+      "rewards/chosen": -0.10034358501434326,
+      "rewards/margins": 0.05721643567085266,
+      "rewards/rejected": -0.15756002068519592,
+      "step": 106
+    },
+    {
+      "epoch": 0.1736683417085427,
+      "grad_norm": 0.7603115439414978,
+      "learning_rate": 4.920192797165511e-06,
+      "logits/chosen": 1.2644569873809814,
+      "logits/rejected": 1.2957494258880615,
+      "logps/chosen": -33.66388702392578,
+      "logps/rejected": -33.66148376464844,
+      "loss": 0.6784,
+      "rewards/accuracies": 0.5703125,
+      "rewards/chosen": -0.11911429464817047,
+      "rewards/margins": 0.039303235709667206,
+      "rewards/rejected": -0.15841752290725708,
+      "step": 108
+    },
+    {
+      "epoch": 0.17688442211055277,
+      "grad_norm": 0.5353249907493591,
+      "learning_rate": 4.912983530488966e-06,
+      "logits/chosen": 1.4405113458633423,
+      "logits/rejected": 1.4680776596069336,
+      "logps/chosen": -32.20611572265625,
+      "logps/rejected": -35.25544738769531,
+      "loss": 0.6688,
+      "rewards/accuracies": 0.609375,
+      "rewards/chosen": -0.1222277581691742,
+      "rewards/margins": 0.056810542941093445,
+      "rewards/rejected": -0.17903831601142883,
+      "step": 110
+    },
+    {
+      "epoch": 0.18010050251256282,
+      "grad_norm": 0.7354335784912109,
+      "learning_rate": 4.905468320399023e-06,
+      "logits/chosen": 1.2397936582565308,
+      "logits/rejected": 1.2630605697631836,
+      "logps/chosen": -29.642383575439453,
+      "logps/rejected": -37.407413482666016,
+      "loss": 0.6483,
+      "rewards/accuracies": 0.71875,
+      "rewards/chosen": -0.09827758371829987,
+      "rewards/margins": 0.1014958843588829,
+      "rewards/rejected": -0.19977346062660217,
+      "step": 112
+    },
+    {
+      "epoch": 0.18331658291457287,
+      "grad_norm": 0.6473943591117859,
+      "learning_rate": 4.897648119753006e-06,
+      "logits/chosen": 1.752604603767395,
+      "logits/rejected": 1.7992733716964722,
+      "logps/chosen": -34.81816101074219,
+      "logps/rejected": -34.017303466796875,
+      "loss": 0.6765,
+      "rewards/accuracies": 0.5546875,
+      "rewards/chosen": -0.11875072121620178,
+      "rewards/margins": 0.04180861636996269,
+      "rewards/rejected": -0.16055932641029358,
+      "step": 114
+    },
+    {
+      "epoch": 0.18653266331658291,
+      "grad_norm": 0.7461528778076172,
+      "learning_rate": 4.889523920078144e-06,
+      "logits/chosen": 1.528036117553711,
+      "logits/rejected": 1.5142472982406616,
+      "logps/chosen": -29.153812408447266,
+      "logps/rejected": -37.897003173828125,
+      "loss": 0.6421,
+      "rewards/accuracies": 0.7265625,
+      "rewards/chosen": -0.11313571035861969,
+      "rewards/margins": 0.1143951416015625,
+      "rewards/rejected": -0.227530837059021,
+      "step": 116
+    },
+    {
+      "epoch": 0.18974874371859296,
+      "grad_norm": 0.6040567755699158,
+      "learning_rate": 4.881096751445863e-06,
+      "logits/chosen": 1.0826690196990967,
+      "logits/rejected": 1.132314682006836,
+      "logps/chosen": -34.31199645996094,
+      "logps/rejected": -37.10968017578125,
+      "loss": 0.649,
+      "rewards/accuracies": 0.6875,
+      "rewards/chosen": -0.11125093698501587,
+      "rewards/margins": 0.09869204461574554,
+      "rewards/rejected": -0.2099429816007614,
+      "step": 118
+    },
+    {
+      "epoch": 0.192964824120603,
+      "grad_norm": 0.6605610251426697,
+      "learning_rate": 4.8723676823411734e-06,
+      "logits/chosen": 1.3900730609893799,
+      "logits/rejected": 1.399127721786499,
+      "logps/chosen": -31.501989364624023,
+      "logps/rejected": -38.05152893066406,
+      "loss": 0.6518,
+      "rewards/accuracies": 0.7265625,
+      "rewards/chosen": -0.12589490413665771,
+      "rewards/margins": 0.09352517873048782,
+      "rewards/rejected": -0.21942007541656494,
+      "step": 120
+    },
+    {
+      "epoch": 0.19618090452261305,
+      "grad_norm": 0.7710764408111572,
+      "learning_rate": 4.8633378195272015e-06,
+      "logits/chosen": 1.6575126647949219,
+      "logits/rejected": 1.6922370195388794,
+      "logps/chosen": -31.622278213500977,
+      "logps/rejected": -34.734527587890625,
+      "loss": 0.6565,
+      "rewards/accuracies": 0.65625,
+      "rewards/chosen": -0.12990064918994904,
+      "rewards/margins": 0.08599180728197098,
+      "rewards/rejected": -0.2158924639225006,
+      "step": 122
+    },
+    {
+      "epoch": 0.19939698492462313,
+      "grad_norm": 0.5700326561927795,
+      "learning_rate": 4.854008307904865e-06,
+      "logits/chosen": 1.2142469882965088,
+      "logits/rejected": 1.2496553659439087,
+      "logps/chosen": -31.798629760742188,
+      "logps/rejected": -38.08250045776367,
+      "loss": 0.6458,
+      "rewards/accuracies": 0.6875,
+      "rewards/chosen": -0.11889669299125671,
+      "rewards/margins": 0.10755539685487747,
+      "rewards/rejected": -0.22645209729671478,
+      "step": 124
+    },
+    {
+      "epoch": 0.20261306532663317,
+      "grad_norm": 0.7354090213775635,
+      "learning_rate": 4.844380330367701e-06,
+      "logits/chosen": 1.4388645887374878,
+      "logits/rejected": 1.4413161277770996,
+      "logps/chosen": -28.97043228149414,
+      "logps/rejected": -37.475364685058594,
+      "loss": 0.6321,
+      "rewards/accuracies": 0.7421875,
+      "rewards/chosen": -0.10781995952129364,
+      "rewards/margins": 0.136630117893219,
+      "rewards/rejected": -0.24445009231567383,
+      "step": 126
+    },
+    {
+      "epoch": 0.20582914572864322,
+      "grad_norm": 0.6026410460472107,
+      "learning_rate": 4.8344551076518985e-06,
+      "logits/chosen": 1.167945146560669,
+      "logits/rejected": 1.181955099105835,
+      "logps/chosen": -33.69917297363281,
+      "logps/rejected": -41.300926208496094,
+      "loss": 0.6472,
+      "rewards/accuracies": 0.6484375,
+      "rewards/chosen": -0.11048279702663422,
+      "rewards/margins": 0.10788710415363312,
+      "rewards/rejected": -0.21836990118026733,
+      "step": 128
+    },
+    {
+      "epoch": 0.20904522613065327,
+      "grad_norm": 0.6147257089614868,
+      "learning_rate": 4.824233898181509e-06,
+      "logits/chosen": 1.3260626792907715,
+      "logits/rejected": 1.3723877668380737,
+      "logps/chosen": -32.34976577758789,
+      "logps/rejected": -36.80198669433594,
+      "loss": 0.6356,
+      "rewards/accuracies": 0.6875,
+      "rewards/chosen": -0.13550475239753723,
+      "rewards/margins": 0.13105657696723938,
+      "rewards/rejected": -0.2665613293647766,
+      "step": 130
+    },
+    {
+      "epoch": 0.2122613065326633,
+      "grad_norm": 0.8525822162628174,
+      "learning_rate": 4.8137179979088995e-06,
+      "logits/chosen": 1.1474467515945435,
+      "logits/rejected": 1.179036259651184,
+      "logps/chosen": -32.16166305541992,
+      "logps/rejected": -35.32681655883789,
+      "loss": 0.6498,
+      "rewards/accuracies": 0.6171875,
+      "rewards/chosen": -0.1576756238937378,
+      "rewards/margins": 0.10601034015417099,
+      "rewards/rejected": -0.2636859714984894,
+      "step": 132
+    },
+    {
+      "epoch": 0.21547738693467336,
+      "grad_norm": 0.6118230223655701,
+      "learning_rate": 4.8029087401504315e-06,
+      "logits/chosen": 1.3549935817718506,
+      "logits/rejected": 1.4064905643463135,
+      "logps/chosen": -32.3475456237793,
+      "logps/rejected": -36.25346374511719,
+      "loss": 0.635,
+      "rewards/accuracies": 0.7265625,
+      "rewards/chosen": -0.11831659078598022,
+      "rewards/margins": 0.13686415553092957,
+      "rewards/rejected": -0.2551807463169098,
+      "step": 134
+    },
+    {
+      "epoch": 0.2186934673366834,
+      "grad_norm": 0.6388143301010132,
+      "learning_rate": 4.791807495417415e-06,
+      "logits/chosen": 1.3601171970367432,
+      "logits/rejected": 1.3299559354782104,
+      "logps/chosen": -33.817562103271484,
+      "logps/rejected": -42.62956237792969,
+      "loss": 0.6258,
+      "rewards/accuracies": 0.7265625,
+      "rewards/chosen": -0.1584807187318802,
+      "rewards/margins": 0.1535823941230774,
+      "rewards/rejected": -0.3120630979537964,
+      "step": 136
+    },
+    {
+      "epoch": 0.22190954773869348,
+      "grad_norm": 0.6631596088409424,
+      "learning_rate": 4.780415671242334e-06,
+      "logits/chosen": 1.1997196674346924,
+      "logits/rejected": 1.24537193775177,
+      "logps/chosen": -31.997718811035156,
+      "logps/rejected": -38.69951248168945,
+      "loss": 0.6454,
+      "rewards/accuracies": 0.703125,
+      "rewards/chosen": -0.1721266806125641,
+      "rewards/margins": 0.11670082062482834,
+      "rewards/rejected": -0.288827508687973,
+      "step": 138
+    },
+    {
+      "epoch": 0.22512562814070353,
+      "grad_norm": 1.039876103401184,
+      "learning_rate": 4.76873471200039e-06,
+      "logits/chosen": 1.4914782047271729,
+      "logits/rejected": 1.5159029960632324,
+      "logps/chosen": -31.84731101989746,
+      "logps/rejected": -38.26771545410156,
+      "loss": 0.6436,
+      "rewards/accuracies": 0.703125,
+      "rewards/chosen": -0.16641682386398315,
+      "rewards/margins": 0.12111803144216537,
+      "rewards/rejected": -0.2875348627567291,
+      "step": 140
+    },
+    {
+      "epoch": 0.22834170854271357,
+      "grad_norm": 0.7036734223365784,
+      "learning_rate": 4.756766098726368e-06,
+      "logits/chosen": 1.2412619590759277,
+      "logits/rejected": 1.2428562641143799,
+      "logps/chosen": -32.30570602416992,
+      "logps/rejected": -37.51960754394531,
+      "loss": 0.6218,
+      "rewards/accuracies": 0.734375,
+      "rewards/chosen": -0.15528711676597595,
+      "rewards/margins": 0.1665349304676056,
+      "rewards/rejected": -0.32182204723358154,
+      "step": 142
+    },
+    {
+      "epoch": 0.23155778894472362,
+      "grad_norm": 0.7187156081199646,
+      "learning_rate": 4.744511348926855e-06,
+      "logits/chosen": 1.284571647644043,
+      "logits/rejected": 1.3178225755691528,
+      "logps/chosen": -31.669775009155273,
+      "logps/rejected": -37.961936950683594,
+      "loss": 0.6329,
+      "rewards/accuracies": 0.6328125,
+      "rewards/chosen": -0.19643092155456543,
+      "rewards/margins": 0.15072323381900787,
+      "rewards/rejected": -0.3471541404724121,
+      "step": 144
+    },
+    {
+      "epoch": 0.23477386934673367,
+      "grad_norm": 0.8957286477088928,
+      "learning_rate": 4.73197201638783e-06,
+      "logits/chosen": 1.2843809127807617,
+      "logits/rejected": 1.2913951873779297,
+      "logps/chosen": -33.99455261230469,
+      "logps/rejected": -38.29543685913086,
+      "loss": 0.6243,
+      "rewards/accuracies": 0.734375,
+      "rewards/chosen": -0.16770941019058228,
+      "rewards/margins": 0.16542242467403412,
+      "rewards/rejected": -0.3331318199634552,
+      "step": 146
+    },
+    {
+      "epoch": 0.2379899497487437,
+      "grad_norm": 0.6802579164505005,
+      "learning_rate": 4.7191496909776665e-06,
+      "logits/chosen": 1.3031625747680664,
+      "logits/rejected": 1.318772554397583,
+      "logps/chosen": -31.817935943603516,
+      "logps/rejected": -38.92062759399414,
+      "loss": 0.6381,
+      "rewards/accuracies": 0.65625,
+      "rewards/chosen": -0.16274359822273254,
+      "rewards/margins": 0.13347209990024567,
+      "rewards/rejected": -0.2962157130241394,
+      "step": 148
+    },
+    {
+      "epoch": 0.24120603015075376,
+      "grad_norm": 0.8087608218193054,
+      "learning_rate": 4.706045998445548e-06,
+      "logits/chosen": 1.1990268230438232,
+      "logits/rejected": 1.19586181640625,
+      "logps/chosen": -35.0596809387207,
+      "logps/rejected": -38.830718994140625,
+      "loss": 0.6387,
+      "rewards/accuracies": 0.6484375,
+      "rewards/chosen": -0.20017872750759125,
+      "rewards/margins": 0.1404918134212494,
+      "rewards/rejected": -0.3406705856323242,
+      "step": 150
+    },
+    {
+      "epoch": 0.24120603015075376,
+      "eval_logits/chosen": 0.505786657333374,
+      "eval_logits/rejected": 0.5080003142356873,
+      "eval_logps/chosen": -32.41748046875,
+      "eval_logps/rejected": -38.779293060302734,
+      "eval_loss": 0.6298141479492188,
+      "eval_rewards/accuracies": 0.7249999642372131,
+      "eval_rewards/chosen": -0.19167786836624146,
+      "eval_rewards/margins": 0.15380695462226868,
+      "eval_rewards/rejected": -0.3454848527908325,
+      "eval_runtime": 48.1694,
+      "eval_samples_per_second": 4.152,
+      "eval_steps_per_second": 1.038,
+      "step": 150
+    },
+    {
+      "epoch": 0.24442211055276383,
+      "grad_norm": 0.6448603272438049,
+      "learning_rate": 4.692662600215339e-06,
+      "logits/chosen": 1.2040199041366577,
+      "logits/rejected": 1.2359912395477295,
+      "logps/chosen": -31.330707550048828,
+      "logps/rejected": -37.42987060546875,
+      "loss": 0.6424,
+      "rewards/accuracies": 0.671875,
+      "rewards/chosen": -0.20019026100635529,
+      "rewards/margins": 0.1243181824684143,
+      "rewards/rejected": -0.3245084285736084,
+      "step": 152
+    },
+    {
+      "epoch": 0.24763819095477388,
+      "grad_norm": 0.7376160025596619,
+      "learning_rate": 4.679001193174931e-06,
+      "logits/chosen": 1.1904234886169434,
+      "logits/rejected": 1.2631607055664062,
+      "logps/chosen": -35.694915771484375,
+      "logps/rejected": -39.23067855834961,
+      "loss": 0.6407,
+      "rewards/accuracies": 0.6640625,
+      "rewards/chosen": -0.20610052347183228,
+      "rewards/margins": 0.13499653339385986,
+      "rewards/rejected": -0.34109705686569214,
+      "step": 154
+    },
+    {
+      "epoch": 0.2508542713567839,
+      "grad_norm": 0.848278284072876,
+      "learning_rate": 4.665063509461098e-06,
+      "logits/chosen": 1.4529380798339844,
+      "logits/rejected": 1.4410022497177124,
+      "logps/chosen": -33.822208404541016,
+      "logps/rejected": -37.43674850463867,
+      "loss": 0.6339,
+      "rewards/accuracies": 0.6953125,
+      "rewards/chosen": -0.2073659896850586,
+      "rewards/margins": 0.14442923665046692,
+      "rewards/rejected": -0.3517952263355255,
+      "step": 156
+    },
+    {
+      "epoch": 0.25407035175879394,
+      "grad_norm": 0.7130846381187439,
+      "learning_rate": 4.650851316239867e-06,
+      "logits/chosen": 1.2848093509674072,
+      "logits/rejected": 1.3309288024902344,
+      "logps/chosen": -33.921539306640625,
+      "logps/rejected": -36.744384765625,
+      "loss": 0.6192,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": -0.1995392143726349,
+      "rewards/margins": 0.17559002339839935,
+      "rewards/rejected": -0.37512922286987305,
+      "step": 158
+    },
+    {
+      "epoch": 0.25728643216080405,
+      "grad_norm": 0.889614462852478,
+      "learning_rate": 4.636366415482474e-06,
+      "logits/chosen": 1.4534492492675781,
+      "logits/rejected": 1.4741634130477905,
+      "logps/chosen": -37.4984130859375,
+      "logps/rejected": -40.52190017700195,
+      "loss": 0.6113,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": -0.2035304307937622,
+      "rewards/margins": 0.19503936171531677,
+      "rewards/rejected": -0.39856982231140137,
+      "step": 160
+    },
+    {
+      "epoch": 0.2605025125628141,
+      "grad_norm": 0.797125518321991,
+      "learning_rate": 4.621610643736878e-06,
+      "logits/chosen": 1.1096376180648804,
+      "logits/rejected": 1.1337332725524902,
+      "logps/chosen": -33.37801742553711,
+      "logps/rejected": -37.59724426269531,
+      "loss": 0.6226,
+      "rewards/accuracies": 0.7421875,
+      "rewards/chosen": -0.23320314288139343,
+      "rewards/margins": 0.17068544030189514,
+      "rewards/rejected": -0.4038885831832886,
+      "step": 162
+    },
+    {
+      "epoch": 0.26371859296482414,
+      "grad_norm": 0.6749808192253113,
+      "learning_rate": 4.60658587189491e-06,
+      "logits/chosen": 1.2708582878112793,
+      "logits/rejected": 1.2609119415283203,
+      "logps/chosen": -34.26423645019531,
+      "logps/rejected": -38.192108154296875,
+      "loss": 0.6222,
+      "rewards/accuracies": 0.6875,
+      "rewards/chosen": -0.19967004656791687,
+      "rewards/margins": 0.1713426113128662,
+      "rewards/rejected": -0.3710126280784607,
+      "step": 164
+    },
+    {
+      "epoch": 0.2669346733668342,
+      "grad_norm": 0.7875650525093079,
+      "learning_rate": 4.5912940049550596e-06,
+      "logits/chosen": 1.5831212997436523,
+      "logits/rejected": 1.5837812423706055,
+      "logps/chosen": -31.933712005615234,
+      "logps/rejected": -40.272857666015625,
+      "loss": 0.6043,
+      "rewards/accuracies": 0.71875,
+      "rewards/chosen": -0.22415216267108917,
+      "rewards/margins": 0.22287428379058838,
+      "rewards/rejected": -0.44702646136283875,
+      "step": 166
+    },
+    {
+      "epoch": 0.27015075376884423,
+      "grad_norm": 0.7695869207382202,
+      "learning_rate": 4.575736981780942e-06,
+      "logits/chosen": 1.2019391059875488,
+      "logits/rejected": 1.1937005519866943,
+      "logps/chosen": -33.98455810546875,
+      "logps/rejected": -36.95637512207031,
+      "loss": 0.6069,
+      "rewards/accuracies": 0.7109375,
+      "rewards/chosen": -0.20682327449321747,
+      "rewards/margins": 0.2069566249847412,
+      "rewards/rejected": -0.4137799143791199,
+      "step": 168
+    },
+    {
+      "epoch": 0.2733668341708543,
+      "grad_norm": 18.707008361816406,
+      "learning_rate": 4.559916774855464e-06,
+      "logits/chosen": 1.1806917190551758,
+      "logits/rejected": 1.175247311592102,
+      "logps/chosen": -31.461313247680664,
+      "logps/rejected": -45.7471923828125,
+      "loss": 0.6132,
+      "rewards/accuracies": 0.8046875,
+      "rewards/chosen": -0.1899334043264389,
+      "rewards/margins": 0.21620748937129974,
+      "rewards/rejected": -0.40614089369773865,
+      "step": 170
+    },
+    {
+      "epoch": 0.2765829145728643,
+      "grad_norm": 0.753237783908844,
+      "learning_rate": 4.543835390030738e-06,
+      "logits/chosen": 1.1212278604507446,
+      "logits/rejected": 1.1133912801742554,
+      "logps/chosen": -33.20658874511719,
+      "logps/rejected": -39.882999420166016,
+      "loss": 0.5982,
+      "rewards/accuracies": 0.7109375,
+      "rewards/chosen": -0.1884865164756775,
+      "rewards/margins": 0.23654288053512573,
+      "rewards/rejected": -0.4250293970108032,
+      "step": 172
+    },
+    {
+      "epoch": 0.27979899497487437,
+      "grad_norm": 0.8276423811912537,
+      "learning_rate": 4.5274948662737535e-06,
+      "logits/chosen": 1.5256576538085938,
+      "logits/rejected": 1.535451054573059,
+      "logps/chosen": -31.11783218383789,
+      "logps/rejected": -39.00643539428711,
+      "loss": 0.629,
+      "rewards/accuracies": 0.640625,
+      "rewards/chosen": -0.23714976012706757,
+      "rewards/margins": 0.16218701004981995,
+      "rewards/rejected": -0.3993367850780487,
+      "step": 174
+    },
+    {
+      "epoch": 0.2830150753768844,
+      "grad_norm": 0.9968907833099365,
+      "learning_rate": 4.510897275407856e-06,
+      "logits/chosen": 1.174997091293335,
+      "logits/rejected": 1.2027724981307983,
+      "logps/chosen": -35.436641693115234,
+      "logps/rejected": -38.52527618408203,
+      "loss": 0.603,
+      "rewards/accuracies": 0.7578125,
+      "rewards/chosen": -0.2507714629173279,
+      "rewards/margins": 0.22110292315483093,
+      "rewards/rejected": -0.4718744158744812,
+      "step": 176
+    },
+    {
+      "epoch": 0.28623115577889446,
+      "grad_norm": 0.8604883551597595,
+      "learning_rate": 4.494044721850066e-06,
+      "logits/chosen": 1.4382469654083252,
+      "logits/rejected": 1.4571839570999146,
+      "logps/chosen": -36.266475677490234,
+      "logps/rejected": -40.861724853515625,
+      "loss": 0.5918,
+      "rewards/accuracies": 0.765625,
+      "rewards/chosen": -0.22194752097129822,
+      "rewards/margins": 0.2503468990325928,
+      "rewards/rejected": -0.472294420003891,
+      "step": 178
+    },
+    {
+      "epoch": 0.2894472361809045,
+      "grad_norm": 0.8287671208381653,
+      "learning_rate": 4.476939342344246e-06,
+      "logits/chosen": 0.9575952291488647,
+      "logits/rejected": 0.9769454598426819,
+      "logps/chosen": -33.822174072265625,
+      "logps/rejected": -40.31230545043945,
+      "loss": 0.6002,
+      "rewards/accuracies": 0.71875,
+      "rewards/chosen": -0.23980066180229187,
+      "rewards/margins": 0.24076348543167114,
+      "rewards/rejected": -0.480564147233963,
+      "step": 180
+    },
+    {
+      "epoch": 0.29266331658291456,
+      "grad_norm": 0.9954908490180969,
+      "learning_rate": 4.459583305690198e-06,
+      "logits/chosen": 1.1955639123916626,
+      "logits/rejected": 1.1979701519012451,
+      "logps/chosen": -35.99625015258789,
+      "logps/rejected": -42.92171096801758,
+      "loss": 0.6081,
+      "rewards/accuracies": 0.703125,
+      "rewards/chosen": -0.2439308762550354,
+      "rewards/margins": 0.21916835010051727,
+      "rewards/rejected": -0.4630991816520691,
+      "step": 182
+    },
+    {
+      "epoch": 0.2958793969849246,
+      "grad_norm": 0.9786974191665649,
+      "learning_rate": 4.4419788124686665e-06,
+      "logits/chosen": 1.3801398277282715,
+      "logits/rejected": 1.347995638847351,
+      "logps/chosen": -34.530696868896484,
+      "logps/rejected": -41.12428283691406,
+      "loss": 0.6282,
+      "rewards/accuracies": 0.6953125,
+      "rewards/chosen": -0.23650634288787842,
+      "rewards/margins": 0.18922509253025055,
+      "rewards/rejected": -0.4257314205169678,
+      "step": 184
+    },
+    {
+      "epoch": 0.29909547738693465,
+      "grad_norm": 10.048331260681152,
+      "learning_rate": 4.424128094762331e-06,
+      "logits/chosen": 1.2418086528778076,
+      "logits/rejected": 1.2690742015838623,
+      "logps/chosen": -34.209205627441406,
+      "logps/rejected": -43.19721221923828,
+      "loss": 0.6351,
+      "rewards/accuracies": 0.6875,
+      "rewards/chosen": -0.2699955105781555,
+      "rewards/margins": 0.17394134402275085,
+      "rewards/rejected": -0.443936824798584,
+      "step": 186
+    },
+    {
+      "epoch": 0.3023115577889447,
+      "grad_norm": 1.1740622520446777,
+      "learning_rate": 4.406033415872801e-06,
+      "logits/chosen": 1.211313247680664,
+      "logits/rejected": 1.2540022134780884,
+      "logps/chosen": -30.689380645751953,
+      "logps/rejected": -41.284637451171875,
+      "loss": 0.5942,
+      "rewards/accuracies": 0.7421875,
+      "rewards/chosen": -0.23609167337417603,
+      "rewards/margins": 0.25958290696144104,
+      "rewards/rejected": -0.49567458033561707,
+      "step": 188
+    },
+    {
+      "epoch": 0.3055276381909548,
+      "grad_norm": 0.7530081868171692,
+      "learning_rate": 4.387697070033649e-06,
+      "logits/chosen": 1.691462516784668,
+      "logits/rejected": 1.7449442148208618,
+      "logps/chosen": -34.56097412109375,
+      "logps/rejected": -40.92518615722656,
+      "loss": 0.5942,
+      "rewards/accuracies": 0.7421875,
+      "rewards/chosen": -0.22781601548194885,
+      "rewards/margins": 0.24525202810764313,
+      "rewards/rejected": -0.4730680286884308,
+      "step": 190
+    },
+    {
+      "epoch": 0.30874371859296484,
+      "grad_norm": 0.9380214810371399,
+      "learning_rate": 4.369121382119523e-06,
+      "logits/chosen": 1.3881587982177734,
+      "logits/rejected": 1.4654074907302856,
+      "logps/chosen": -35.41065979003906,
+      "logps/rejected": -40.2274284362793,
+      "loss": 0.5986,
+      "rewards/accuracies": 0.7109375,
+      "rewards/chosen": -0.28155750036239624,
+      "rewards/margins": 0.2501455545425415,
+      "rewards/rejected": -0.531702995300293,
+      "step": 192
+    },
+    {
+      "epoch": 0.3119597989949749,
+      "grad_norm": 0.977056622505188,
+      "learning_rate": 4.350308707351373e-06,
+      "logits/chosen": 1.2062150239944458,
+      "logits/rejected": 1.2337925434112549,
+      "logps/chosen": -33.4559211730957,
+      "logps/rejected": -39.458404541015625,
+      "loss": 0.5739,
+      "rewards/accuracies": 0.796875,
+      "rewards/chosen": -0.22728925943374634,
+      "rewards/margins": 0.30084502696990967,
+      "rewards/rejected": -0.5281342267990112,
+      "step": 194
+    },
+    {
+      "epoch": 0.31517587939698494,
+      "grad_norm": 0.8578616380691528,
+      "learning_rate": 4.331261430997835e-06,
+      "logits/chosen": 1.1321465969085693,
+      "logits/rejected": 1.1912870407104492,
+      "logps/chosen": -35.213966369628906,
+      "logps/rejected": -40.169456481933594,
+      "loss": 0.5716,
+      "rewards/accuracies": 0.765625,
+      "rewards/chosen": -0.21415600180625916,
+      "rewards/margins": 0.31074848771095276,
+      "rewards/rejected": -0.5249044895172119,
+      "step": 196
+    },
+    {
+      "epoch": 0.318391959798995,
+      "grad_norm": 0.739769458770752,
+      "learning_rate": 4.3119819680728e-06,
+      "logits/chosen": 1.0423147678375244,
+      "logits/rejected": 1.0700112581253052,
+      "logps/chosen": -34.76524353027344,
+      "logps/rejected": -39.74441909790039,
+      "loss": 0.6,
+      "rewards/accuracies": 0.734375,
+      "rewards/chosen": -0.22830314934253693,
+      "rewards/margins": 0.2578757107257843,
+      "rewards/rejected": -0.48617884516716003,
+      "step": 198
+    },
+    {
+      "epoch": 0.32160804020100503,
+      "grad_norm": 1.007567286491394,
+      "learning_rate": 4.292472763029213e-06,
+      "logits/chosen": 1.282730221748352,
+      "logits/rejected": 1.3393166065216064,
+      "logps/chosen": -31.768949508666992,
+      "logps/rejected": -36.599937438964844,
+      "loss": 0.5986,
+      "rewards/accuracies": 0.734375,
+      "rewards/chosen": -0.23056712746620178,
+      "rewards/margins": 0.24101990461349487,
+      "rewards/rejected": -0.47158706188201904,
+      "step": 200
+    },
+    {
+      "epoch": 0.32160804020100503,
+      "eval_logits/chosen": 0.48468613624572754,
+      "eval_logits/rejected": 0.48436060547828674,
+      "eval_logps/chosen": -32.83020782470703,
+      "eval_logps/rejected": -40.13883590698242,
+      "eval_loss": 0.5988396406173706,
+      "eval_rewards/accuracies": 0.7049999833106995,
+      "eval_rewards/chosen": -0.23295056819915771,
+      "eval_rewards/margins": 0.2484888881444931,
+      "eval_rewards/rejected": -0.481439471244812,
+      "eval_runtime": 48.1896,
+      "eval_samples_per_second": 4.15,
+      "eval_steps_per_second": 1.038,
+      "step": 200
+    }
+  ],
+  "logging_steps": 2,
+  "max_steps": 621,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.140837960442184e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcf7d186e3568d77f55778a26c607e4da9c0a96dd9d37a4828aaf8e3780e13fb
+size 5752

checkpoint-300/README.md

@@ -0,0 +1,208 @@
+---
+base_model: /p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct
+- llama-factory
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1

checkpoint-300/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/p/scratch/taco-vlm/xiao4/models/Qwen2.5-VL-7B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

checkpoint-300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a178154b2410189b0e8d6a87279fdfd036d38473a9ab77c575cafd58b1bd625
+size 40385208

checkpoint-300/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-300/chat_template.json

@@ -0,0 +1,3 @@
+{
+  "chat_template": "{% set image_count = namespace(value=0) %}{% set video_count = namespace(value=0) %}{% for message in messages %}{% if loop.first and message['role'] != 'system' %}<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n{% endif %}<|im_start|>{{ message['role'] }}\n{% if message['content'] is string %}{{ message['content'] }}<|im_end|>\n{% else %}{% for content in message['content'] %}{% if content['type'] == 'image' or 'image' in content or 'image_url' in content %}{% set image_count.value = image_count.value + 1 %}{% if add_vision_id %}Picture {{ image_count.value }}: {% endif %}<|vision_start|><|image_pad|><|vision_end|>{% elif content['type'] == 'video' or 'video' in content %}{% set video_count.value = video_count.value + 1 %}{% if add_vision_id %}Video {{ video_count.value }}: {% endif %}<|vision_start|><|video_pad|><|vision_end|>{% elif 'text' in content %}{{ content['text'] }}{% endif %}{% endfor %}<|im_end|>\n{% endif %}{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant\n{% endif %}"
+}

checkpoint-300/preprocessor_config.json

@@ -0,0 +1,32 @@
+{
+  "crop_size": null,
+  "default_to_square": true,
+  "do_center_crop": null,
+  "do_convert_rgb": true,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_processor_type": "Qwen2VLImageProcessorFast",
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "max_pixels": 12845056,
+  "merge_size": 2,
+  "min_pixels": 3136,
+  "patch_size": 14,
+  "processor_class": "Qwen2_5_VLProcessor",
+  "resample": 3,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "longest_edge": 1003520,
+    "shortest_edge": 3136
+  },
+  "temporal_patch_size": 2
+}

checkpoint-300/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef002048764051a71fb00f8f978e9ec32b780dc850bdb059af362cc56494234b
+size 15024