Training in progress, step 150, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: NousResearch/GPT4-x-Vicuna-13b-fp16
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "NousResearch/GPT4-x-Vicuna-13b-fp16",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_dropout": 0.3,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj",
+    "down_proj",
+    "k_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:973a642ce8dd0811bde345daed173355aa4b81061449f56a31f75352b3c86d8d
+size 1001465824

last-checkpoint/added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "[PAD]": 32000
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10541ce8023d5d7b4dd8b1cd8be0bc87e4c207f77eec04a574375d5cad945c7e
+size 509176980

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fac3710ef7ffe30b6c2b32528c44008b152bab173e59dddec40227ce311dc440
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f3d7d50f8f1895fe1249332d7197df475e97e2eff47be6e1c6deb05367a502b
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
+size 499723

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,52 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "add_prefix_space": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "32000": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "padding_side": "right",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1108 @@
+{
+  "best_metric": 0.8442216515541077,
+  "best_model_checkpoint": "miner_id_24/checkpoint-150",
+  "epoch": 0.0512952038984355,
+  "eval_steps": 150,
+  "global_step": 150,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.00034196802598957,
+      "grad_norm": 1.149609088897705,
+      "learning_rate": 2e-05,
+      "loss": 1.4571,
+      "step": 1
+    },
+    {
+      "epoch": 0.00034196802598957,
+      "eval_loss": 1.6692867279052734,
+      "eval_runtime": 229.1785,
+      "eval_samples_per_second": 5.376,
+      "eval_steps_per_second": 2.688,
+      "step": 1
+    },
+    {
+      "epoch": 0.00068393605197914,
+      "grad_norm": 0.9893769025802612,
+      "learning_rate": 4e-05,
+      "loss": 1.0747,
+      "step": 2
+    },
+    {
+      "epoch": 0.0010259040779687098,
+      "grad_norm": 0.9581325650215149,
+      "learning_rate": 6e-05,
+      "loss": 0.9225,
+      "step": 3
+    },
+    {
+      "epoch": 0.00136787210395828,
+      "grad_norm": 0.7733741402626038,
+      "learning_rate": 8e-05,
+      "loss": 1.0066,
+      "step": 4
+    },
+    {
+      "epoch": 0.0017098401299478498,
+      "grad_norm": 1.051627516746521,
+      "learning_rate": 0.0001,
+      "loss": 1.0399,
+      "step": 5
+    },
+    {
+      "epoch": 0.0020518081559374197,
+      "grad_norm": 0.8995798230171204,
+      "learning_rate": 0.00012,
+      "loss": 1.0725,
+      "step": 6
+    },
+    {
+      "epoch": 0.0023937761819269898,
+      "grad_norm": 0.6106855273246765,
+      "learning_rate": 0.00014,
+      "loss": 0.8916,
+      "step": 7
+    },
+    {
+      "epoch": 0.00273574420791656,
+      "grad_norm": 0.7131401300430298,
+      "learning_rate": 0.00016,
+      "loss": 1.0587,
+      "step": 8
+    },
+    {
+      "epoch": 0.00307771223390613,
+      "grad_norm": 0.5911877155303955,
+      "learning_rate": 0.00018,
+      "loss": 0.9831,
+      "step": 9
+    },
+    {
+      "epoch": 0.0034196802598956996,
+      "grad_norm": 0.5909619331359863,
+      "learning_rate": 0.0002,
+      "loss": 0.9505,
+      "step": 10
+    },
+    {
+      "epoch": 0.0037616482858852697,
+      "grad_norm": 0.4864940941333771,
+      "learning_rate": 0.00019999981242804101,
+      "loss": 0.7427,
+      "step": 11
+    },
+    {
+      "epoch": 0.004103616311874839,
+      "grad_norm": 0.7676584124565125,
+      "learning_rate": 0.00019999924971286767,
+      "loss": 1.0542,
+      "step": 12
+    },
+    {
+      "epoch": 0.00444558433786441,
+      "grad_norm": 0.4486151933670044,
+      "learning_rate": 0.00019999831185659098,
+      "loss": 0.8147,
+      "step": 13
+    },
+    {
+      "epoch": 0.0047875523638539795,
+      "grad_norm": 0.6128121614456177,
+      "learning_rate": 0.00019999699886272926,
+      "loss": 0.9666,
+      "step": 14
+    },
+    {
+      "epoch": 0.005129520389843549,
+      "grad_norm": 0.4076813757419586,
+      "learning_rate": 0.00019999531073620815,
+      "loss": 0.7688,
+      "step": 15
+    },
+    {
+      "epoch": 0.00547148841583312,
+      "grad_norm": 0.35011258721351624,
+      "learning_rate": 0.0001999932474833605,
+      "loss": 0.8118,
+      "step": 16
+    },
+    {
+      "epoch": 0.005813456441822689,
+      "grad_norm": 0.5845876336097717,
+      "learning_rate": 0.0001999908091119265,
+      "loss": 0.8527,
+      "step": 17
+    },
+    {
+      "epoch": 0.00615542446781226,
+      "grad_norm": 0.3738354742527008,
+      "learning_rate": 0.00019998799563105356,
+      "loss": 0.8481,
+      "step": 18
+    },
+    {
+      "epoch": 0.0064973924938018296,
+      "grad_norm": 0.44720950722694397,
+      "learning_rate": 0.00019998480705129632,
+      "loss": 0.8133,
+      "step": 19
+    },
+    {
+      "epoch": 0.006839360519791399,
+      "grad_norm": 0.34272846579551697,
+      "learning_rate": 0.00019998124338461645,
+      "loss": 0.7256,
+      "step": 20
+    },
+    {
+      "epoch": 0.00718132854578097,
+      "grad_norm": 0.44286996126174927,
+      "learning_rate": 0.0001999773046443829,
+      "loss": 0.9067,
+      "step": 21
+    },
+    {
+      "epoch": 0.007523296571770539,
+      "grad_norm": 0.5272486209869385,
+      "learning_rate": 0.00019997299084537163,
+      "loss": 0.921,
+      "step": 22
+    },
+    {
+      "epoch": 0.007865264597760109,
+      "grad_norm": 0.4485596716403961,
+      "learning_rate": 0.0001999683020037655,
+      "loss": 0.864,
+      "step": 23
+    },
+    {
+      "epoch": 0.008207232623749679,
+      "grad_norm": 0.5138843059539795,
+      "learning_rate": 0.00019996323813715452,
+      "loss": 0.9542,
+      "step": 24
+    },
+    {
+      "epoch": 0.00854920064973925,
+      "grad_norm": 0.40785813331604004,
+      "learning_rate": 0.0001999577992645354,
+      "loss": 0.7781,
+      "step": 25
+    },
+    {
+      "epoch": 0.00889116867572882,
+      "grad_norm": 0.3282235562801361,
+      "learning_rate": 0.0001999519854063118,
+      "loss": 0.7735,
+      "step": 26
+    },
+    {
+      "epoch": 0.00923313670171839,
+      "grad_norm": 0.8032826781272888,
+      "learning_rate": 0.000199945796584294,
+      "loss": 0.5827,
+      "step": 27
+    },
+    {
+      "epoch": 0.009575104727707959,
+      "grad_norm": 0.34260094165802,
+      "learning_rate": 0.00019993923282169904,
+      "loss": 0.7877,
+      "step": 28
+    },
+    {
+      "epoch": 0.009917072753697529,
+      "grad_norm": 0.2961389422416687,
+      "learning_rate": 0.00019993229414315043,
+      "loss": 0.7275,
+      "step": 29
+    },
+    {
+      "epoch": 0.010259040779687098,
+      "grad_norm": 0.38878557085990906,
+      "learning_rate": 0.00019992498057467824,
+      "loss": 0.719,
+      "step": 30
+    },
+    {
+      "epoch": 0.01060100880567667,
+      "grad_norm": 0.4567428231239319,
+      "learning_rate": 0.00019991729214371886,
+      "loss": 0.9252,
+      "step": 31
+    },
+    {
+      "epoch": 0.01094297683166624,
+      "grad_norm": 0.4523990750312805,
+      "learning_rate": 0.00019990922887911495,
+      "loss": 0.7931,
+      "step": 32
+    },
+    {
+      "epoch": 0.011284944857655809,
+      "grad_norm": 0.3310764729976654,
+      "learning_rate": 0.0001999007908111154,
+      "loss": 0.6515,
+      "step": 33
+    },
+    {
+      "epoch": 0.011626912883645379,
+      "grad_norm": 0.39969560503959656,
+      "learning_rate": 0.00019989197797137508,
+      "loss": 0.7453,
+      "step": 34
+    },
+    {
+      "epoch": 0.011968880909634948,
+      "grad_norm": 0.2909970283508301,
+      "learning_rate": 0.00019988279039295483,
+      "loss": 0.6965,
+      "step": 35
+    },
+    {
+      "epoch": 0.01231084893562452,
+      "grad_norm": 0.4190930426120758,
+      "learning_rate": 0.0001998732281103213,
+      "loss": 0.9058,
+      "step": 36
+    },
+    {
+      "epoch": 0.01265281696161409,
+      "grad_norm": 0.3214966356754303,
+      "learning_rate": 0.00019986329115934682,
+      "loss": 0.8127,
+      "step": 37
+    },
+    {
+      "epoch": 0.012994784987603659,
+      "grad_norm": 0.33438751101493835,
+      "learning_rate": 0.0001998529795773092,
+      "loss": 0.9961,
+      "step": 38
+    },
+    {
+      "epoch": 0.013336753013593229,
+      "grad_norm": 0.3781830072402954,
+      "learning_rate": 0.00019984229340289176,
+      "loss": 0.9191,
+      "step": 39
+    },
+    {
+      "epoch": 0.013678721039582798,
+      "grad_norm": 0.3430039584636688,
+      "learning_rate": 0.00019983123267618304,
+      "loss": 0.9132,
+      "step": 40
+    },
+    {
+      "epoch": 0.01402068906557237,
+      "grad_norm": 0.2451884001493454,
+      "learning_rate": 0.00019981979743867668,
+      "loss": 0.5962,
+      "step": 41
+    },
+    {
+      "epoch": 0.01436265709156194,
+      "grad_norm": 0.32023003697395325,
+      "learning_rate": 0.00019980798773327124,
+      "loss": 0.7075,
+      "step": 42
+    },
+    {
+      "epoch": 0.01470462511755151,
+      "grad_norm": 0.30107682943344116,
+      "learning_rate": 0.00019979580360427016,
+      "loss": 0.8956,
+      "step": 43
+    },
+    {
+      "epoch": 0.015046593143541079,
+      "grad_norm": 0.32114484906196594,
+      "learning_rate": 0.00019978324509738144,
+      "loss": 0.7539,
+      "step": 44
+    },
+    {
+      "epoch": 0.015388561169530648,
+      "grad_norm": 0.42000991106033325,
+      "learning_rate": 0.00019977031225971756,
+      "loss": 0.7863,
+      "step": 45
+    },
+    {
+      "epoch": 0.015730529195520218,
+      "grad_norm": 0.3725033700466156,
+      "learning_rate": 0.00019975700513979525,
+      "loss": 0.7707,
+      "step": 46
+    },
+    {
+      "epoch": 0.016072497221509788,
+      "grad_norm": 0.4243452847003937,
+      "learning_rate": 0.0001997433237875354,
+      "loss": 0.7683,
+      "step": 47
+    },
+    {
+      "epoch": 0.016414465247499357,
+      "grad_norm": 0.3983890116214752,
+      "learning_rate": 0.00019972926825426271,
+      "loss": 0.7036,
+      "step": 48
+    },
+    {
+      "epoch": 0.016756433273488927,
+      "grad_norm": 0.520100474357605,
+      "learning_rate": 0.00019971483859270573,
+      "loss": 0.98,
+      "step": 49
+    },
+    {
+      "epoch": 0.0170984012994785,
+      "grad_norm": 0.40005993843078613,
+      "learning_rate": 0.0001997000348569964,
+      "loss": 0.9834,
+      "step": 50
+    },
+    {
+      "epoch": 0.01744036932546807,
+      "grad_norm": 0.3966430127620697,
+      "learning_rate": 0.00019968485710267008,
+      "loss": 0.7311,
+      "step": 51
+    },
+    {
+      "epoch": 0.01778233735145764,
+      "grad_norm": 0.6581923961639404,
+      "learning_rate": 0.00019966930538666518,
+      "loss": 0.8044,
+      "step": 52
+    },
+    {
+      "epoch": 0.01812430537744721,
+      "grad_norm": 0.5657528042793274,
+      "learning_rate": 0.000199653379767323,
+      "loss": 0.7815,
+      "step": 53
+    },
+    {
+      "epoch": 0.01846627340343678,
+      "grad_norm": 0.3066444993019104,
+      "learning_rate": 0.00019963708030438753,
+      "loss": 0.7245,
+      "step": 54
+    },
+    {
+      "epoch": 0.01880824142942635,
+      "grad_norm": 0.33863499760627747,
+      "learning_rate": 0.00019962040705900523,
+      "loss": 0.9059,
+      "step": 55
+    },
+    {
+      "epoch": 0.019150209455415918,
+      "grad_norm": 0.3568740487098694,
+      "learning_rate": 0.00019960336009372477,
+      "loss": 0.594,
+      "step": 56
+    },
+    {
+      "epoch": 0.019492177481405488,
+      "grad_norm": 0.36903116106987,
+      "learning_rate": 0.00019958593947249677,
+      "loss": 0.9073,
+      "step": 57
+    },
+    {
+      "epoch": 0.019834145507395057,
+      "grad_norm": 0.3279438018798828,
+      "learning_rate": 0.0001995681452606737,
+      "loss": 0.7419,
+      "step": 58
+    },
+    {
+      "epoch": 0.020176113533384627,
+      "grad_norm": 0.4194377064704895,
+      "learning_rate": 0.00019954997752500938,
+      "loss": 0.8736,
+      "step": 59
+    },
+    {
+      "epoch": 0.020518081559374197,
+      "grad_norm": 0.4778840243816376,
+      "learning_rate": 0.00019953143633365898,
+      "loss": 0.7657,
+      "step": 60
+    },
+    {
+      "epoch": 0.02086004958536377,
+      "grad_norm": 0.6129888296127319,
+      "learning_rate": 0.0001995125217561787,
+      "loss": 0.8737,
+      "step": 61
+    },
+    {
+      "epoch": 0.02120201761135334,
+      "grad_norm": 0.409097820520401,
+      "learning_rate": 0.00019949323386352546,
+      "loss": 0.8391,
+      "step": 62
+    },
+    {
+      "epoch": 0.02154398563734291,
+      "grad_norm": 0.5153107643127441,
+      "learning_rate": 0.0001994735727280565,
+      "loss": 0.8581,
+      "step": 63
+    },
+    {
+      "epoch": 0.02188595366333248,
+      "grad_norm": 0.4380418360233307,
+      "learning_rate": 0.00019945353842352944,
+      "loss": 0.644,
+      "step": 64
+    },
+    {
+      "epoch": 0.02222792168932205,
+      "grad_norm": 0.3869611620903015,
+      "learning_rate": 0.00019943313102510174,
+      "loss": 0.6523,
+      "step": 65
+    },
+    {
+      "epoch": 0.022569889715311618,
+      "grad_norm": 0.37510204315185547,
+      "learning_rate": 0.00019941235060933052,
+      "loss": 0.6735,
+      "step": 66
+    },
+    {
+      "epoch": 0.022911857741301188,
+      "grad_norm": 0.42392924427986145,
+      "learning_rate": 0.00019939119725417227,
+      "loss": 0.893,
+      "step": 67
+    },
+    {
+      "epoch": 0.023253825767290758,
+      "grad_norm": 0.36124387383461,
+      "learning_rate": 0.00019936967103898249,
+      "loss": 0.4957,
+      "step": 68
+    },
+    {
+      "epoch": 0.023595793793280327,
+      "grad_norm": 0.43561598658561707,
+      "learning_rate": 0.00019934777204451548,
+      "loss": 0.7754,
+      "step": 69
+    },
+    {
+      "epoch": 0.023937761819269897,
+      "grad_norm": 0.3700435757637024,
+      "learning_rate": 0.00019932550035292392,
+      "loss": 0.7972,
+      "step": 70
+    },
+    {
+      "epoch": 0.02427972984525947,
+      "grad_norm": 0.34192344546318054,
+      "learning_rate": 0.00019930285604775883,
+      "loss": 0.7994,
+      "step": 71
+    },
+    {
+      "epoch": 0.02462169787124904,
+      "grad_norm": 0.43785643577575684,
+      "learning_rate": 0.00019927983921396884,
+      "loss": 0.8146,
+      "step": 72
+    },
+    {
+      "epoch": 0.02496366589723861,
+      "grad_norm": 0.41266417503356934,
+      "learning_rate": 0.00019925644993790026,
+      "loss": 0.7318,
+      "step": 73
+    },
+    {
+      "epoch": 0.02530563392322818,
+      "grad_norm": 0.42958229780197144,
+      "learning_rate": 0.00019923268830729647,
+      "loss": 0.4806,
+      "step": 74
+    },
+    {
+      "epoch": 0.02564760194921775,
+      "grad_norm": 0.6878901124000549,
+      "learning_rate": 0.00019920855441129786,
+      "loss": 1.0463,
+      "step": 75
+    },
+    {
+      "epoch": 0.025989569975207318,
+      "grad_norm": 0.40566718578338623,
+      "learning_rate": 0.00019918404834044125,
+      "loss": 0.9204,
+      "step": 76
+    },
+    {
+      "epoch": 0.026331538001196888,
+      "grad_norm": 0.4665778577327728,
+      "learning_rate": 0.00019915917018665963,
+      "loss": 0.691,
+      "step": 77
+    },
+    {
+      "epoch": 0.026673506027186458,
+      "grad_norm": 0.34144479036331177,
+      "learning_rate": 0.00019913392004328192,
+      "loss": 0.4463,
+      "step": 78
+    },
+    {
+      "epoch": 0.027015474053176027,
+      "grad_norm": 0.43922317028045654,
+      "learning_rate": 0.00019910829800503252,
+      "loss": 0.9006,
+      "step": 79
+    },
+    {
+      "epoch": 0.027357442079165597,
+      "grad_norm": 0.6301386952400208,
+      "learning_rate": 0.0001990823041680309,
+      "loss": 0.8589,
+      "step": 80
+    },
+    {
+      "epoch": 0.027699410105155167,
+      "grad_norm": 0.3700405955314636,
+      "learning_rate": 0.0001990559386297914,
+      "loss": 0.7178,
+      "step": 81
+    },
+    {
+      "epoch": 0.02804137813114474,
+      "grad_norm": 0.40341895818710327,
+      "learning_rate": 0.0001990292014892227,
+      "loss": 0.8898,
+      "step": 82
+    },
+    {
+      "epoch": 0.02838334615713431,
+      "grad_norm": 0.40765297412872314,
+      "learning_rate": 0.00019900209284662754,
+      "loss": 0.7831,
+      "step": 83
+    },
+    {
+      "epoch": 0.02872531418312388,
+      "grad_norm": 0.40845540165901184,
+      "learning_rate": 0.0001989746128037024,
+      "loss": 0.6554,
+      "step": 84
+    },
+    {
+      "epoch": 0.02906728220911345,
+      "grad_norm": 0.5446910858154297,
+      "learning_rate": 0.00019894676146353698,
+      "loss": 0.6161,
+      "step": 85
+    },
+    {
+      "epoch": 0.02940925023510302,
+      "grad_norm": 0.4155116677284241,
+      "learning_rate": 0.00019891853893061386,
+      "loss": 0.9986,
+      "step": 86
+    },
+    {
+      "epoch": 0.029751218261092588,
+      "grad_norm": 0.30781111121177673,
+      "learning_rate": 0.00019888994531080815,
+      "loss": 0.5578,
+      "step": 87
+    },
+    {
+      "epoch": 0.030093186287082158,
+      "grad_norm": 0.3865339457988739,
+      "learning_rate": 0.00019886098071138708,
+      "loss": 0.6909,
+      "step": 88
+    },
+    {
+      "epoch": 0.030435154313071727,
+      "grad_norm": 0.6464967727661133,
+      "learning_rate": 0.0001988316452410096,
+      "loss": 0.8395,
+      "step": 89
+    },
+    {
+      "epoch": 0.030777122339061297,
+      "grad_norm": 0.39004752039909363,
+      "learning_rate": 0.00019880193900972596,
+      "loss": 0.7707,
+      "step": 90
+    },
+    {
+      "epoch": 0.031119090365050867,
+      "grad_norm": 0.401058554649353,
+      "learning_rate": 0.00019877186212897727,
+      "loss": 0.7911,
+      "step": 91
+    },
+    {
+      "epoch": 0.031461058391040436,
+      "grad_norm": 0.47253596782684326,
+      "learning_rate": 0.0001987414147115951,
+      "loss": 0.6839,
+      "step": 92
+    },
+    {
+      "epoch": 0.031803026417030006,
+      "grad_norm": 0.34148743748664856,
+      "learning_rate": 0.00019871059687180103,
+      "loss": 0.7035,
+      "step": 93
+    },
+    {
+      "epoch": 0.032144994443019576,
+      "grad_norm": 0.40911146998405457,
+      "learning_rate": 0.00019867940872520646,
+      "loss": 0.8046,
+      "step": 94
+    },
+    {
+      "epoch": 0.032486962469009145,
+      "grad_norm": 0.36442333459854126,
+      "learning_rate": 0.00019864785038881168,
+      "loss": 0.6757,
+      "step": 95
+    },
+    {
+      "epoch": 0.032828930494998715,
+      "grad_norm": 0.33227601647377014,
+      "learning_rate": 0.00019861592198100597,
+      "loss": 0.5055,
+      "step": 96
+    },
+    {
+      "epoch": 0.033170898520988285,
+      "grad_norm": 0.39125221967697144,
+      "learning_rate": 0.00019858362362156675,
+      "loss": 0.7066,
+      "step": 97
+    },
+    {
+      "epoch": 0.033512866546977854,
+      "grad_norm": 0.3918190002441406,
+      "learning_rate": 0.00019855095543165935,
+      "loss": 0.7265,
+      "step": 98
+    },
+    {
+      "epoch": 0.03385483457296743,
+      "grad_norm": 0.42384183406829834,
+      "learning_rate": 0.00019851791753383653,
+      "loss": 0.6806,
+      "step": 99
+    },
+    {
+      "epoch": 0.034196802598957,
+      "grad_norm": 0.4679703116416931,
+      "learning_rate": 0.00019848451005203795,
+      "loss": 0.8846,
+      "step": 100
+    },
+    {
+      "epoch": 0.03453877062494657,
+      "grad_norm": 0.472374826669693,
+      "learning_rate": 0.00019845073311158972,
+      "loss": 0.8286,
+      "step": 101
+    },
+    {
+      "epoch": 0.03488073865093614,
+      "grad_norm": 0.5372505187988281,
+      "learning_rate": 0.000198416586839204,
+      "loss": 0.8248,
+      "step": 102
+    },
+    {
+      "epoch": 0.03522270667692571,
+      "grad_norm": 0.33633288741111755,
+      "learning_rate": 0.00019838207136297847,
+      "loss": 0.8867,
+      "step": 103
+    },
+    {
+      "epoch": 0.03556467470291528,
+      "grad_norm": 0.3873573839664459,
+      "learning_rate": 0.0001983471868123958,
+      "loss": 0.7565,
+      "step": 104
+    },
+    {
+      "epoch": 0.03590664272890485,
+      "grad_norm": 0.35947027802467346,
+      "learning_rate": 0.0001983119333183233,
+      "loss": 0.7207,
+      "step": 105
+    },
+    {
+      "epoch": 0.03624861075489442,
+      "grad_norm": 0.48201021552085876,
+      "learning_rate": 0.00019827631101301224,
+      "loss": 0.6401,
+      "step": 106
+    },
+    {
+      "epoch": 0.03659057878088399,
+      "grad_norm": 0.4126957356929779,
+      "learning_rate": 0.0001982403200300976,
+      "loss": 0.8179,
+      "step": 107
+    },
+    {
+      "epoch": 0.03693254680687356,
+      "grad_norm": 0.37395864725112915,
+      "learning_rate": 0.00019820396050459737,
+      "loss": 0.6854,
+      "step": 108
+    },
+    {
+      "epoch": 0.03727451483286313,
+      "grad_norm": 0.5181075930595398,
+      "learning_rate": 0.00019816723257291203,
+      "loss": 0.771,
+      "step": 109
+    },
+    {
+      "epoch": 0.0376164828588527,
+      "grad_norm": 0.3693534731864929,
+      "learning_rate": 0.00019813013637282425,
+      "loss": 0.7866,
+      "step": 110
+    },
+    {
+      "epoch": 0.03795845088484227,
+      "grad_norm": 0.497036337852478,
+      "learning_rate": 0.00019809267204349816,
+      "loss": 0.8671,
+      "step": 111
+    },
+    {
+      "epoch": 0.038300418910831836,
+      "grad_norm": 0.6101623773574829,
+      "learning_rate": 0.00019805483972547885,
+      "loss": 0.6834,
+      "step": 112
+    },
+    {
+      "epoch": 0.038642386936821406,
+      "grad_norm": 0.3899027109146118,
+      "learning_rate": 0.00019801663956069203,
+      "loss": 0.6377,
+      "step": 113
+    },
+    {
+      "epoch": 0.038984354962810976,
+      "grad_norm": 0.38232648372650146,
+      "learning_rate": 0.00019797807169244326,
+      "loss": 0.7705,
+      "step": 114
+    },
+    {
+      "epoch": 0.039326322988800545,
+      "grad_norm": 0.3305700123310089,
+      "learning_rate": 0.00019793913626541757,
+      "loss": 0.7263,
+      "step": 115
+    },
+    {
+      "epoch": 0.039668291014790115,
+      "grad_norm": 0.3599172532558441,
+      "learning_rate": 0.00019789983342567884,
+      "loss": 0.7474,
+      "step": 116
+    },
+    {
+      "epoch": 0.040010259040779685,
+      "grad_norm": 0.3204912543296814,
+      "learning_rate": 0.00019786016332066928,
+      "loss": 0.6778,
+      "step": 117
+    },
+    {
+      "epoch": 0.040352227066769254,
+      "grad_norm": 0.3829420208930969,
+      "learning_rate": 0.00019782012609920887,
+      "loss": 0.6515,
+      "step": 118
+    },
+    {
+      "epoch": 0.040694195092758824,
+      "grad_norm": 0.2973586618900299,
+      "learning_rate": 0.00019777972191149484,
+      "loss": 0.6318,
+      "step": 119
+    },
+    {
+      "epoch": 0.041036163118748394,
+      "grad_norm": 0.3322150707244873,
+      "learning_rate": 0.00019773895090910098,
+      "loss": 0.9753,
+      "step": 120
+    },
+    {
+      "epoch": 0.04137813114473797,
+      "grad_norm": 0.33772385120391846,
+      "learning_rate": 0.0001976978132449773,
+      "loss": 0.6391,
+      "step": 121
+    },
+    {
+      "epoch": 0.04172009917072754,
+      "grad_norm": 0.4290957748889923,
+      "learning_rate": 0.0001976563090734492,
+      "loss": 0.6189,
+      "step": 122
+    },
+    {
+      "epoch": 0.04206206719671711,
+      "grad_norm": 0.3153519630432129,
+      "learning_rate": 0.00019761443855021704,
+      "loss": 0.7684,
+      "step": 123
+    },
+    {
+      "epoch": 0.04240403522270668,
+      "grad_norm": 0.466972678899765,
+      "learning_rate": 0.0001975722018323556,
+      "loss": 0.7727,
+      "step": 124
+    },
+    {
+      "epoch": 0.04274600324869625,
+      "grad_norm": 0.3417704403400421,
+      "learning_rate": 0.00019752959907831332,
+      "loss": 0.5135,
+      "step": 125
+    },
+    {
+      "epoch": 0.04308797127468582,
+      "grad_norm": 0.3414458632469177,
+      "learning_rate": 0.00019748663044791184,
+      "loss": 0.7005,
+      "step": 126
+    },
+    {
+      "epoch": 0.04342993930067539,
+      "grad_norm": 0.45409920811653137,
+      "learning_rate": 0.0001974432961023454,
+      "loss": 0.7558,
+      "step": 127
+    },
+    {
+      "epoch": 0.04377190732666496,
+      "grad_norm": 0.40951547026634216,
+      "learning_rate": 0.00019739959620418009,
+      "loss": 0.7712,
+      "step": 128
+    },
+    {
+      "epoch": 0.04411387535265453,
+      "grad_norm": 0.40087854862213135,
+      "learning_rate": 0.0001973555309173535,
+      "loss": 0.6852,
+      "step": 129
+    },
+    {
+      "epoch": 0.0444558433786441,
+      "grad_norm": 0.42778703570365906,
+      "learning_rate": 0.00019731110040717382,
+      "loss": 0.6751,
+      "step": 130
+    },
+    {
+      "epoch": 0.04479781140463367,
+      "grad_norm": 0.4334496557712555,
+      "learning_rate": 0.00019726630484031941,
+      "loss": 0.5747,
+      "step": 131
+    },
+    {
+      "epoch": 0.045139779430623236,
+      "grad_norm": 0.43662187457084656,
+      "learning_rate": 0.00019722114438483815,
+      "loss": 0.8596,
+      "step": 132
+    },
+    {
+      "epoch": 0.045481747456612806,
+      "grad_norm": 0.27613282203674316,
+      "learning_rate": 0.0001971756192101467,
+      "loss": 0.7824,
+      "step": 133
+    },
+    {
+      "epoch": 0.045823715482602376,
+      "grad_norm": 0.40083047747612,
+      "learning_rate": 0.00019712972948703004,
+      "loss": 0.7484,
+      "step": 134
+    },
+    {
+      "epoch": 0.046165683508591945,
+      "grad_norm": 0.41493287682533264,
+      "learning_rate": 0.0001970834753876406,
+      "loss": 0.8322,
+      "step": 135
+    },
+    {
+      "epoch": 0.046507651534581515,
+      "grad_norm": 0.4495963156223297,
+      "learning_rate": 0.00019703685708549786,
+      "loss": 0.6168,
+      "step": 136
+    },
+    {
+      "epoch": 0.046849619560571085,
+      "grad_norm": 0.30632326006889343,
+      "learning_rate": 0.0001969898747554876,
+      "loss": 0.7717,
+      "step": 137
+    },
+    {
+      "epoch": 0.047191587586560654,
+      "grad_norm": 0.45101413130760193,
+      "learning_rate": 0.00019694252857386103,
+      "loss": 0.8749,
+      "step": 138
+    },
+    {
+      "epoch": 0.047533555612550224,
+      "grad_norm": 0.4110430181026459,
+      "learning_rate": 0.00019689481871823462,
+      "loss": 0.8129,
+      "step": 139
+    },
+    {
+      "epoch": 0.047875523638539794,
+      "grad_norm": 0.4156266450881958,
+      "learning_rate": 0.00019684674536758894,
+      "loss": 0.807,
+      "step": 140
+    },
+    {
+      "epoch": 0.04821749166452936,
+      "grad_norm": 0.4598490297794342,
+      "learning_rate": 0.00019679830870226818,
+      "loss": 0.9876,
+      "step": 141
+    },
+    {
+      "epoch": 0.04855945969051894,
+      "grad_norm": 0.28661856055259705,
+      "learning_rate": 0.00019674950890397964,
+      "loss": 0.6236,
+      "step": 142
+    },
+    {
+      "epoch": 0.04890142771650851,
+      "grad_norm": 0.4644462466239929,
+      "learning_rate": 0.0001967003461557927,
+      "loss": 0.6955,
+      "step": 143
+    },
+    {
+      "epoch": 0.04924339574249808,
+      "grad_norm": 0.3590112626552582,
+      "learning_rate": 0.00019665082064213854,
+      "loss": 0.6318,
+      "step": 144
+    },
+    {
+      "epoch": 0.04958536376848765,
+      "grad_norm": 0.3631165325641632,
+      "learning_rate": 0.00019660093254880902,
+      "loss": 0.7603,
+      "step": 145
+    },
+    {
+      "epoch": 0.04992733179447722,
+      "grad_norm": 0.5181772708892822,
+      "learning_rate": 0.00019655068206295628,
+      "loss": 0.9442,
+      "step": 146
+    },
+    {
+      "epoch": 0.05026929982046679,
+      "grad_norm": 0.34273993968963623,
+      "learning_rate": 0.00019650006937309202,
+      "loss": 0.6786,
+      "step": 147
+    },
+    {
+      "epoch": 0.05061126784645636,
+      "grad_norm": 0.5383986234664917,
+      "learning_rate": 0.00019644909466908663,
+      "loss": 0.913,
+      "step": 148
+    },
+    {
+      "epoch": 0.05095323587244593,
+      "grad_norm": 0.330531507730484,
+      "learning_rate": 0.00019639775814216863,
+      "loss": 0.8166,
+      "step": 149
+    },
+    {
+      "epoch": 0.0512952038984355,
+      "grad_norm": 0.47129181027412415,
+      "learning_rate": 0.00019634605998492387,
+      "loss": 0.8368,
+      "step": 150
+    },
+    {
+      "epoch": 0.0512952038984355,
+      "eval_loss": 0.8442216515541077,
+      "eval_runtime": 231.1814,
+      "eval_samples_per_second": 5.329,
+      "eval_steps_per_second": 2.665,
+      "step": 150
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 1632,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 150,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 4,
+        "early_stopping_threshold": 0.0
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.3089463257268224e+17,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bc5daaf9e0d9fec3c865f7953c9366de525ccafe8d40e89fb5ea889d17770f1
+size 6776