runtime error

Exit code: 1. Reason: ading checkpoint shards: 100%|██████████| 2/2 [00:03<00:00, 1.86s/it] generation_config.json: 0%| | 0.00/189 [00:00<?, ?B/s] generation_config.json: 100%|██████████| 189/189 [00:00<00:00, 1.32MB/s] adapter_config.json: 0%| | 0.00/943 [00:00<?, ?B/s] adapter_config.json: 100%|██████████| 943/943 [00:00<00:00, 6.33MB/s] adapter_model.safetensors: 0%| | 0.00/3.20G [00:00<?, ?B/s] adapter_model.safetensors: 4%|▍ | 142M/3.20G [00:01<00:21, 139MB/s] adapter_model.safetensors: 19%|█▉ | 604M/3.20G [00:02<00:11, 232MB/s] adapter_model.safetensors: 90%|█████████ | 2.88G/3.20G [00:03<00:00, 985MB/s] adapter_model.safetensors: 100%|██████████| 3.20G/3.20G [00:04<00:00, 793MB/s] Traceback (most recent call last): File "/app/app.py", line 18, in <module> model = PeftModel.from_pretrained(base_model, model_repo_id) File "/usr/local/lib/python3.10/site-packages/peft/peft_model.py", line 568, in from_pretrained load_result = model.load_adapter( File "/usr/local/lib/python3.10/site-packages/peft/peft_model.py", line 1368, in load_adapter load_result = set_peft_model_state_dict( File "/usr/local/lib/python3.10/site-packages/peft/utils/save_and_load.py", line 565, in set_peft_model_state_dict load_result = model.load_state_dict(peft_model_state_dict, strict=False) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2629, in load_state_dict raise RuntimeError( RuntimeError: Error(s) in loading state_dict for PeftModelForCausalLM: size mismatch for base_model.model.model.embed_tokens.weight: copying a param with shape torch.Size([128257, 3072]) from checkpoint, the shape in current model is torch.Size([128256, 3072]). size mismatch for base_model.model.lm_head.weight: copying a param with shape torch.Size([128257, 3072]) from checkpoint, the shape in current model is torch.Size([128256, 3072]).

Container logs: