Initial commit with model files

.gitattributes

@@ -0,0 +1,11 @@
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.tar.gz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+*.json filter=lfs diff=lfs merge=lfs -text

llama_finetuned/README.md

@@ -0,0 +1,202 @@
+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

llama_finetuned/gptqmodel_4bit/quant_log.csv

@@ -0,0 +1,225 @@
+layer,module,loss,damp,time
+0,self_attn.k_proj,0.20497,0.01000,2.207
+0,self_attn.v_proj,0.00201,0.01000,1.465
+0,self_attn.q_proj,0.30193,0.01000,1.488
+0,self_attn.o_proj,0.00019,0.01000,1.464
+0,mlp.up_proj,0.14901,0.01000,1.538
+0,mlp.gate_proj,0.17876,0.01000,1.495
+0,mlp.down_proj,0.00063,0.01000,5.856
+1,self_attn.k_proj,0.16532,0.01000,1.578
+1,self_attn.v_proj,0.00722,0.01000,1.452
+1,self_attn.q_proj,0.29823,0.01000,1.474
+1,self_attn.o_proj,0.00045,0.01000,1.478
+1,mlp.up_proj,0.28177,0.01000,1.518
+1,mlp.gate_proj,0.33167,0.01000,1.473
+1,mlp.down_proj,0.15022,0.01000,5.751
+2,self_attn.k_proj,0.74963,0.01000,1.564
+2,self_attn.v_proj,0.03474,0.01000,1.430
+2,self_attn.q_proj,1.15687,0.01000,1.666
+2,self_attn.o_proj,0.00045,0.01000,1.496
+2,mlp.up_proj,0.48363,0.01000,1.516
+2,mlp.gate_proj,0.60252,0.01000,1.495
+2,mlp.down_proj,0.00285,0.01000,5.790
+3,self_attn.k_proj,0.63965,0.01000,1.489
+3,self_attn.v_proj,0.06011,0.01000,1.460
+3,self_attn.q_proj,1.14490,0.01000,1.496
+3,self_attn.o_proj,0.00123,0.01000,1.484
+3,mlp.up_proj,0.63800,0.01000,1.504
+3,mlp.gate_proj,0.91154,0.01000,1.496
+3,mlp.down_proj,0.00571,0.01000,5.737
+4,self_attn.k_proj,0.55720,0.01000,1.480
+4,self_attn.v_proj,0.05674,0.01000,1.434
+4,self_attn.q_proj,0.97971,0.01000,1.475
+4,self_attn.o_proj,0.00259,0.01000,1.512
+4,mlp.up_proj,0.80503,0.01000,1.489
+4,mlp.gate_proj,1.30177,0.01000,1.478
+4,mlp.down_proj,0.00961,0.01000,5.808
+5,self_attn.k_proj,0.87404,0.01000,1.499
+5,self_attn.v_proj,0.05538,0.01000,1.452
+5,self_attn.q_proj,1.40345,0.01000,1.458
+5,self_attn.o_proj,0.00271,0.01000,1.511
+5,mlp.up_proj,0.95746,0.01000,1.506
+5,mlp.gate_proj,1.53153,0.01000,1.486
+5,mlp.down_proj,0.01401,0.01000,5.878
+6,self_attn.k_proj,0.72873,0.01000,1.496
+6,self_attn.v_proj,0.06350,0.01000,1.504
+6,self_attn.q_proj,1.31834,0.01000,1.515
+6,self_attn.o_proj,0.00532,0.01000,1.470
+6,mlp.up_proj,1.06246,0.01000,1.510
+6,mlp.gate_proj,1.72084,0.01000,1.486
+6,mlp.down_proj,0.01771,0.01000,5.809
+7,self_attn.k_proj,0.77504,0.01000,1.491
+7,self_attn.v_proj,0.06837,0.01000,1.457
+7,self_attn.q_proj,1.30903,0.01000,1.466
+7,self_attn.o_proj,0.00768,0.01000,1.511
+7,mlp.up_proj,1.18322,0.01000,1.507
+7,mlp.gate_proj,1.79183,0.01000,1.485
+7,mlp.down_proj,0.02134,0.01000,5.715
+8,self_attn.k_proj,1.00739,0.01000,1.496
+8,self_attn.v_proj,0.09484,0.01000,1.494
+8,self_attn.q_proj,1.66843,0.01000,1.532
+8,self_attn.o_proj,0.01138,0.01000,1.469
+8,mlp.up_proj,1.25729,0.01000,1.497
+8,mlp.gate_proj,1.93310,0.01000,1.482
+8,mlp.down_proj,0.02314,0.01000,5.739
+9,self_attn.k_proj,0.97315,0.01000,1.477
+9,self_attn.v_proj,0.13074,0.01000,1.420
+9,self_attn.q_proj,1.65996,0.01000,1.490
+9,self_attn.o_proj,0.01420,0.01000,1.485
+9,mlp.up_proj,1.31191,0.01000,1.509
+9,mlp.gate_proj,2.03104,0.01000,1.490
+9,mlp.down_proj,0.02473,0.01000,5.790
+10,self_attn.k_proj,1.16183,0.01000,1.472
+10,self_attn.v_proj,0.10278,0.01000,1.447
+10,self_attn.q_proj,1.93474,0.01000,1.588
+10,self_attn.o_proj,0.01309,0.01000,1.457
+10,mlp.up_proj,1.37429,0.01000,1.542
+10,mlp.gate_proj,1.98475,0.01000,1.489
+10,mlp.down_proj,0.02690,0.01000,5.896
+11,self_attn.k_proj,1.11032,0.01000,1.531
+11,self_attn.v_proj,0.10660,0.01000,1.440
+11,self_attn.q_proj,1.76108,0.01000,1.533
+11,self_attn.o_proj,0.01489,0.01000,1.556
+11,mlp.up_proj,1.47593,0.01000,1.523
+11,mlp.gate_proj,2.06277,0.01000,1.498
+11,mlp.down_proj,0.02968,0.01000,5.827
+12,self_attn.k_proj,0.88908,0.01000,1.492
+12,self_attn.v_proj,0.12811,0.01000,1.450
+12,self_attn.q_proj,1.57054,0.01000,1.478
+12,self_attn.o_proj,0.01993,0.01000,1.467
+12,mlp.up_proj,1.54407,0.01000,1.516
+12,mlp.gate_proj,2.04233,0.01000,1.456
+12,mlp.down_proj,0.03332,0.01000,5.828
+13,self_attn.k_proj,1.29054,0.01000,1.499
+13,self_attn.v_proj,0.14054,0.01000,1.453
+13,self_attn.q_proj,2.03878,0.01000,1.468
+13,self_attn.o_proj,0.01995,0.01000,1.466
+13,mlp.up_proj,1.60221,0.01000,1.540
+13,mlp.gate_proj,2.11504,0.01000,1.534
+13,mlp.down_proj,0.03465,0.01000,5.802
+14,self_attn.k_proj,1.29833,0.01000,1.494
+14,self_attn.v_proj,0.13893,0.01000,1.462
+14,self_attn.q_proj,1.95875,0.01000,1.540
+14,self_attn.o_proj,0.02435,0.01000,1.510
+14,mlp.up_proj,1.72852,0.01000,1.499
+14,mlp.gate_proj,2.40024,0.01000,1.493
+14,mlp.down_proj,0.04297,0.01000,5.825
+15,self_attn.k_proj,1.19167,0.01000,1.490
+15,self_attn.v_proj,0.16916,0.01000,1.440
+15,self_attn.q_proj,2.28649,0.01000,1.483
+15,self_attn.o_proj,0.02458,0.01000,1.498
+15,mlp.up_proj,1.79800,0.01000,1.510
+15,mlp.gate_proj,2.63445,0.01000,1.500
+15,mlp.down_proj,0.04993,0.01000,5.849
+16,self_attn.k_proj,1.17782,0.01000,1.688
+16,self_attn.v_proj,0.14366,0.01000,1.456
+16,self_attn.q_proj,2.00419,0.01000,1.478
+16,self_attn.o_proj,0.02002,0.01000,1.486
+16,mlp.up_proj,1.82658,0.01000,1.500
+16,mlp.gate_proj,2.83391,0.01000,1.502
+16,mlp.down_proj,0.05562,0.01000,5.837
+17,self_attn.k_proj,1.23447,0.01000,1.571
+17,self_attn.v_proj,0.15860,0.01000,1.471
+17,self_attn.q_proj,2.06489,0.01000,1.513
+17,self_attn.o_proj,0.01764,0.01000,1.509
+17,mlp.up_proj,1.86635,0.01000,1.516
+17,mlp.gate_proj,2.96908,0.01000,1.496
+17,mlp.down_proj,0.06157,0.01000,5.869
+18,self_attn.k_proj,1.33415,0.01000,1.533
+18,self_attn.v_proj,0.15418,0.01000,1.471
+18,self_attn.q_proj,2.05141,0.01000,1.473
+18,self_attn.o_proj,0.01136,0.01000,1.478
+18,mlp.up_proj,1.92367,0.01000,1.539
+18,mlp.gate_proj,3.09713,0.01000,1.498
+18,mlp.down_proj,0.06182,0.01000,5.831
+19,self_attn.k_proj,1.20882,0.01000,1.522
+19,self_attn.v_proj,0.16848,0.01000,1.480
+19,self_attn.q_proj,2.08260,0.01000,1.484
+19,self_attn.o_proj,0.01204,0.01000,1.489
+19,mlp.up_proj,1.99953,0.01000,1.524
+19,mlp.gate_proj,3.27914,0.01000,1.527
+19,mlp.down_proj,0.06632,0.01000,5.739
+20,self_attn.k_proj,1.30188,0.01000,1.517
+20,self_attn.v_proj,0.18483,0.01000,1.527
+20,self_attn.q_proj,2.13806,0.01000,1.568
+20,self_attn.o_proj,0.01415,0.01000,1.506
+20,mlp.up_proj,2.15121,0.01000,1.524
+20,mlp.gate_proj,3.49964,0.01000,1.575
+20,mlp.down_proj,0.07272,0.01000,5.907
+21,self_attn.k_proj,1.28493,0.01000,1.603
+21,self_attn.v_proj,0.19875,0.01000,1.440
+21,self_attn.q_proj,2.06501,0.01000,1.502
+21,self_attn.o_proj,0.01691,0.01000,1.484
+21,mlp.up_proj,2.29450,0.01000,1.507
+21,mlp.gate_proj,3.76310,0.01000,1.498
+21,mlp.down_proj,0.08457,0.01000,5.753
+22,self_attn.k_proj,1.33741,0.01000,1.512
+22,self_attn.v_proj,0.23293,0.01000,1.470
+22,self_attn.q_proj,2.07345,0.01000,1.524
+22,self_attn.o_proj,0.01621,0.01000,1.548
+22,mlp.up_proj,2.41273,0.01000,1.522
+22,mlp.gate_proj,3.91765,0.01000,1.511
+22,mlp.down_proj,0.08732,0.01000,5.917
+23,self_attn.k_proj,1.31320,0.01000,1.518
+23,self_attn.v_proj,0.25737,0.01000,1.470
+23,self_attn.q_proj,2.16603,0.01000,1.499
+23,self_attn.o_proj,0.01396,0.01000,1.509
+23,mlp.up_proj,2.55272,0.01000,1.516
+23,mlp.gate_proj,4.12648,0.01000,1.501
+23,mlp.down_proj,0.09488,0.01000,6.082
+24,self_attn.k_proj,1.30349,0.01000,1.568
+24,self_attn.v_proj,0.32369,0.01000,1.514
+24,self_attn.q_proj,2.18847,0.01000,1.530
+24,self_attn.o_proj,0.01625,0.01000,1.529
+24,mlp.up_proj,2.76134,0.01000,1.553
+24,mlp.gate_proj,4.46655,0.01000,1.531
+24,mlp.down_proj,0.10605,0.01000,5.900
+25,self_attn.k_proj,1.27495,0.01000,1.538
+25,self_attn.v_proj,0.34449,0.01000,1.527
+25,self_attn.q_proj,2.23903,0.01000,1.490
+25,self_attn.o_proj,0.01893,0.01000,1.501
+25,mlp.up_proj,3.02008,0.01000,1.535
+25,mlp.gate_proj,4.88290,0.01000,1.522
+25,mlp.down_proj,0.12015,0.01000,5.817
+26,self_attn.k_proj,1.34739,0.01000,1.598
+26,self_attn.v_proj,0.33584,0.01000,1.492
+26,self_attn.q_proj,2.18536,0.01000,1.644
+26,self_attn.o_proj,0.02690,0.01000,1.515
+26,mlp.up_proj,3.25457,0.01000,1.549
+26,mlp.gate_proj,5.27174,0.01000,1.523
+26,mlp.down_proj,0.13506,0.01000,5.841
+27,self_attn.k_proj,1.48341,0.01000,1.591
+27,self_attn.v_proj,0.46845,0.01000,1.485
+27,self_attn.q_proj,2.31207,0.01000,1.513
+27,self_attn.o_proj,0.03408,0.01000,1.561
+27,mlp.up_proj,3.67523,0.01000,1.555
+27,mlp.gate_proj,5.90887,0.01000,1.516
+27,mlp.down_proj,0.16378,0.01000,5.812
+28,self_attn.k_proj,1.23230,0.01000,1.511
+28,self_attn.v_proj,0.44825,0.01000,1.453
+28,self_attn.q_proj,2.21801,0.01000,1.510
+28,self_attn.o_proj,0.06715,0.01000,1.488
+28,mlp.up_proj,4.15845,0.01000,1.574
+28,mlp.gate_proj,6.39001,0.01000,1.504
+28,mlp.down_proj,0.20997,0.01000,5.943
+29,self_attn.k_proj,1.36047,0.01000,1.505
+29,self_attn.v_proj,0.55055,0.01000,1.459
+29,self_attn.q_proj,2.44167,0.01000,1.477
+29,self_attn.o_proj,0.05338,0.01000,1.506
+29,mlp.up_proj,4.49159,0.01000,1.515
+29,mlp.gate_proj,6.60862,0.01000,1.496
+29,mlp.down_proj,0.27847,0.01000,5.858
+30,self_attn.k_proj,1.27395,0.01000,1.612
+30,self_attn.v_proj,0.73981,0.01000,1.495
+30,self_attn.q_proj,2.19805,0.01000,1.564
+30,self_attn.o_proj,0.12117,0.01000,1.559
+30,mlp.up_proj,4.80780,0.01000,1.551
+30,mlp.gate_proj,7.24052,0.01000,1.533
+30,mlp.down_proj,0.43298,0.01000,5.796
+31,self_attn.k_proj,0.97036,0.01000,1.487
+31,self_attn.v_proj,0.44060,0.01000,1.448
+31,self_attn.q_proj,1.90233,0.01000,1.585
+31,self_attn.o_proj,0.17504,0.01000,1.501
+31,mlp.up_proj,4.47512,0.01000,1.515
+31,mlp.gate_proj,6.52746,0.01000,1.487
+31,mlp.down_proj,1.10257,0.01000,5.953