Update README.md

README.md

@@ -1,21 +1,54 @@
 ---
 library_name: peft
+tags:
+- tiiuae
+- code
+- instruct
+- databricks-dolly-15k
+- falcon-40b
+datasets:
+- databricks/databricks-dolly-15k
+base_model: tiiuae/falcon-40b
 ---
-## Training procedure
 
+For our finetuning process, we utilized the tiiuae/falcon-40b model and the Databricks-dolly-15k dataset. 
 
-The following `bitsandbytes` quantization config was used during training:
-- quant_method: bitsandbytes
-- load_in_8bit: False
-- load_in_4bit: True
-- llm_int8_threshold: 6.0
-- llm_int8_skip_modules: None
-- llm_int8_enable_fp32_cpu_offload: False
-- llm_int8_has_fp16_weight: False
-- bnb_4bit_quant_type: nf4
-- bnb_4bit_use_double_quant: True
-- bnb_4bit_compute_dtype: bfloat16
-### Framework versions
+This dataset, a meticulous compilation of over 15,000 records, was a result of the dedicated work of thousands of Databricks professionals. It was specifically designed to further improve the interactive capabilities of ChatGPT-like systems.
+The dataset contributors crafted prompt / response pairs across eight distinct instruction categories. Besides the seven categories mentioned in the InstructGPT paper, they also ventured into an open-ended, free-form category. The contributors, emphasizing genuine and original content, refrained from sourcing information online, except in special cases where Wikipedia was the source for certain instruction categories. There was also a strict directive against the use of generative AI for crafting instructions or responses.
+The contributors could address questions from their peers. Rephrasing the original question was encouraged, and there was a clear preference to answer only those queries they were certain about.
+In some categories, the data comes with reference texts sourced from Wikipedia. Users might find bracketed Wikipedia citation numbers (like [42]) within the context field of the dataset. For smoother downstream applications, it's advisable to exclude these.
 
+Our finetuning was conducted using the [MonsterAPI](https://monsterapi.ai)'s intuitive, no-code [LLM finetuner](https://docs.monsterapi.ai/fine-tune-a-large-language-model-llm). 
 
-- PEFT 0.5.0
+Highlighting the cost-effectiveness and efficiency of the process, 
+the entire session was finished in just 5 hours and 40 minutes, leveraging an A6000 48GB GPU. 
+The total cost for this efficient run was a mere `$11.8`.
+
+#### Hyperparameters & Run details:
+- Epochs: 1
+- Cost: $11.8
+- Model Path: tiiuae/falcon-40b
+- Dataset: databricks/databricks-dolly-15k
+- Learning rate: 0.0002
+- Data split: Training 90% / Validation 10%
+- Gradient accumulation steps: 4
+
+license: apache-2.0
+---
+
+######
+
+Prompt Used: 
+
+### INSTRUCTION:
+[instruction]
+
+[context]
+
+### RESPONSE:
+[response]
+
+Loss metrics
+
+Training loss (Blue) Validation Loss (orange):
+![training loss](train-loss.png "Training loss")