Minekorkmz/sentiment_analysis

Description

This model was obtained by fine-tuning the Llama-2 7B large language model with the LoRA technique. The aim is to develop a sentiment analysis system in Turkish language by training the model according to the sentences in the given data set. The evaluation metrics of the model were calculated and the following results were obtained.

Dataset

The training data set consists of 152715 rows and the eval data set consists of 16968 rows. It includes social media posts and product reviews.

Uses

from transformers import AutoConfig from transformers import AutoModelForSequenceClassification

config = AutoConfig.from_pretrained("Minekorkmz/model_yurt_1200") num_labels = config.num_labels

base_model = AutoModelForSequenceClassification.from_pretrained( "meta-llama/Llama-2-7b-chat-hf", num_labels=num_labels )

model = PeftModel.from_pretrained(base_model, "Minekorkmz/model_yurt_1200") tokenizer = AutoTokenizer.from_pretrained("Minekorkmz/model_yurt_1200")

from transformers import pipeline

sentiment_task = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer, return_all_scores=True)

print(sentiment_task("çok kötü bir ürün oldu sevemedim"))

Training procedure

The following bitsandbytes quantization config was used during training:

• quant_method: bitsandbytes
• load_in_8bit: False
• load_in_4bit: True
• llm_int8_threshold: 6.0
• llm_int8_skip_modules: None
• llm_int8_enable_fp32_cpu_offload: False
• llm_int8_has_fp16_weight: False
• bnb_4bit_quant_type: nf4
• bnb_4bit_use_double_quant: True
• bnb_4bit_compute_dtype: float16

Framework versions

• PEFT 0.4.0
• accelerate 0.26.0
• bitsandbytes 0.41.1
• transformers 4.35.0
• trl 0.4.7