Hugging Face's logo

Apel-sin
/
t-pro-it-1.0-exl2

Russian
exl2
Model card Files
xet
Community
Apel-sin commited on
Commit
c8bcc8e
·
verified ·
1 Parent(s): e2367c3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +78 -8
README.md CHANGED
@@ -1,32 +1,75 @@
1
  ---
2
  language:
3
  - ru
4
- base_model: t-tech/T-pro-it-1.0
 
5
  tags:
6
  - exl2
7
  ---
8
- # T-pro-it-1.0
9
-
10
  **_This is a converted version of the original [T-pro-it-1.0](https://huggingface.co/t-tech/T-pro-it-1.0) model into EXL2._**
11
 
 
 
 
 
 
12
  **🚨 T-pro is designed for further fine-tuning and is not intended as a ready-to-use conversational assistant. Users are advised to exercise caution and are responsible for any additional training and oversight required to ensure the model's responses meet acceptable ethical and safety standards. The responsibility for incorporating this model into industrial or commercial solutions lies entirely with those who choose to deploy it.**
13
 
14
  ## Description
15
 
16
- T-pro-it-1.0 was trained in bf16.
17
-
18
- Detailed model card’s coming soon…
19
 
20
  ### 📚 Dataset
21
 
22
- Detailed model card’s coming soon…
 
 
 
 
 
 
 
 
 
 
23
 
24
  ## 📊 Benchmarks
25
 
26
- Detailed model card’s coming soon…
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
 
28
  ## 👨‍💻 Examples of usage
29
 
 
30
 
31
  ```python
32
  from transformers import AutoTokenizer, AutoModelForCausalLM
@@ -82,4 +125,31 @@ Output:
82
  Поиск закономерностей — его цель, открыть тайны бытия.
83
  От распознавания лиц до понимания речи,
84
  Машинное обучение — это ключ, что открывает двери.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
85
  ```
 
1
  ---
2
  language:
3
  - ru
4
+ base_model:
5
+ - t-tech/T-pro-it-1.0
6
  tags:
7
  - exl2
8
  ---
 
 
9
  **_This is a converted version of the original [T-pro-it-1.0](https://huggingface.co/t-tech/T-pro-it-1.0) model into EXL2._**
10
 
11
+
12
+ # Original model card:
13
+
14
+ # T-pro-it-1.0
15
+
16
  **🚨 T-pro is designed for further fine-tuning and is not intended as a ready-to-use conversational assistant. Users are advised to exercise caution and are responsible for any additional training and oversight required to ensure the model's responses meet acceptable ethical and safety standards. The responsibility for incorporating this model into industrial or commercial solutions lies entirely with those who choose to deploy it.**
17
 
18
  ## Description
19
 
20
+ T-pro-it-1.0 is a model built upon the Qwen 2.5 model family and incorporates both continual pre-training and alignment techniques.
 
 
21
 
22
  ### 📚 Dataset
23
 
24
+ Pre-training Stage 1:
25
+ 100B tokens, consisting of diverse Russian data from Common Crawl, books, code, and proprietary datasets, mixed with re-played English data (English added as it is the primary language of the base model).
26
+
27
+ Pre-training Stage 2:
28
+ 40B tokens, a mix of instruction and pre-training data.
29
+
30
+ Supervised Fine-Tuning (SFT):
31
+ 1B tokens, a mix of diverse instruction data.
32
+
33
+ Preference Tuning:
34
+ 1B tokens, training the model to be helpful.
35
 
36
  ## 📊 Benchmarks
37
 
38
+ Proprietary models:
39
+
40
+ | Benchmark | T-pro-it-1.0 | GPT-4o | GPT-4o-mini | GigaChat Max 1.0.26.20 |
41
+ |------------------------------------------------|-----------------------|------------------------------|-----------------------|---------------------|
42
+ | [MERA](https://mera.a-ai.ru) | <u>0.629</u> | **0.642** | 0.57 | 0.588 |
43
+ | [MaMuRaMu](https://mera.a-ai.ru/ru/tasks/22) | <u>0.841</u> | **0.874** | 0.779 | 0.824 |
44
+ | ruMMLU-PRO | <u>0.665</u> | **0.713** | 0.573 | 0.535 |
45
+ | ruGSM8K | **0.941** | <u>0.931</u> | 0.888 | 0.892 |
46
+ | ruMATH | **0.776** | <u>0.771</u> | 0.724 | 0.589 |
47
+ | ruMBPP | **0.805** | <u>0.802</u> | 0.79 | 0.626 |
48
+ | [ruCodeEval](https://mera.a-ai.ru/ru/tasks/23) | 0.432 / 0.626 / 0.677 | <u>0.529 / 0.649 / 0.683</u> | **0.704 / 0.753 / 0.768** | 0.077 / 0.093 / 0.098 |
49
+ | Arena-Hard-Ru | **90.17** | <u>84.87</u> | 81 | - |
50
+ | MT Bench Ru | <u>8.7</u> | **8.706** | 8.45 | 8.53 |
51
+ | Alpaca Eval Ru | <u>47.61</u> | **50** | 45.51 | 38.13 |
52
+
53
+ Open-source models:
54
+
55
+ | Benchmark | T-pro-it-1.0 | Qwen-2.5-32B-Instruct | RuAdapt-Qwen-32B-Instruct-v1 | gemma-2-27b-it | Llama-3.3-70B-Instruct |
56
+ |------------------------------------------------|---------------------------|-------------------------------|------------------------------|------------------------------|------------------------|
57
+ | [MERA](https://mera.a-ai.ru) | **0.629** | 0.578 | <u>0.615</u> | 0.574 | 0.567 |
58
+ | [MaMuRaMu](https://mera.a-ai.ru/ru/tasks/22) | **0.841** | <u>0.824</u> | 0.812 | 0.768 | 0.818 |
59
+ | ruMMLU-PRO | **0.665** | 0.637 | 0.631 | 0.470 | <u>0.653</u> |
60
+ | ruGSM8K | **0.941** | 0.926 | 0.923 | 0.894 | <u>0.934</u> |
61
+ | ruMATH | **0.776** | 0.727 | <u>0.742</u> | 0.538 | 0.636 |
62
+ | ruMBPP | 0.805 | **0.825** | <u>0.813</u> | 0.708 | 0.77 |
63
+ | [ruCodeEval](https://mera.a-ai.ru/ru/tasks/23) | **0.432 / 0.626 / 0.677** | 0.06 / 0.098 / 0.116 | 0.426 / 0.561 / 0.598 | <u>0.259 / 0.586 / 0.689</u> | 0.112 / 0.166 / 0.189 |
64
+ | Arena-Hard-Ru | **90.17** | 74.54 | <u>80.23</u> | 66.4 | 76.51 |
65
+ | MT Bench Ru | **8.7** | 8.15 | 8.39 | 7.96 | <u>8.26</u> |
66
+ | Alpaca Eval Ru | **47.61** | 35.01 | <u>43.15</u> | 38.82 | - |
67
+
68
+ Detailed evaluation results can be found in our [habr post](https://habr.com/ru/companies/tbank/articles/865582/)
69
 
70
  ## 👨‍💻 Examples of usage
71
 
72
+ ### HF Usage
73
 
74
  ```python
75
  from transformers import AutoTokenizer, AutoModelForCausalLM
 
125
  Поиск закономерностей — его цель, открыть тайны бытия.
126
  От распознавания лиц до понимания речи,
127
  Машинное обучение — это ключ, что открывает двери.
128
+ ```
129
+
130
+ ### VLLM Usage
131
+
132
+ ```python
133
+ from transformers import AutoTokenizer
134
+ from vllm import LLM, SamplingParams
135
+
136
+ model_name = "t-tech/T-pro-it-1.0"
137
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
138
+ llm = LLM(model=model_name, max_model_len=8192)
139
+ sampling_params = SamplingParams(temperature=0.7,
140
+ repetition_penalty=1.05,
141
+ top_p=0.8, top_k=70)
142
+
143
+ prompt = "Напиши стих про машинное обучение"
144
+ messages = [
145
+ {"role": "system", "content": "Ты T-pro, виртуальный ассистент в Т-Технологии. Твоя задача - быть полезным диалоговым ассистентом."},
146
+ {"role": "user", "content": prompt}
147
+ ]
148
+
149
+ prompt_token_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
150
+
151
+ outputs = llm.generate(prompt_token_ids=prompt_token_ids, sampling_params=sampling_params)
152
+
153
+ generated_text = [output.outputs[0].text for output in outputs]
154
+ print(generated_text)
155
  ```