osanseviero commited on
Commit
6d09a72
·
verified ·
1 Parent(s): adb7468

Add minor reference to transformers

Browse files
Files changed (1) hide show
  1. README.md +18 -1
README.md CHANGED
@@ -13,7 +13,7 @@ Mistral-7B-v0.3 has the following changes compared to [Mistral-7B-v0.2](https://
13
 
14
  ## Installation
15
 
16
- It is recommended to use `mistralai/Mistral-7B-Instruct-v0.3` with [mistral-inference](https://github.com/mistralai/mistral-inference)
17
 
18
  ```
19
  pip install mistral_inference
@@ -115,6 +115,23 @@ result = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])
115
  print(result)
116
  ```
117
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
118
  ## Limitations
119
 
120
  The Mistral 7B Instruct model is a quick demonstration that the base model can be easily fine-tuned to achieve compelling performance.
 
13
 
14
  ## Installation
15
 
16
+ It is recommended to use `mistralai/Mistral-7B-Instruct-v0.3` with [mistral-inference](https://github.com/mistralai/mistral-inference). For HF transformers code snippets, please keep scrolling.
17
 
18
  ```
19
  pip install mistral_inference
 
115
  print(result)
116
  ```
117
 
118
+ ## Generate with `transformers`
119
+
120
+ If you want to use Hugging Face `transformers` to generate text, you can do something like this.
121
+
122
+ ```py
123
+ from transformers import AutoModelForCausalLM, AutoTokenizer
124
+
125
+ model_id = "mistralai/Mistral-7B-Instruct-v0.3"
126
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
127
+
128
+ model = AutoModelForCausalLM.from_pretrained(model_id)
129
+ inputs = tokenizer("Hello my name is", return_tensors="pt")
130
+
131
+ outputs = model.generate(**inputs, max_new_tokens=20)
132
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
133
+ ```
134
+
135
  ## Limitations
136
 
137
  The Mistral 7B Instruct model is a quick demonstration that the base model can be easily fine-tuned to achieve compelling performance.