Spaces:
Running
Running
Jan Biermeyer
commited on
Commit
·
c8a749d
1
Parent(s):
0241ab6
tonekizer fixed
Browse files- requirements.txt +2 -0
requirements.txt
CHANGED
|
@@ -21,6 +21,8 @@ bitsandbytes>=0.43.0 # For 8-bit quantization (CPU compatible)
|
|
| 21 |
|
| 22 |
# NLP utilities
|
| 23 |
nltk>=3.8.0
|
|
|
|
|
|
|
| 24 |
|
| 25 |
# Utilities
|
| 26 |
python-dotenv>=1.0.0
|
|
|
|
| 21 |
|
| 22 |
# NLP utilities
|
| 23 |
nltk>=3.8.0
|
| 24 |
+
sentencepiece>=0.1.99 # Required for Llama/Mistral tokenizers
|
| 25 |
+
protobuf>=3.20.0 # Required by sentencepiece
|
| 26 |
|
| 27 |
# Utilities
|
| 28 |
python-dotenv>=1.0.0
|