feat: add vllm support

README.md

@@ -65,25 +65,6 @@ extra_gated_eu_disallowed: true
 - Pooling: lasttoken pooling
 
 
-## Training Recipe
-- High-quality supervised finetuning
-
-
-## 📑 Open-source Plan
-
-- [x] Model Checkpoint 
-    - [x] [KaLM-embedding-multilingual-mini-v1](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-v1)
-    - [x] [KaLM-embedding-multilingual-mini-instruct-v1](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1)
-    - [x] [KaLM-embedding-multilingual-mini-instruct-v1.5](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1.5)
-    - [x] [KaLM-embedding-multilingual-mini-instruct-v2](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v2)
-    - [x] [KaLM-embedding-multilingual-mini-instruct-v2.5](https://huggingface.co/KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5)
-    - [x] [KaLM-Embedding-Gemma3-12B-2511](https://huggingface.co/tencent/KaLM-Embedding-Gemma3-12B-2511)
-- [x] Training and Evaluation Code: [HITsz-TMG/KaLM-Embedding](https://github.com/HITsz-TMG/KaLM-Embedding)
-- [x] Technical Report: [KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model](https://arxiv.org/abs/2506.20923v4)
-- [x] Pre-training Data: [Pre-training Data](https://huggingface.co/datasets/HIT-TMG/KaLM-embedding-pretrain-data)
-- [x] Fine-tuning Data: [Fine-tuning Data](https://huggingface.co/datasets/KaLM-Embedding/KaLM-embedding-finetuning-data)
-
-
 ## Usage
 ### sentence-transformers support
 Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
@@ -162,6 +143,27 @@ tensor([[0.9034, 0.2563],
 '''
 ```
 
+### vllm support
+Note: Since [vllm](https://github.com/vllm-project/vllm/tree/main) only supports the [Gemma3ForCausalLM](https://huggingface.co/docs/transformers/en/model_doc/gemma3#transformers.Gemma3ForCausalLM) model class and not [Gemma3TextModel](https://huggingface.co/docs/transformers/en/model_doc/gemma3#transformers.Gemma3TextModel), model parameters must be loaded by specifying the CausalLM branch via `revision="CausalLM"`.
+
+```python
+from vllm import LLM
+
+sentences = ["This is an example sentence", "Each sentence is converted"]
+
+# Create an LLM.
+# You should pass task="embed" for embedding models
+model = LLM(
+    model="tencent/KaLM-Embedding-Gemma3-12B-2511",
+    task="embed",
+    enforce_eager=True,
+    revision="CausalLM",  # specify the CausalLM branch for Gemma3ForCausalLM config
+)
+
+outputs = model.embed(sentences)
+embeddings = [output.outputs.embedding for output in outputs]
+```
+
 
 ## Citation
 If you find this model useful, please consider giving a star and citation.