Add new SentenceTransformer model

0c85e90 verified 8 months ago

31.6 kB

metadata

language:
  - en
license: apache-2.0
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:5822
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: nomic-ai/modernbert-embed-base
widget:
  - source_sentence: >-
      publicly announced on August 27, 2018 that it was writing down the
      carrying 

      value of its investment in DR to “nil,” after acknowledging that DR’s
      failure to 

      achieve FDA approval for its technology and missing “milestone and
      projected 

      revenue targets.”72  Although Senetas understood that such an action
      could 

      negatively affect DR’s future funding,73 it states that, as a publicly
      traded company
    sentences:
      - What is the plaintiff required to do with the Complaint?
      - >-
        What potential effect on DR did Senetas understand could result from
        writing down the investment?
      - >-
        What is the Federal Circuit case citation number for Shea v. United
        States?
  - source_sentence: >-
      has any subpoena or contempt authority, the sort of powers incident to
      Congress’s “broad power 

      of inquiry.”  Soucie, 448 F.2d at 1075 & n.27; see McGrain, 273 U.S. at
      168–69, 180. 

      Given these considerations, the Commission does not exercise “substantial
      independent 

      authority.”  Accord Flaherty v. Ross, 373 F. Supp. 3d 97, 106–10 (D.D.C.
      2019).5  The upshot is
    sentences:
      - Who had personal knowledge of the first and third events?
      - What is said about the Commission's exercise of authority?
      - What are the page references for the decision in Cole?
  - source_sentence: >-
      pending.  The Court will begin by explaining why it denies the plaintiff’s
      motion for leave to file 

      a second amended complaint in No. 11-445.  The Court will then discuss the
      plaintiff’s Motion 

      for Sanctions, filed in No. 11-443.  Third, the Court will address the
      plaintiff’s two remaining 

      policy-or-practice claims, which challenge the CIA’s Assignment of Rights
      Policy and
    sentences:
      - What type of motion did the court deny the plaintiff?
      - What policies are being challenged by the plaintiff’s remaining claims?
      - >-
        What is cited in the location where the argument about crafting task
        orders is made?
  - source_sentence: >-
      (11th ed. 2003).  While Defendant correctly identifies that labor costs
      are often contemplated in 

      pricing firm fixed-price contracts, the same is true for a wide variety of
      other costs.  See, e.g., 

      Lakeshore Eng’g Servs., 748 F.3d at 1343 (“[T]he mechanism of pricing such
      jobs involves 

      identification of costs for those jobs, including labor, equipment, and
      materials . . . .”).  

      60
    sentences:
      - >-
        What other costs, besides labor, are involved in the pricing mechanism
        of jobs according to Lakeshore Eng’g Servs.?
      - What did the letters seek from the recipients regarding dmed.ai?
      - >-
        To which community do the sensitive operations related to the withheld
        information belong?
  - source_sentence: >-
      protests pursuant to 28 U.S.C. § 1491(b).  See 28 U.S.C. § 1491(b). 
      Section 1491(b)(1) grants the 

      17 
       
      court jurisdiction over protests filed “by an interested party objecting
      to a solicitation by a Federal 

      agency for bids or proposals for a proposed contract . . . or any alleged
      violation of statute or
    sentences:
      - Which agency's declaration is mentioned?
      - Under which U.S. Code section are the protests filed?
      - What does the camera do when it begins and ends recording?
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: ModernBERT Embed base Legal Matryoshka
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.5919629057187017
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.6352395672333848
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.7032457496136012
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7666151468315301
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5919629057187017
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.5682637815558991
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.42627511591962897
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.24080370942812984
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.20118495620814011
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5470118495620814
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6664090674909839
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.750772797527048
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.6774170353263864
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6316503520522063
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6706536977021021
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.5857805255023184
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.616692426584235
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6908809891808346
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7666151468315301
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5857805255023184
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.5574446161772283
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.4176197836166924
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.24173106646058728
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.19835136527563108
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5352910870685214
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.651468315301391
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7518031942297784
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.6721586770579754
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6236285910551748
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.662035963048261
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.5672333848531684
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5873261205564142
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6646058732612056
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7310664605873262
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5672333848531684
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.538382277176713
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.40092735703245747
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.23075734157650693
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.19062339000515197
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5151983513652756
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6263523956723338
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.720504894384338
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.6453735180059756
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.6008856504992514
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6377344841740619
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.4992272024729521
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5301391035548686
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6136012364760433
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.678516228748068
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.4992272024729521
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.47449768160741884
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.3653786707882534
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.215919629057187
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.16950025759917567
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.45814013395157127
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.5705821741370428
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.6683410613086037
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5891910709474154
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5385613699369493
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.5782906402819172
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.36321483771251933
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.401854714064915
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.47295208655332305
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.527047913446677
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.36321483771251933
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.35136527563111797
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.2782071097372488
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.16506955177743432
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.12364760432766615
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.33912931478619274
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.43637300360638853
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.5142967542503863
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.44443933760269555
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4003011457030003
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.44619919912465567
            name: Cosine Map@100

ModernBERT Embed base Legal Matryoshka

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: nomic-ai/modernbert-embed-base
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- json
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("manishh16/modernbert-embed-base-legal-matryoshka-2")
# Run inference
sentences = [
    'protests pursuant to 28 U.S.C. § 1491(b).  See 28 U.S.C. § 1491(b).  Section 1491(b)(1) grants the \n17 \n \ncourt jurisdiction over protests filed “by an interested party objecting to a solicitation by a Federal \nagency for bids or proposals for a proposed contract . . . or any alleged violation of statute or',
    'Under which U.S. Code section are the protests filed?',
    "Which agency's declaration is mentioned?",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: dim_768
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 768
}
```

Metric	Value
cosine_accuracy@1	0.592
cosine_accuracy@3	0.6352
cosine_accuracy@5	0.7032
cosine_accuracy@10	0.7666
cosine_precision@1	0.592
cosine_precision@3	0.5683
cosine_precision@5	0.4263
cosine_precision@10	0.2408
cosine_recall@1	0.2012
cosine_recall@3	0.547
cosine_recall@5	0.6664
cosine_recall@10	0.7508
cosine_ndcg@10	0.6774
cosine_mrr@10	0.6317
cosine_map@100	0.6707

Information Retrieval

Dataset: dim_512
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 512
}
```

Metric	Value
cosine_accuracy@1	0.5858
cosine_accuracy@3	0.6167
cosine_accuracy@5	0.6909
cosine_accuracy@10	0.7666
cosine_precision@1	0.5858
cosine_precision@3	0.5574
cosine_precision@5	0.4176
cosine_precision@10	0.2417
cosine_recall@1	0.1984
cosine_recall@3	0.5353
cosine_recall@5	0.6515
cosine_recall@10	0.7518
cosine_ndcg@10	0.6722
cosine_mrr@10	0.6236
cosine_map@100	0.662

Information Retrieval

Dataset: dim_256
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 256
}
```

Metric	Value
cosine_accuracy@1	0.5672
cosine_accuracy@3	0.5873
cosine_accuracy@5	0.6646
cosine_accuracy@10	0.7311
cosine_precision@1	0.5672
cosine_precision@3	0.5384
cosine_precision@5	0.4009
cosine_precision@10	0.2308
cosine_recall@1	0.1906
cosine_recall@3	0.5152
cosine_recall@5	0.6264
cosine_recall@10	0.7205
cosine_ndcg@10	0.6454
cosine_mrr@10	0.6009
cosine_map@100	0.6377

Information Retrieval

Dataset: dim_128
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 128
}
```

Metric	Value
cosine_accuracy@1	0.4992
cosine_accuracy@3	0.5301
cosine_accuracy@5	0.6136
cosine_accuracy@10	0.6785
cosine_precision@1	0.4992
cosine_precision@3	0.4745
cosine_precision@5	0.3654
cosine_precision@10	0.2159
cosine_recall@1	0.1695
cosine_recall@3	0.4581
cosine_recall@5	0.5706
cosine_recall@10	0.6683
cosine_ndcg@10	0.5892
cosine_mrr@10	0.5386
cosine_map@100	0.5783

Information Retrieval

Dataset: dim_64
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 64
}
```

Metric	Value
cosine_accuracy@1	0.3632
cosine_accuracy@3	0.4019
cosine_accuracy@5	0.473
cosine_accuracy@10	0.527
cosine_precision@1	0.3632
cosine_precision@3	0.3514
cosine_precision@5	0.2782
cosine_precision@10	0.1651
cosine_recall@1	0.1236
cosine_recall@3	0.3391
cosine_recall@5	0.4364
cosine_recall@10	0.5143
cosine_ndcg@10	0.4444
cosine_mrr@10	0.4003
cosine_map@100	0.4462

Training Details

Training Dataset

json

Dataset: json
Size: 5,822 training samples
Columns: positive and anchor
Approximate statistics based on the first 1000 samples:
positive anchor
type string string
details
min: 26 tokens
mean: 96.76 tokens
max: 156 tokens

min: 8 tokens
mean: 16.59 tokens
max: 49 tokens

	positive	anchor
type	string	string
details	min: 26 tokens mean: 96.76 tokens max: 156 tokens	min: 8 tokens mean: 16.59 tokens max: 49 tokens

Samples:

positive	anchor
`properly authenticated. See id. at 367, 19 A.3d at 429 (Harrell, J., dissenting). Four years later, in Sublet, 442 Md. at 637-38, 113 A.3d at 697-98, we adopted the reasonable juror test for social media evidence and applied it in the three cases that were consolidated for purposes of the opinion: Sublet v. State, Harris v. State, and Monge-`	`How many years after the dissent did the adoption of the reasonable juror test occur?`
`to (1) a public-interest fee waiver, (2) the expedited processing of a request, or (3) the release of information that implicates personal privacy, all are personal to a requester and thus cannot be assigned. See, e.g., RTC Commercial Loan Trust 1995-NP1A v. Winthrop Mgmt., 923 F. Supp. 83, 88 (E.D. Va. 1996) (holding that “certain rights are purely personal and cannot be assigned”).`	`What type of fee waiver is mentioned as being personal to a requester?`
`‘IRO’] staff that reviews Agency records and makes public release determinations with an eye toward evaluating directorate-specific equities.” Id. ¶ 4. Ms. Meeks also explains that “records frequently involve the equities of multiple directorates,” and “[w]hen records implicate the operational interests of multiple directorates, the reviews are conducted by the relevant IROs`	`Who conducts the reviews when the records implicate the operational interests of multiple directorates?`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
gradient_accumulation_steps: 16
learning_rate: 2e-05
num_train_epochs: 4
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: False
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 16
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 4
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: False
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_768_cosine_ndcg@10	dim_512_cosine_ndcg@10	dim_256_cosine_ndcg@10	dim_128_cosine_ndcg@10	dim_64_cosine_ndcg@10
0.8791	10	91.6964	-	-	-	-	-
1.0	12	-	0.6483	0.6445	0.6004	0.5232	0.4001
1.7033	20	39.6429	-	-	-	-	-
2.0	24	-	0.6764	0.6716	0.6361	0.5736	0.4374
2.5275	30	30.1905	-	-	-	-	-
3.0	36	-	0.6768	0.6699	0.6441	0.5869	0.4416
3.3516	40	26.8879	-	-	-	-	-
3.7033	44	-	0.6774	0.6722	0.6454	0.5892	0.4444

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.11.11
Sentence Transformers: 4.0.1
Transformers: 4.50.2
PyTorch: 2.6.0+cu124
Accelerate: 1.5.2
Datasets: 3.5.0
Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}