Hugging Face's logo

ybWw
/
bge-base-financial-matryoshka

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:6300
loss:MatryoshkaLoss
loss:MultipleNegativesRankingLoss
Eval Results
text-embeddings-inference
Model card Files
xet
Community
bge-base-financial-matryoshka / README.md
ybWw's picture
ybWw
Add new SentenceTransformer model
6a73c76 verified
preview code
|
raw
history blame contribute delete
29.6 kB
metadata
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:6300
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en-v1.5
widget:
  - source_sentence: >-
      In 2023, the foreign exchange effect on APAC's revenues showed an
      unfavorable impact of $1,759 million.
    sentences:
      - >-
        How did the change in non-rolling chip win percentage at The Venetian
        Macao from 2022 to 2023 affect overall revenues?
      - What was the foreign exchange effect on APAC's revenues in 2023?
      - What are the characteristics of Garmin’s Force Trolling Motors?
  - source_sentence: >-
      The estimated cost of inventory for expected product returns was $226
      million and $194 million as of May 31, 2023 and 2022, respectively, and
      was recorded in Prepaid expenses and other current assets on the
      Consolidated Balance Sheets.
    sentences:
      - >-
        What factors influenced the cash conversion cycle during fiscal year
        2023?
      - >-
        What were the estimated costs of inventory for expected product returns
        as of May 31, 2023 and 2022, and how were these costs recorded on the
        Consolidated Balance Sheets?
      - >-
        What are the potential consequences of the company being involved in
        lawsuits related to their acquisitions?
  - source_sentence: >-
      The Concession requires the company to increase its investment in
      non-gaming projects by up to 20% in the following year if Macao's annual
      market gross gaming revenue achieves or exceeds 180 billion patacas.
    sentences:
      - >-
        What is the required increase in non-gaming investment for the company
        if Macao's annual gross gaming revenue exceeds a certain threshold?
      - What is the valuation allowance of the company as of January 31, 2023?
      - >-
        What was the percentage decrease in net operating revenues due to
        foreign currency exchange rate fluctuations in 2023?
  - source_sentence: >-
      During the 2023 audit, a critical matter was the auditing of Delta Air
      Lines' employee benefit plan assets. Specifically, the valuation of assets
      measured at Net Asset Value (NAV) posed a challenge due to the significant
      judgment involved in estimating their fair value, primarily affected by a
      delay in obtaining relevant data from investment fund managers.
    sentences:
      - What is the mission of the company described in the text?
      - >-
        What was the percentage increase or decrease in total research and
        development expenses for the company from the previous year?
      - >-
        What significant accounting matter related to Delta Air Lines' employee
        benefit plans was addressed in the 2023 audit?
  - source_sentence: >-
      A 1% increase in medical cost PMPM trend factors led to an increase in
      medical costs payable by $1,128 million for the most recent two months as
      of December 31, 2023.
    sentences:
      - >-
        What financial impact would a 1% increase in medical cost PMPM trend
        factors have on medical costs payable for the most recent two months as
        of December 31, 2023?
      - How does Kroger present its financial performance in its reporting?
      - >-
        What type of merchandising strategy does AutoZone employ to ensure
        customer satisfaction?
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: SentenceTransformer based on BAAI/bge-base-en-v1.5
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.6842857142857143
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8185714285714286
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8571428571428571
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.6842857142857143
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27285714285714285
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1714285714285714
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08999999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.6842857142857143
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8185714285714286
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8571428571428571
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.7945477585006253
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7606122448979591
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7642505598444385
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.6771428571428572
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8128571428571428
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8571428571428571
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.6771428571428572
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27095238095238094
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1714285714285714
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08999999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.6771428571428572
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8128571428571428
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8571428571428571
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.7898291331612092
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7544075963718821
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7578545086967818
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.6828571428571428
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8128571428571428
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8557142857142858
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.89
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.6828571428571428
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27095238095238094
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17114285714285712
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08899999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.6828571428571428
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8128571428571428
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8557142857142858
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.89
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.789006338619091
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7562443310657596
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7601401161539556
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.6657142857142857
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8314285714285714
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.88
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.6657142857142857
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.26666666666666666
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.16628571428571426
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.088
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.6657142857142857
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8314285714285714
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.88
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.774536493588219
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.740644557823129
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7444027281070795
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.6357142857142857
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.7685714285714286
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8085714285714286
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.8514285714285714
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.6357142857142857
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2561904761904762
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1617142857142857
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08514285714285713
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.6357142857142857
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.7685714285714286
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8085714285714286
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.8514285714285714
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.7446661935095178
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7103339002267574
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7149306052438406
            name: Cosine Map@100

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: BAAI/bge-base-en-v1.5
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 768 tokens
  • Similarity Function: Cosine Similarity
  • Training Dataset:
    • json

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ybWw/bge-base-financial-matryoshka")
# Run inference
sentences = [
    'A 1% increase in medical cost PMPM trend factors led to an increase in medical costs payable by $1,128 million for the most recent two months as of December 31, 2023.',
    'What financial impact would a 1% increase in medical cost PMPM trend factors have on medical costs payable for the most recent two months as of December 31, 2023?',
    'How does Kroger present its financial performance in its reporting?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Metric Value
cosine_accuracy@1 0.6843
cosine_accuracy@3 0.8186
cosine_accuracy@5 0.8571
cosine_accuracy@10 0.9
cosine_precision@1 0.6843
cosine_precision@3 0.2729
cosine_precision@5 0.1714
cosine_precision@10 0.09
cosine_recall@1 0.6843
cosine_recall@3 0.8186
cosine_recall@5 0.8571
cosine_recall@10 0.9
cosine_ndcg@10 0.7945
cosine_mrr@10 0.7606
cosine_map@100 0.7643

Information Retrieval

Metric Value
cosine_accuracy@1 0.6771
cosine_accuracy@3 0.8129
cosine_accuracy@5 0.8571
cosine_accuracy@10 0.9
cosine_precision@1 0.6771
cosine_precision@3 0.271
cosine_precision@5 0.1714
cosine_precision@10 0.09
cosine_recall@1 0.6771
cosine_recall@3 0.8129
cosine_recall@5 0.8571
cosine_recall@10 0.9
cosine_ndcg@10 0.7898
cosine_mrr@10 0.7544
cosine_map@100 0.7579

Information Retrieval

Metric Value
cosine_accuracy@1 0.6829
cosine_accuracy@3 0.8129
cosine_accuracy@5 0.8557
cosine_accuracy@10 0.89
cosine_precision@1 0.6829
cosine_precision@3 0.271
cosine_precision@5 0.1711
cosine_precision@10 0.089
cosine_recall@1 0.6829
cosine_recall@3 0.8129
cosine_recall@5 0.8557
cosine_recall@10 0.89
cosine_ndcg@10 0.789
cosine_mrr@10 0.7562
cosine_map@100 0.7601

Information Retrieval

Metric Value
cosine_accuracy@1 0.6657
cosine_accuracy@3 0.8
cosine_accuracy@5 0.8314
cosine_accuracy@10 0.88
cosine_precision@1 0.6657
cosine_precision@3 0.2667
cosine_precision@5 0.1663
cosine_precision@10 0.088
cosine_recall@1 0.6657
cosine_recall@3 0.8
cosine_recall@5 0.8314
cosine_recall@10 0.88
cosine_ndcg@10 0.7745
cosine_mrr@10 0.7406
cosine_map@100 0.7444

Information Retrieval

Metric Value
cosine_accuracy@1 0.6357
cosine_accuracy@3 0.7686
cosine_accuracy@5 0.8086
cosine_accuracy@10 0.8514
cosine_precision@1 0.6357
cosine_precision@3 0.2562
cosine_precision@5 0.1617
cosine_precision@10 0.0851
cosine_recall@1 0.6357
cosine_recall@3 0.7686
cosine_recall@5 0.8086
cosine_recall@10 0.8514
cosine_ndcg@10 0.7447
cosine_mrr@10 0.7103
cosine_map@100 0.7149

Training Details

Training Dataset

json

  • Dataset: json
  • Size: 6,300 training samples
  • Columns: positive and anchor
  • Approximate statistics based on the first 1000 samples:
    positive anchor
    type string string
    details
    • min: 6 tokens
    • mean: 46.17 tokens
    • max: 272 tokens
    • min: 8 tokens
    • mean: 20.72 tokens
    • max: 42 tokens
  • Samples:
    positive anchor
    For fiscal 2023, the net cash provided by operating activities was $7,111 million. What was the net cash provided by operating activities in fiscal 2023?
    Penalties for impermissible use or disclosure of PHI were increased by the HITECH Act by imposing tiered penalties of more than $50,000 per violation and up to $1.5 million per year for identical violations. What are the consequences of impermissible use or disclosure of PHI according to the HITECH Act?
    Global Day of Joy is Hasbro’s annual, company-wide day of service and has become a cherished tradition. Global Day of Joy takes place every December, and employees from each Hasbro office participate in service projects to benefit a variety of organizations. What is the purpose of the Global Day of Joy at Hasbro?
  • Loss: MatryoshkaLoss with these parameters:
    {
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: epoch
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 16
  • gradient_accumulation_steps: 16
  • learning_rate: 2e-05
  • num_train_epochs: 4
  • lr_scheduler_type: cosine
  • warmup_ratio: 0.1
  • bf16: True
  • tf32: True
  • load_best_model_at_end: True
  • optim: adamw_torch_fused
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: epoch
  • prediction_loss_only: True
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 16
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 16
  • eval_accumulation_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 4
  • max_steps: -1
  • lr_scheduler_type: cosine
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.1
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: True
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: True
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional

Training Logs

Epoch Step Training Loss dim_768_cosine_map@100 dim_512_cosine_map@100 dim_256_cosine_map@100 dim_128_cosine_map@100 dim_64_cosine_map@100
0.8122 10 1.5383 - - - - -
0.9746 12 - 0.7575 0.7597 0.7428 0.7303 0.6889
1.6244 20 0.6278 - - - - -
1.9492 24 - 0.7623 0.7580 0.7582 0.7417 0.7100
2.4365 30 0.4388 - - - - -
2.9239 36 - 0.7649 0.7576 0.7571 0.7465 0.7142
3.2487 40 0.3729 - - - - -
3.8985 48 - 0.7643 0.7579 0.7601 0.7444 0.7149
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.8.20
  • Sentence Transformers: 3.2.1
  • Transformers: 4.41.2
  • PyTorch: 2.1.2+cu121
  • Accelerate: 1.0.1
  • Datasets: 2.19.1
  • Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss 

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss 

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}