dnth/ssf-retriever-modernbert-embed-base-v2

SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the ssf-train-valid-v2 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nomic-ai/modernbert-embed-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid-v2

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("dnth/ssf-retriever-modernbert-embed-base-v2")
# Run inference
sentences = [
    "The Financial Planner/Insurance Agent/Bancassurance Specialist is responsible for developing and implementing financial plans to help customers meet their objectives, and managing customer relationships. He/She understands customer profiles and recommends suitable solutions to them. He is also in charge of attracting his own customers through networking sessions, relationship building and referrals. The Financial Planner/Insurance Agent/Bancassurance Specialist's duties might require him to work on weekends and after office hours and may involve travel to customers' locations. He has a friendly and outgoing nature and is able to build rapport with customers easily in order to establish trust.",
    'The Financial Planner/Insurance Agent/Bancassurance Specialist designs and executes tailored financial strategies to assist clients in achieving their financial goals while maintaining strong client relationships. This role requires a thorough understanding of client needs to propose appropriate financial products. The specialist actively seeks new clients through networking, referrals, and relationship management. Flexibility in working hours, including evenings, weekends, and occasional travel to client sites, is essential. Strong interpersonal skills and an approachable demeanor enable effective trust-building with customers.',
    'The Financial Analyst/Investment Consultant/Bancassurance Coordinator conducts in-depth market research and analyzes investment portfolios to provide strategic advice to corporate clients. This role focuses on evaluating financial data rather than direct customer acquisition and requires collaboration with internal teams rather than individual networking. Work hours are generally standard office hours with minimal client site visits. The position demands analytical proficiency and detailed reporting skills, with limited emphasis on personal rapport or direct sales activities.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8627, 0.5380],
#         [0.8627, 1.0000, 0.5945],
#         [0.5380, 0.5945, 1.0000]])

Training Details

Training Dataset

ssf-train-valid-v2

• Dataset: ssf-train-valid-v2 at fb19ed3
• Size: 3,016 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 57 tokens mean: 167.72 tokens max: 403 tokens	min: 54 tokens mean: 135.01 tokens max: 250 tokens	min: 15 tokens mean: 100.13 tokens max: 212 tokens

• Samples:

  anchor	positive	negative
  The Senior Technician (Automatic Fare Collection) is responsible for preventive and corrective maintenance of Automatic Fare Collection (AFC) systems. His/Her duties include troubleshooting of faults, providing technical guidance and on-the-job coaching to his team, as well as supervising the work of contractors and external stakeholders in ensuring compliance to safety requirements and operating standards. He is required to carry out his duties in the depot, workshop and/or at various train stations during train operating hours. He is technically inclined and well-verse in various AFC systems and machines and builds good teamwork amongst his team to support maintenance activities.	The Senior Technician (Automatic Fare Collection) oversees both preventive and corrective maintenance tasks related to Automatic Fare Collection (AFC) systems. This role involves diagnosing system faults, offering expert technical advice, and mentoring team members on operational procedures. The technician also manages contractors and external partners, ensuring adherence to safety protocols and operational guidelines. Responsibilities are performed across depots, workshops, and multiple train stations during service hours. A solid technical background in AFC equipment and strong team leadership skills are essential to effectively support maintenance operations.	The Senior Technician (Automatic Fare Collection) is responsible for conducting routine inspections and repairs of automated ticket vending machines in retail environments. This role includes assisting customers with machine usage, coordinating with sales teams, and ensuring compliance with commercial service standards. Work is primarily carried out within shopping malls during business hours. The technician requires customer service skills and basic technical knowledge of vending equipment but is not involved in supervising maintenance teams or external contractors.
  The Venue Operations Manager is responsible for overseeing the running of venue operations, including the logistics requirements. He/She works closely with event services department to ensure client requirements are fulfilled in compliance to local health and safety standards. He reviews event plans to ensure generation of maximum yield for organisation. Meticulous and resourceful, he possesses excellent problem-solving skills and is able to react quickly to deviations in the project plans. He is able to work in a flexible workweek, including weekends, evenings, and public holidays, and is comfortable working in both an indoor and outdoor environment depending on the nature and requirements of the events.	The Venue Operations Manager oversees all aspects of venue management, coordinating logistics and collaborating with the event services team to meet client expectations while adhering to local health and safety regulations. This role involves reviewing event strategies to optimize organizational revenue and demands strong problem-solving abilities to address unforeseen challenges promptly. The manager must be adaptable to a variable schedule, including weekends, evenings, and public holidays, and comfortable operating in diverse environments, both indoor and outdoor, based on event needs.	The Venue Marketing Manager is responsible for developing and executing promotional campaigns for the venue, working closely with the sales department to attract clients and increase bookings. They analyze market trends to maximize revenue opportunities and must be creative and communicative, addressing branding challenges as they arise. This role typically operates standard weekday hours and focuses primarily on indoor office environments rather than on-site event coordination.
  The Liquefied Natural Gas Research Analyst supports the LNG Trading team in identifying opportunities for closing deals and increasing portfolio value. He/She conducts research using market and economic data for the preparation of market reports and analyses data regarding risks associated with trading deals. He engages clients on presentations on market insights and liaises with key stakeholders for the preparation of hedging activities. He works in a dynamic and fast-paced environment where he must provide accurate analyses and research material to support the closing of deals. He is a decisive, analytical and self-motivated individual who is comfortable with numbers and able to work under pressure.	The Liquefied Natural Gas Research Analyst collaborates closely with the LNG Trading team to discover profitable trading opportunities and enhance portfolio performance. This role involves gathering and analyzing market and economic data to produce comprehensive reports and assess the risks linked to trading transactions. The analyst presents market insights to clients and coordinates with stakeholders to facilitate hedging strategies. Operating in a fast-moving and challenging environment, the analyst must deliver precise research and data analyses to aid deal execution. Candidates should be analytical, proactive, confident with numerical data, and capable of working effectively under tight deadlines.	The Renewable Energy Project Coordinator manages the planning and execution of solar and wind energy initiatives within the energy sector. This position requires coordinating with multiple vendors and regulatory bodies, overseeing project timelines, and ensuring compliance with environmental standards. The coordinator communicates progress updates to stakeholders and resolves logistical challenges to keep projects on track. Strong organizational skills, experience in project management, and the ability to navigate regulatory frameworks are essential. This role focuses on operational delivery rather than market research or trading analysis.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid-v2

• Dataset: ssf-train-valid-v2 at fb19ed3
• Size: 754 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 754 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 58 tokens mean: 166.89 tokens max: 347 tokens	min: 58 tokens mean: 136.23 tokens max: 256 tokens	min: 19 tokens mean: 99.02 tokens max: 202 tokens

• Samples:

  anchor	positive	negative
  The Head of Client Implementation is responsible for driving strategic relationship building activities and ensuring client implementation plans are carried out at standards that are satisfactory to clients. He/She oversees the team's compliance with implementation policies and regulations when executing tasks during implementation, often collaborating with relevant internal teams for the successful design and delivery of client implementation plans. The Head of Client Implementation possesses strong leadership, interpersonal and teamwork skills. His work environment is highly diverse and evolves based on clients' requirements and needs. He is flexible to change and has quick problem-solving skills.	The Head of Client Implementation leads the strategic development of client relationships and ensures that implementation projects meet client expectations and quality standards. This role supervises the team's adherence to implementation guidelines and regulatory requirements, coordinating closely with internal departments to design and deliver effective client onboarding plans. The Head of Client Implementation demonstrates strong leadership, excellent communication, and teamwork abilities. Working in a dynamic and multicultural environment, this individual adapts swiftly to changing client demands and efficiently resolves challenges.	The Project Manager in Client Services focuses on managing project timelines and resource allocation within the client support division. Instead of overseeing implementation strategies, this role emphasizes coordinating operational activities and ensuring service delivery efficiency. The Project Manager requires strong organizational and planning skills and works under a structured reporting hierarchy, primarily handling routine client service requests rather than strategic relationship management.
  The Set Designer executes the realisation of the overall visual aspects of the production by conceptualising the set design or locations for film, video or a concept based on the script and the overall visual concept. He/She is responsible for studying scripts and interpreting the descriptions of locations, creating sketches and drawings, translating these into technical drawings and models for sets to be built or locations to be identified and prepared. He lays out a comprehensive set of requirements for the materials needed for set design. He oversees the right look or feel for a production and ensure that other areas of technical production such as lighting or sound recording are aligned with the set. He also identifies and commissions set construction vendors. He prepares cost estimates for set design and construction and manages project schedule of his team. He oversees the construction of set and ensures that the sets evoke the intended style, mood and theme. He/She may specialis...	The Set Designer is responsible for bringing the visual concept of a production to life by designing and coordinating the physical environment for film, video, or other visual media based on the script and overall artistic direction. This role involves analyzing scripts to interpret location and set requirements, producing sketches and detailed technical drawings, and developing models for set construction or location preparation. The Set Designer specifies material needs, supervises the aesthetic alignment of the set with lighting and sound departments, and manages relationships with construction vendors. They prepare cost estimates, oversee project timelines, and ensure sets reflect the intended mood, style, and theme. Specialization in set decoration or prop management may be required, with responsibilities including the artistic selection and upkeep of props. The role demands creativity, strong coordination with other technical teams, proficiency in design techniques and fabricatio...	The Set Designer manages the logistical planning and operational execution of event setups, coordinating venue layouts and technical support for conferences, exhibitions, and corporate functions. Responsible for assessing client requirements, preparing floor plans, and directing the installation of audio-visual equipment and staging, the Set Designer ensures all elements align with event objectives and branding guidelines. They oversee vendor
  The Senior Producer/Producer - News is responsible for managing the daily news agenda, overall news content and flow of a newscast. He/She has an overview of the agendas being pursued and looks out for newsworthy stories that will attract and engage target audience. He manages the resourcing of news content and is responsible for assigning news stories to internal teams. He is also required to edit scripts and reports that are submitted for the newscast and ensure that editorial guidelines and policies set by the organisation are adhered to. He is in charge of maintaining the newscast's presence and image on social media platforms and monitors the competitors' activities and trends. In addition, he collaborates with the production teams to ensure continuity of content across different programmes throughout the day. He typically works in a newsroom but may be assigned field activities at times. He should be able to adapt quickly to changes and perform well in a dynamic environment. He s...	The Senior Producer/Producer - News oversees the daily planning and execution of news broadcasts, ensuring the seamless flow and compelling content throughout the newscast. This role involves identifying and prioritizing news stories that resonate with the target audience while managing the allocation of editorial resources across teams. The incumbent reviews and refines scripts and reports to guarantee compliance with the organization’s editorial policies. Additionally, this position maintains the newscast’s brand presence on social media channels and keeps abreast of competitor activities and industry trends. Collaboration with production units is essential to maintain content consistency across multiple programs during the broadcast day. Operating primarily within a newsroom environment, the Senior Producer/Producer must be adaptable to fast-paced changes and possess solid knowledge of both local and global current events. Experience with production technology such as cameras, audio...	The Marketing Coordinator - News Media is responsible for planning and executing marketing campaigns to promote news programming across various digital platforms. This role focuses on developing brand awareness strategies, managing advertising budgets, and coordinating with external agencies for content creation and distribution. The incumbent monitors market trends and audience analytics to optimize campaign performance but does not

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: False
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	6	0.1487	0.0112
2.0	12	0.0095	0.0037
3.0	18	0.0047	0.0026
4.0	24	0.0041	0.0023
5.0	30	0.0032	0.0022

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.8
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}