UmarAzam/bge-base-en-v1.5-industrialtech

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5 on the wikipedia_subsets dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • wikipedia_subsets
• Language: en

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("UmarAzam/bge-base-en-v1.5-industrialtech")
# Run inference
sentences = [
    'Culdrose, was and on board killed Angels Aircraft BuNo 155029 and 6 BuNo, (Skyhawk) the top a loop at 1532 hrs., Niagara Falls International Airport New York, during the Western York Air Show, Lt. Cmdr . Gershon . Second pilot Lt. Andy Caputi, ejects safely with only Skyhawk crashed on while in a The demonstration team resumes duties 20, Ohio but omits maneuver resulted in crash, flies with USAF LTV Corsair II, 69‑6198, 4450th Group power, caught fire Midwest suburb Oklahoma as he attempted steer it less-populous area ejecting but fighter impacted house one missing, said Press International report found . This unit was secretly Lockheed F-117 Nighthawks at this time 8 AugustA General Dynamics F-16A Fighting, 81-0750, of the 421st Tactical Fighter, crashed during mission in northwest, killing the pilot Crashed Test Range pilot, Lieutenant S. Brad . aircraft suffered into terrain 1 SeptemberA Navy Boeing CH-46D, BuNo, \'72\' on takeoff to engine failure the Indian . The helicopter struck the . Quick response Fife´s damage the secured the helicopter was hanging the of the destroyer the deck . All crew passengers aboard without major injuries The helicopter was assigned Helicopter Support Squadron (HC-11) Det 6 aboard the combat stores ship September Texas National Guard AH-1G Cobra number 67-15737 D/1/124 CAV "Lone Star Div . take-off at',
    ' at RNAS Culdrose, was lost and all four on board killed.\n\n13 JulyBlue Angels Aircraft 5, BuNo 155029, and 6, BuNo 154992, (Douglas A-4F Skyhawk) collide at the top of a loop at 1532\xa0hrs., Niagara Falls International Airport, New York, during the Western New York Air Show \'85, killing Lt. Cmdr. Michael Gershon. Second pilot, Lt. Andy Caputi, ejects safely with only minor injuries. One Skyhawk crashed on airport grounds while the second fighter impacted in a nearby auto junkyard. The demonstration team resumes show duties 20 July at Dayton, Ohio but omits maneuver that resulted in crash, and flies with five aircraft rather than six.\n\n8 AugustA USAF LTV A-7D Corsair II, 69‑6198, of the 4450th Tactical Group, lost power, caught fire and crashed into Midwest City, a suburb of Oklahoma City, Oklahoma, pilot Maj. Dennis D. Nielson staying with aircraft as he attempted to steer it towards less-populous area before ejecting, but fighter impacted house, killing one, injuring one, one missing, said a United Press International report. Second victim found on 9 August. This unit was secretly operating Lockheed F-117 Nighthawks at this time.\n\n8 AugustA USAF General Dynamics F-16A Block 15F Fighting Falcon, 81-0750, of the 421st Tactical Fighter Squadron, crashed during a training mission in northwest Utah, killing the pilot. Crashed onto the Utah Test and Training Range killing pilot, First Lieutenant S. Brad Peale. The aircraft suffered a controlled flight into terrain (CFIT).\n\n1 SeptemberA U.S. Navy Boeing Vertol CH-46D Sea Knight, BuNo 151918, \'72\', crashed on takeoff due to an engine failure aboard the destroyer  in the Indian Ocean. The helicopter struck the Sea Sparrow launcher. Quick response of Fife´s damage control team extinguished the fires and secured the helicopter which was hanging from the side of the destroyer below the helicopter deck. All 16 crew and passengers aboard escaped without major injuries. The helicopter was assigned to Helicopter Combat Support Squadron 11 (HC-11) Det. 6 aboard the combat stores ship .\n\n15 September A Texas Army National Guard AH-1G Cobra Tail number 67-15737 of D/1/124 CAV of 49th "Lone Star" Div. crashed shortly after take-off at',
    ' astronomy at Uppsala during 1890–1897 and later at Lund, worked in several fields of astronomy, including celestial mechanics and photometry. He was one of the leading founders of stellar statistics, applying mathematical statistics to astronomical problems. || \n|-id=678\n| 8678 Bäl ||  || Bäl, is a small and typical country parish on the Swedish island of Gotland, often associated on Gotland with the well-known song "Farewell to Bäl". || \n|-id=679\n| 8679 Tingstäde ||  || Tingstäde, is a parish on Gotland. In Tingstäde Träsk, a swamp that is the second largest lake on the island, the remains of a timber construction involving some 10~000 logs, probably from the sixth century, is still visible on the lake floor. || \n|-id=680\n| 8680 Rone ||  || Rone, a small parish on Gotland, Sweden, is well known for the lyrics to the song Rune from Rone. Nearby Uggarde Rojr, a 3000-year-old burial mound from the Bronze Age with a diameter of 50 meters and a height of 7 meters, is one of the biggest in Sweden. || \n|-id=681\n| 8681 Burs ||  || Burs is a small parish on the Swedish island of Gotland. Gustav Edman (1881–1912), well known for his height (2.46 meters) and strength, was born in Burs. Burs also has the remains of the largest house (67 × 11 meters) in Sweden from the Roman Iron Age. || \n|-id=682\n| 8682 Kräklingbo ||  || Kräklingbo, is a small parish on the Swedish island of Gotland. Located here on a hill are the remains of a fortification nearly 2000 years old, the biggest in Scandinavia. From that hill many of the medieval churches on the island can be seen. || \n|-id=683\n| 8683 Sjölander ||  || Nils Göran Sjölander (born 1951), a Swedish astronomer and formerly librarian at Uppsala Observatory, studies dwarf galaxies and has a keen interest in the history of astronomy. || \n|-id=684\n| 8684 Reichwein ||  || Adolf Reichwein (1898–1944), resistance fighter in Nazi Germany || \n|-id=685\n| 8685',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9519, 0.5107],
#         [0.9519, 1.0000, 0.5335],
#         [0.5107, 0.5335, 1.0000]])

Evaluation

Metrics

Semantic Similarity

• Datasets: sts-dev and sts-test
• Evaluated with EmbeddingSimilarityEvaluator

Metric	sts-dev	sts-test
pearson_cosine	0.8243	0.8018
spearman_cosine	0.8299	0.7929

Training Details

Training Dataset

wikipedia_subsets

• Dataset: wikipedia_subsets at 72f5c2f
• Size: 127,336 training samples
• Columns: text
• Approximate statistics based on the first 1000 samples:

  	text
  type	string
  details	min: 510 tokens mean: 512.0 tokens max: 512 tokens

• Samples:

  text
  from Brocade Communications Systems) IBM 2029: Dense Wavelength Division Multiplexer (OEM from Nortel) IBM 2031: Storage area network (SAN) Fibre Channel switch (OEM from McData) IBM 2032: Storage area network (SAN) Fibre Channel switch (OEM from McData) IBM 2053: Storage area network (SAN) Fibre Channel switch (OEM from Cisco) IBM 2054: Storage area network (SAN) Fibre Channel switch (OEM from Cisco) IBM 2061: Storage area network (SAN) Fibre Channel switch (OEM from Cisco) IBM 2062: Storage area network (SAN) Fibre Channel switch (OEM from Cisco) IBM 2103-H07: SAN Fibre Channel Hub IBM 2109: Storage area network (SAN) Fibre Channel switch (OEM from Brocade Communications Systems) IBM 2498: Storage area network (SAN) Fibre Channel switch (OEM from Brocade Communications Systems) IBM 2499: Storage area network (SAN) Fibre Channel switch (OEM from Brocade Communications Systems) IBM 3534: Storage area network (SAN) Fibre Channel switch (OEM from Brocade Communications Syste...
  the Ministry of Defense was seriously wounded. Wired speculated that the assassinations could indicate that whoever was behind Stuxnet felt that it was not sufficient to stop the nuclear program. That same Wired article suggested the Iranian government could have been behind the assassinations. In January 2010, another Iranian nuclear scientist, a physics professor at Tehran University, was killed in a similar bomb explosion. On 11 January 2012, a director of the Natanz nuclear enrichment facility, Mostafa Ahmadi Roshan, was killed in an attack quite similar to the one that killed Shahriari. An analysis by the FAS demonstrates that Iran's enrichment capacity grew during 2010. The study indicated that Iran's centrifuges appeared to be performing 60% better than in the previous year, which would significantly reduce Tehran's time to produce bomb-grade uranium. The FAS report was reviewed by an official with the IAEA who affirmed the study. European and US officials, along with private...
  arred attorney and activist against obscenity and violence in media and entertainment Horace Henry White (B.A. 1886, LL.B 1887) – American lawyer, authored legal volumes White's Notarial Guide and White's Analytical Index Walton J. Wood – American attorney and jurist who served as the first public defender in United States history (1914–1921) Jurists Tamara W. Ashford (J.D. 1994) – Article I judge of the United States Tax Court Jennings Bailey (B.L. 1890) – District Judge for the United States District Court for the District of Columbia Jeffrey S. Bivins (J.D. 1986) – Chief Justice of the Supreme Court of Tennessee Claria Horn Boom (J.D. 1994) – United States district judge of the United States District Court for Eastern and Western Kentucky John P. Bourcier (J.D. 1953) – former justice of the Rhode Island Supreme Court John K. Bush (B.A. 1986) – U.S. Circuit Court Judge, United States Court of Appeals for the 6th Circuit (2017–present) Charles Hardy Carr (B.A. 1925) – United...

• Loss: DenoisingAutoEncoderLoss

Evaluation Dataset

wikipedia_subsets

• Dataset: wikipedia_subsets at 72f5c2f
• Size: 10,000 evaluation samples
• Columns: text
• Approximate statistics based on the first 1000 samples:

  	text
  type	string
  details	min: 511 tokens mean: 512.0 tokens max: 512 tokens

• Samples:

  text
  ATC may issue instructions that pilots are required to obey, or advisories (known as flight information in some countries) that pilots may, at their discretion, disregard. The pilot in command is the final authority for the safe operation of the aircraft and may, in an emergency, deviate from ATC instructions to the extent required to maintain safe operation of their aircraft. Language Pursuant to requirements of the International Civil Aviation Organization (ICAO), ATC operations are conducted either in the English language or the language used by the station on the ground. In practice, the native language for a region is used; however, English must be used upon request. History In 1920, Croydon Airport, London, was the first airport in the world to introduce air traffic control. The "aerodrome control tower" was a wooden hut high with windows on all four sides. It was commissioned on February 25, 1920 and provided basic traffic, weather and location information to pilots. In th...
  exploratory meetings in Havana, Cuba. These first contacts were meant to settle the details of where, how and when the next stage of the process – secret encounters to set an agenda for talks – would be held. In July 2011, the government appointed senior officials to participate in the process: Frank Pearl, serving as environment minister; Sergio Jaramillo Caro, national security adviser to the president; and President Santos' brother Enrique Santos, former director of El Tiempo. For the magazine Semana, Enrique Santos' inclusion was a 'gesture of confidence' by President Santos to the guerrilla, because of the familial ties between the two men and Enrique Santos' past involvement in dialogues with the guerrilla. The FARC negotiating team was joined by Mauricio Jaramillo and Marcos Calarcá. Secret negotiations continued despite the death of Alfonso Cano, the FARC leader, in a military operation in November 2011. Semana reported that both negotiating parties had agreed to the principl...
  word, “Nahidagsa” or “Dinagsa,” which means “to swarm, to invade or to flock.” Jose Flores and his family discovered the site around the early 1920s. Before the barangay site was settled, the area was then forested. Engaged in primitive farming, Flores claimed huge tracts of land in the area. Until such time, that people from the poblacion and the neighboring barrios settled and flocked the site. These migrants, forming a small sitio, decided to celebrate this accomplishment with a fiesta. At that time, a vendor came to the hamlet selling a statue of San Jose. Accordingly, the hamlet heads came into an agreement to purchase the image and made Saint Joseph as patron saint of Dagsa. It was at this moment that the annual fiesta date of the barangay falls every March 19. The village is also famed for its waterfalls, which is a fifteen-minute trek from the barangay site. Hibod-Hibod Early records account that Hibod-hibod existed as a hamlet in 1947 under the civil jurisdiction of baranga...

• Loss: DenoisingAutoEncoderLoss

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• learning_rate: 3e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 3e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	sts-dev_spearman_cosine	sts-test_spearman_cosine
-1	-1	-	-	0.8950	-
0.0031	100	10.3472	-	-	-
0.0063	200	8.3377	-	-	-
0.0094	300	7.7643	-	-	-
0.0126	400	7.5862	-	-	-
0.0157	500	7.4872	-	-	-
0.0188	600	7.431	-	-	-
0.0220	700	7.3389	-	-	-
0.0251	800	7.2523	-	-	-
0.0283	900	7.1291	-	-	-
0.0314	1000	7.0278	6.9694	0.8918	-
0.0346	1100	6.9028	-	-	-
0.0377	1200	6.8726	-	-	-
0.0408	1300	6.7327	-	-	-
0.0440	1400	6.7287	-	-	-
0.0471	1500	6.6202	-	-	-
0.0503	1600	6.5443	-	-	-
0.0534	1700	6.4895	-	-	-
0.0565	1800	6.4378	-	-	-
0.0597	1900	6.3352	-	-	-
0.0628	2000	6.2969	6.2575	0.8869	-
0.0660	2100	6.1986	-	-	-
0.0691	2200	6.1851	-	-	-
0.0722	2300	6.149	-	-	-
0.0754	2400	6.1183	-	-	-
0.0785	2500	6.0767	-	-	-
0.0817	2600	6.0205	-	-	-
0.0848	2700	5.985	-	-	-
0.0880	2800	5.9859	-	-	-
0.0911	2900	5.9257	-	-	-
0.0942	3000	5.8159	5.8102	0.8842	-
0.0974	3100	5.8286	-	-	-
0.1005	3200	5.7575	-	-	-
0.1037	3300	5.7128	-	-	-
0.1068	3400	5.6786	-	-	-
0.1099	3500	5.6711	-	-	-
0.1131	3600	5.6193	-	-	-
0.1162	3700	5.6226	-	-	-
0.1194	3800	5.5549	-	-	-
0.1225	3900	5.5437	-	-	-
0.1257	4000	5.4732	5.4955	0.8806	-
0.1288	4100	5.4374	-	-	-
0.1319	4200	5.3952	-	-	-
0.1351	4300	5.4191	-	-	-
0.1382	4400	5.4089	-	-	-
0.1414	4500	5.3452	-	-	-
0.1445	4600	5.3458	-	-	-
0.1476	4700	5.3801	-	-	-
0.1508	4800	5.3075	-	-	-
0.1539	4900	5.2999	-	-	-
0.1571	5000	5.2472	5.2619	0.8765	-
0.1602	5100	5.191	-	-	-
0.1633	5200	5.2209	-	-	-
0.1665	5300	5.2038	-	-	-
0.1696	5400	5.2406	-	-	-
0.1728	5500	5.1717	-	-	-
0.1759	5600	5.1279	-	-	-
0.1791	5700	5.1836	-	-	-
0.1822	5800	5.1161	-	-	-
0.1853	5900	5.1219	-	-	-
0.1885	6000	5.1243	5.1005	0.8735	-
0.1916	6100	5.155	-	-	-
0.1948	6200	5.087	-	-	-
0.1979	6300	5.0865	-	-	-
0.2010	6400	5.0264	-	-	-
0.2042	6500	5.032	-	-	-
0.2073	6600	5.0212	-	-	-
0.2105	6700	4.9717	-	-	-
0.2136	6800	5.0071	-	-	-
0.2167	6900	5.0103	-	-	-
0.2199	7000	4.9357	4.9584	0.8724	-
0.2230	7100	4.9565	-	-	-
0.2262	7200	4.9408	-	-	-
0.2293	7300	4.931	-	-	-
0.2325	7400	4.8922	-	-	-
0.2356	7500	4.9181	-	-	-
0.2387	7600	4.9021	-	-	-
0.2419	7700	4.8602	-	-	-
0.2450	7800	4.9398	-	-	-
0.2482	7900	4.9074	-	-	-
0.2513	8000	4.8251	4.8419	0.8689	-
0.2544	8100	4.8566	-	-	-
0.2576	8200	4.8288	-	-	-
0.2607	8300	4.8351	-	-	-
0.2639	8400	4.8141	-	-	-
0.2670	8500	4.7755	-	-	-
0.2702	8600	4.8115	-	-	-
0.2733	8700	4.7736	-	-	-
0.2764	8800	4.7721	-	-	-
0.2796	8900	4.7012	-	-	-
0.2827	9000	4.8072	4.7406	0.8655	-
0.2859	9100	4.7441	-	-	-
0.2890	9200	4.7136	-	-	-
0.2921	9300	4.745	-	-	-
0.2953	9400	4.7384	-	-	-
0.2984	9500	4.661	-	-	-
0.3016	9600	4.6335	-	-	-
0.3047	9700	4.6959	-	-	-
0.3078	9800	4.625	-	-	-
0.3110	9900	4.7273	-	-	-
0.3141	10000	4.7072	4.6561	0.8615	-
0.3173	10100	4.6342	-	-	-
0.3204	10200	4.6606	-	-	-
0.3236	10300	4.657	-	-	-
0.3267	10400	4.6195	-	-	-
0.3298	10500	4.6763	-	-	-
0.3330	10600	4.6475	-	-	-
0.3361	10700	4.6147	-	-	-
0.3393	10800	4.6247	-	-	-
0.3424	10900	4.5936	-	-	-
0.3455	11000	4.5609	4.5800	0.8585	-
0.3487	11100	4.559	-	-	-
0.3518	11200	4.5905	-	-	-
0.3550	11300	4.5575	-	-	-
0.3581	11400	4.5924	-	-	-
0.3612	11500	4.5825	-	-	-
0.3644	11600	4.5578	-	-	-
0.3675	11700	4.5742	-	-	-
0.3707	11800	4.5391	-	-	-
0.3738	11900	4.5596	-	-	-
0.3770	12000	4.4874	4.5099	0.8566	-
0.3801	12100	4.532	-	-	-
0.3832	12200	4.4948	-	-	-
0.3864	12300	4.5366	-	-	-
0.3895	12400	4.545	-	-	-
0.3927	12500	4.4721	-	-	-
0.3958	12600	4.4681	-	-	-
0.3989	12700	4.469	-	-	-
0.4021	12800	4.4814	-	-	-
0.4052	12900	4.5382	-	-	-
0.4084	13000	4.4786	4.4597	0.8515	-
0.4115	13100	4.422	-	-	-
0.4147	13200	4.4686	-	-	-
0.4178	13300	4.4084	-	-	-
0.4209	13400	4.4259	-	-	-
0.4241	13500	4.4519	-	-	-
0.4272	13600	4.4467	-	-	-
0.4304	13700	4.4647	-	-	-
0.4335	13800	4.39	-	-	-
0.4366	13900	4.4241	-	-	-
0.4398	14000	4.4488	4.4065	0.8506	-
0.4429	14100	4.3923	-	-	-
0.4461	14200	4.4596	-	-	-
0.4492	14300	4.3667	-	-	-
0.4523	14400	4.4501	-	-	-
0.4555	14500	4.3571	-	-	-
0.4586	14600	4.3877	-	-	-
0.4618	14700	4.4558	-	-	-
0.4649	14800	4.3584	-	-	-
0.4681	14900	4.411	-	-	-
0.4712	15000	4.3778	4.3572	0.8500	-
0.4743	15100	4.3908	-	-	-
0.4775	15200	4.3076	-	-	-
0.4806	15300	4.3315	-	-	-
0.4838	15400	4.3367	-	-	-
0.4869	15500	4.336	-	-	-
0.4900	15600	4.331	-	-	-
0.4932	15700	4.351	-	-	-
0.4963	15800	4.3209	-	-	-
0.4995	15900	4.3554	-	-	-
0.5026	16000	4.3224	4.3209	0.8472	-
0.5057	16100	4.3311	-	-	-
0.5089	16200	4.322	-	-	-
0.5120	16300	4.3634	-	-	-
0.5152	16400	4.3304	-	-	-
0.5183	16500	4.3295	-	-	-
0.5215	16600	4.3121	-	-	-
0.5246	16700	4.3006	-	-	-
0.5277	16800	4.2614	-	-	-
0.5309	16900	4.3475	-	-	-
0.5340	17000	4.3133	4.2841	0.8468	-
0.5372	17100	4.3047	-	-	-
0.5403	17200	4.2768	-	-	-
0.5434	17300	4.2894	-	-	-
0.5466	17400	4.234	-	-	-
0.5497	17500	4.2807	-	-	-
0.5529	17600	4.3028	-	-	-
0.5560	17700	4.2595	-	-	-
0.5592	17800	4.3193	-	-	-
0.5623	17900	4.243	-	-	-
0.5654	18000	4.2656	4.2499	0.8422	-
0.5686	18100	4.2928	-	-	-
0.5717	18200	4.2857	-	-	-
0.5749	18300	4.2464	-	-	-
0.5780	18400	4.2631	-	-	-
0.5811	18500	4.27	-	-	-
0.5843	18600	4.2945	-	-	-
0.5874	18700	4.2068	-	-	-
0.5906	18800	4.2322	-	-	-
0.5937	18900	4.2418	-	-	-
0.5968	19000	4.1714	4.2251	0.8409	-
0.6000	19100	4.2393	-	-	-
0.6031	19200	4.153	-	-	-
0.6063	19300	4.2169	-	-	-
0.6094	19400	4.2302	-	-	-
0.6126	19500	4.2307	-	-	-
0.6157	19600	4.2149	-	-	-
0.6188	19700	4.143	-	-	-
0.6220	19800	4.1904	-	-	-
0.6251	19900	4.2463	-	-	-
0.6283	20000	4.2314	4.1942	0.8388	-
0.6314	20100	4.2125	-	-	-
0.6345	20200	4.2346	-	-	-
0.6377	20300	4.2259	-	-	-
0.6408	20400	4.1786	-	-	-
0.6440	20500	4.1379	-	-	-
0.6471	20600	4.2254	-	-	-
0.6502	20700	4.2269	-	-	-
0.6534	20800	4.1565	-	-	-
0.6565	20900	4.2129	-	-	-
0.6597	21000	4.226	4.1734	0.8404	-
0.6628	21100	4.1841	-	-	-
0.6660	21200	4.1172	-	-	-
0.6691	21300	4.159	-	-	-
0.6722	21400	4.1531	-	-	-
0.6754	21500	4.1903	-	-	-
0.6785	21600	4.1821	-	-	-
0.6817	21700	4.1583	-	-	-
0.6848	21800	4.238	-	-	-
0.6879	21900	4.1866	-	-	-
0.6911	22000	4.1435	4.1537	0.8387	-
0.6942	22100	4.1315	-	-	-
0.6974	22200	4.1852	-	-	-
0.7005	22300	4.1223	-	-	-
0.7037	22400	4.1397	-	-	-
0.7068	22500	4.1068	-	-	-
0.7099	22600	4.1622	-	-	-
0.7131	22700	4.2065	-	-	-
0.7162	22800	4.1434	-	-	-
0.7194	22900	4.1234	-	-	-
0.7225	23000	4.0956	4.1336	0.8365	-
0.7256	23100	4.1458	-	-	-
0.7288	23200	4.1617	-	-	-
0.7319	23300	4.1244	-	-	-
0.7351	23400	4.127	-	-	-
0.7382	23500	4.1105	-	-	-
0.7413	23600	4.1451	-	-	-
0.7445	23700	4.1275	-	-	-
0.7476	23800	4.1049	-	-	-
0.7508	23900	4.1308	-	-	-
0.7539	24000	4.136	4.1163	0.8343	-
0.7571	24100	4.1141	-	-	-
0.7602	24200	4.1334	-	-	-
0.7633	24300	4.21	-	-	-
0.7665	24400	4.1238	-	-	-
0.7696	24500	4.175	-	-	-
0.7728	24600	4.1295	-	-	-
0.7759	24700	4.0938	-	-	-
0.7790	24800	4.0994	-	-	-
0.7822	24900	4.1181	-	-	-
0.7853	25000	4.0947	4.1008	0.8334	-
0.7885	25100	4.1724	-	-	-
0.7916	25200	4.0633	-	-	-
0.7947	25300	4.1391	-	-	-
0.7979	25400	4.0763	-	-	-
0.8010	25500	4.144	-	-	-
0.8042	25600	4.0499	-	-	-
0.8073	25700	4.0879	-	-	-
0.8105	25800	4.0466	-	-	-
0.8136	25900	4.1114	-	-	-
0.8167	26000	4.1317	4.0859	0.8317	-
0.8199	26100	4.0735	-	-	-
0.8230	26200	4.0672	-	-	-
0.8262	26300	4.0624	-	-	-
0.8293	26400	4.0972	-	-	-
0.8324	26500	4.1008	-	-	-
0.8356	26600	4.034	-	-	-
0.8387	26700	4.0665	-	-	-
0.8419	26800	4.0938	-	-	-
0.8450	26900	4.0661	-	-	-
0.8481	27000	4.0533	4.0766	0.8308	-
0.8513	27100	4.0373	-	-	-
0.8544	27200	4.0699	-	-	-
0.8576	27300	4.0583	-	-	-
0.8607	27400	4.0354	-	-	-
0.8639	27500	4.0874	-	-	-
0.8670	27600	4.1063	-	-	-
0.8701	27700	4.0701	-	-	-
0.8733	27800	4.0937	-	-	-
0.8764	27900	4.0728	-	-	-
0.8796	28000	4.1167	4.0648	0.8302	-
0.8827	28100	4.0884	-	-	-
0.8858	28200	4.0893	-	-	-
0.8890	28300	4.1053	-	-	-
0.8921	28400	4.1227	-	-	-
0.8953	28500	4.0107	-	-	-
0.8984	28600	4.0814	-	-	-
0.9016	28700	4.0591	-	-	-
0.9047	28800	4.0424	-	-	-
0.9078	28900	4.0209	-	-	-
0.9110	29000	4.0668	4.0563	0.8308	-
0.9141	29100	4.0698	-	-	-
0.9173	29200	4.0294	-	-	-
0.9204	29300	4.0519	-	-	-
0.9235	29400	4.0626	-	-	-
0.9267	29500	4.0963	-	-	-
0.9298	29600	4.0785	-	-	-
0.9330	29700	4.0212	-	-	-
0.9361	29800	4.0567	-	-	-
0.9392	29900	4.1014	-	-	-
0.9424	30000	4.0272	4.0486	0.8301	-
0.9455	30100	4.0466	-	-	-
0.9487	30200	4.0446	-	-	-
0.9518	30300	4.0253	-	-	-
0.9550	30400	4.0528	-	-	-
0.9581	30500	4.0786	-	-	-
0.9612	30600	4.0663	-	-	-
0.9644	30700	4.0342	-	-	-
0.9675	30800	4.0533	-	-	-
0.9707	30900	4.0597	-	-	-
0.9738	31000	4.0389	4.0437	0.8299	-
0.9769	31100	4.0713	-	-	-
0.9801	31200	4.0543	-	-	-
0.9832	31300	4.0239	-	-	-
0.9864	31400	4.0993	-	-	-
0.9895	31500	4.0426	-	-	-
0.9926	31600	4.0237	-	-	-
0.9958	31700	4.0243	-	-	-
0.9989	31800	4.0755	-	-	-
-1	-1	-	-	-	0.7929

Framework Versions

• Python: 3.10.18
• Sentence Transformers: 5.0.0
• Transformers: 4.53.2
• PyTorch: 2.7.1+cu126
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

DenoisingAutoEncoderLoss

@inproceedings{wang-2021-TSDAE,
    title = "TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding Learning",
    author = "Wang, Kexin and Reimers, Nils and Gurevych, Iryna",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
    month = nov,
    year = "2021",
    address = "Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    pages = "671--688",
    url = "https://arxiv.org/abs/2104.06979",
}