tmy100000001/LitDD_crossencoder

CrossEncoder based on ncbi/MedCPT-Cross-Encoder

This is a Cross Encoder model finetuned from ncbi/MedCPT-Cross-Encoder using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: ncbi/MedCPT-Cross-Encoder
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
    ['Its8, a fission yeast homolog of Mcd4 and Pig-n, is involved in GPI anchor synthesis and shares an essential function with calcineurin in cytokinesis. In fission yeast, calcineurin is required for cytokinesis and ion homeostasis; however, most of its physiological roles remain obscure. To identify genes that share an essential function with calcineurin, we screened for mutations that confer sensitivity to the calcineurin inhibitor FK506 and high temperature and isolated the mutant its8-1. its8(+) encodes a homolog of the budding yeast MCD4 and human Pig-n that are involved in glycosylphosphatidylinositol (GPI) anchor synthesis. Consistently, reduced inositol labeling of proteins suggested impaired GPI anchor synthesis in its8-1 mutants. The temperature upshift induced a further decrease in inositol labeling and caused dramatic increases in the frequency of septation in its8-1 mutants. BE49385A, an inhibitor of MCD4 and Pig-n, also increased the septation index of the wild-type cell. Osmotic stabilization suppressed these morphological defects, indicating that cell wall weakness caused by impaired GPI anchor synthesis resulted in abnormal cytokinesis. Furthermore, calcineurin-deleted cells exhibited hypersensitivity to BE49385A, and FK506 exacerbated the cytokinesis defects of the its8-1 mutant. Thus, calcineurin and Its8 may share an essential function in cytokinesis and cell viability through the regulation of cell wall integrity.', 'G2P01414 - PIGN - 606097.0 - HGNC:8967 - MCD4; PIG-N - PIGN-related multiple congenital anomalies-hypotonia-seizures syndrome - 614080 - nan - biallelic_autosomal - nan - definitive - absent gene product; altered gene product structure - inframe_insertion; missense_variant; inframe_deletion - undetermined - inferred'],
    ['Its8, a fission yeast homolog of Mcd4 and Pig-n, is involved in GPI anchor synthesis and shares an essential function with calcineurin in cytokinesis. In fission yeast, calcineurin is required for cytokinesis and ion homeostasis; however, most of its physiological roles remain obscure. To identify genes that share an essential function with calcineurin, we screened for mutations that confer sensitivity to the calcineurin inhibitor FK506 and high temperature and isolated the mutant its8-1. its8(+) encodes a homolog of the budding yeast MCD4 and human Pig-n that are involved in glycosylphosphatidylinositol (GPI) anchor synthesis. Consistently, reduced inositol labeling of proteins suggested impaired GPI anchor synthesis in its8-1 mutants. The temperature upshift induced a further decrease in inositol labeling and caused dramatic increases in the frequency of septation in its8-1 mutants. BE49385A, an inhibitor of MCD4 and Pig-n, also increased the septation index of the wild-type cell. Osmotic stabilization suppressed these morphological defects, indicating that cell wall weakness caused by impaired GPI anchor synthesis resulted in abnormal cytokinesis. Furthermore, calcineurin-deleted cells exhibited hypersensitivity to BE49385A, and FK506 exacerbated the cytokinesis defects of the its8-1 mutant. Thus, calcineurin and Its8 may share an essential function in cytokinesis and cell viability through the regulation of cell wall integrity.', 'G2P02996 - MFF - 614785.0 - HGNC:24858 - C2ORF33; GL004 - MFF-related encephalopathy due to defective mitochondrial and peroxisomal fission - 617086 - nan - biallelic_autosomal - nan - definitive - absent gene product - nan - loss of function - inferred'],
    ['Its8, a fission yeast homolog of Mcd4 and Pig-n, is involved in GPI anchor synthesis and shares an essential function with calcineurin in cytokinesis. In fission yeast, calcineurin is required for cytokinesis and ion homeostasis; however, most of its physiological roles remain obscure. To identify genes that share an essential function with calcineurin, we screened for mutations that confer sensitivity to the calcineurin inhibitor FK506 and high temperature and isolated the mutant its8-1. its8(+) encodes a homolog of the budding yeast MCD4 and human Pig-n that are involved in glycosylphosphatidylinositol (GPI) anchor synthesis. Consistently, reduced inositol labeling of proteins suggested impaired GPI anchor synthesis in its8-1 mutants. The temperature upshift induced a further decrease in inositol labeling and caused dramatic increases in the frequency of septation in its8-1 mutants. BE49385A, an inhibitor of MCD4 and Pig-n, also increased the septation index of the wild-type cell. Osmotic stabilization suppressed these morphological defects, indicating that cell wall weakness caused by impaired GPI anchor synthesis resulted in abnormal cytokinesis. Furthermore, calcineurin-deleted cells exhibited hypersensitivity to BE49385A, and FK506 exacerbated the cytokinesis defects of the its8-1 mutant. Thus, calcineurin and Its8 may share an essential function in cytokinesis and cell viability through the regulation of cell wall integrity.', 'G2P01166 - CERT1 - 604677.0 - HGNC:2205 - CERT; COL4A3BP; GPBP; STARD11 - CERT1-related intellectual disability - 616351 - nan - monoallelic_autosomal - restricted mutation set - definitive - altered gene product structure - nan - gain of function - inferred'],
    ['Its8, a fission yeast homolog of Mcd4 and Pig-n, is involved in GPI anchor synthesis and shares an essential function with calcineurin in cytokinesis. In fission yeast, calcineurin is required for cytokinesis and ion homeostasis; however, most of its physiological roles remain obscure. To identify genes that share an essential function with calcineurin, we screened for mutations that confer sensitivity to the calcineurin inhibitor FK506 and high temperature and isolated the mutant its8-1. its8(+) encodes a homolog of the budding yeast MCD4 and human Pig-n that are involved in glycosylphosphatidylinositol (GPI) anchor synthesis. Consistently, reduced inositol labeling of proteins suggested impaired GPI anchor synthesis in its8-1 mutants. The temperature upshift induced a further decrease in inositol labeling and caused dramatic increases in the frequency of septation in its8-1 mutants. BE49385A, an inhibitor of MCD4 and Pig-n, also increased the septation index of the wild-type cell. Osmotic stabilization suppressed these morphological defects, indicating that cell wall weakness caused by impaired GPI anchor synthesis resulted in abnormal cytokinesis. Furthermore, calcineurin-deleted cells exhibited hypersensitivity to BE49385A, and FK506 exacerbated the cytokinesis defects of the its8-1 mutant. Thus, calcineurin and Its8 may share an essential function in cytokinesis and cell viability through the regulation of cell wall integrity.', 'G2P01646 - STAT2 - 600556.0 - HGNC:11363 - STAT113 - STAT2-related viral induced severe multiorgan dysfunction related with impaired mitochondrial fission - nan - nan - biallelic_autosomal - nan - limited - absent gene product - nan - loss of function - inferred'],
    ['Its8, a fission yeast homolog of Mcd4 and Pig-n, is involved in GPI anchor synthesis and shares an essential function with calcineurin in cytokinesis. In fission yeast, calcineurin is required for cytokinesis and ion homeostasis; however, most of its physiological roles remain obscure. To identify genes that share an essential function with calcineurin, we screened for mutations that confer sensitivity to the calcineurin inhibitor FK506 and high temperature and isolated the mutant its8-1. its8(+) encodes a homolog of the budding yeast MCD4 and human Pig-n that are involved in glycosylphosphatidylinositol (GPI) anchor synthesis. Consistently, reduced inositol labeling of proteins suggested impaired GPI anchor synthesis in its8-1 mutants. The temperature upshift induced a further decrease in inositol labeling and caused dramatic increases in the frequency of septation in its8-1 mutants. BE49385A, an inhibitor of MCD4 and Pig-n, also increased the septation index of the wild-type cell. Osmotic stabilization suppressed these morphological defects, indicating that cell wall weakness caused by impaired GPI anchor synthesis resulted in abnormal cytokinesis. Furthermore, calcineurin-deleted cells exhibited hypersensitivity to BE49385A, and FK506 exacerbated the cytokinesis defects of the its8-1 mutant. Thus, calcineurin and Its8 may share an essential function in cytokinesis and cell viability through the regulation of cell wall integrity.', 'G2P03036 - NDUFA8 - 603359.0 - HGNC:7692 - MGC793; PGIV - NDUFA8-related developmental disorder - nan - nan - biallelic_autosomal - nan - strong - absent gene product - nan - loss of function - inferred'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'Its8, a fission yeast homolog of Mcd4 and Pig-n, is involved in GPI anchor synthesis and shares an essential function with calcineurin in cytokinesis. In fission yeast, calcineurin is required for cytokinesis and ion homeostasis; however, most of its physiological roles remain obscure. To identify genes that share an essential function with calcineurin, we screened for mutations that confer sensitivity to the calcineurin inhibitor FK506 and high temperature and isolated the mutant its8-1. its8(+) encodes a homolog of the budding yeast MCD4 and human Pig-n that are involved in glycosylphosphatidylinositol (GPI) anchor synthesis. Consistently, reduced inositol labeling of proteins suggested impaired GPI anchor synthesis in its8-1 mutants. The temperature upshift induced a further decrease in inositol labeling and caused dramatic increases in the frequency of septation in its8-1 mutants. BE49385A, an inhibitor of MCD4 and Pig-n, also increased the septation index of the wild-type cell. Osmotic stabilization suppressed these morphological defects, indicating that cell wall weakness caused by impaired GPI anchor synthesis resulted in abnormal cytokinesis. Furthermore, calcineurin-deleted cells exhibited hypersensitivity to BE49385A, and FK506 exacerbated the cytokinesis defects of the its8-1 mutant. Thus, calcineurin and Its8 may share an essential function in cytokinesis and cell viability through the regulation of cell wall integrity.',
    [
        'G2P01414 - PIGN - 606097.0 - HGNC:8967 - MCD4; PIG-N - PIGN-related multiple congenital anomalies-hypotonia-seizures syndrome - 614080 - nan - biallelic_autosomal - nan - definitive - absent gene product; altered gene product structure - inframe_insertion; missense_variant; inframe_deletion - undetermined - inferred',
        'G2P02996 - MFF - 614785.0 - HGNC:24858 - C2ORF33; GL004 - MFF-related encephalopathy due to defective mitochondrial and peroxisomal fission - 617086 - nan - biallelic_autosomal - nan - definitive - absent gene product - nan - loss of function - inferred',
        'G2P01166 - CERT1 - 604677.0 - HGNC:2205 - CERT; COL4A3BP; GPBP; STARD11 - CERT1-related intellectual disability - 616351 - nan - monoallelic_autosomal - restricted mutation set - definitive - altered gene product structure - nan - gain of function - inferred',
        'G2P01646 - STAT2 - 600556.0 - HGNC:11363 - STAT113 - STAT2-related viral induced severe multiorgan dysfunction related with impaired mitochondrial fission - nan - nan - biallelic_autosomal - nan - limited - absent gene product - nan - loss of function - inferred',
        'G2P03036 - NDUFA8 - 603359.0 - HGNC:7692 - MGC793; PGIV - NDUFA8-related developmental disorder - nan - nan - biallelic_autosomal - nan - strong - absent gene product - nan - loss of function - inferred',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Classification

• Dataset: anno_test
• Evaluated with CrossEncoderClassificationEvaluator

Metric	Value
accuracy	0.824
accuracy_threshold	0.9996
f1	0.3842
f1_threshold	0.9992
precision	0.2698
recall	0.6667
average_precision	0.329

Training Details

Training Dataset

Unnamed Dataset

• Size: 35,474 training samples
• Columns: tiab, g2p_lgmde, and label
• Approximate statistics based on the first 1000 samples:

  	tiab	g2p_lgmde	label
  type	string	string	int
  details	min: 30 characters mean: 1348.52 characters max: 2802 characters	min: 177 characters mean: 258.9 characters max: 407 characters	0: ~72.90% 1: ~27.10%

• Samples:

  tiab	g2p_lgmde	label
  Its8, a fission yeast homolog of Mcd4 and Pig-n, is involved in GPI anchor synthesis and shares an essential function with calcineurin in cytokinesis. In fission yeast, calcineurin is required for cytokinesis and ion homeostasis; however, most of its physiological roles remain obscure. To identify genes that share an essential function with calcineurin, we screened for mutations that confer sensitivity to the calcineurin inhibitor FK506 and high temperature and isolated the mutant its8-1. its8(+) encodes a homolog of the budding yeast MCD4 and human Pig-n that are involved in glycosylphosphatidylinositol (GPI) anchor synthesis. Consistently, reduced inositol labeling of proteins suggested impaired GPI anchor synthesis in its8-1 mutants. The temperature upshift induced a further decrease in inositol labeling and caused dramatic increases in the frequency of septation in its8-1 mutants. BE49385A, an inhibitor of MCD4 and Pig-n, also increased the septation index of the wild-type cell. Os...	G2P01414 - PIGN - 606097.0 - HGNC:8967 - MCD4; PIG-N - PIGN-related multiple congenital anomalies-hypotonia-seizures syndrome - 614080 - nan - biallelic_autosomal - nan - definitive - absent gene product; altered gene product structure - inframe_insertion; missense_variant; inframe_deletion - undetermined - inferred	1
  Its8, a fission yeast homolog of Mcd4 and Pig-n, is involved in GPI anchor synthesis and shares an essential function with calcineurin in cytokinesis. In fission yeast, calcineurin is required for cytokinesis and ion homeostasis; however, most of its physiological roles remain obscure. To identify genes that share an essential function with calcineurin, we screened for mutations that confer sensitivity to the calcineurin inhibitor FK506 and high temperature and isolated the mutant its8-1. its8(+) encodes a homolog of the budding yeast MCD4 and human Pig-n that are involved in glycosylphosphatidylinositol (GPI) anchor synthesis. Consistently, reduced inositol labeling of proteins suggested impaired GPI anchor synthesis in its8-1 mutants. The temperature upshift induced a further decrease in inositol labeling and caused dramatic increases in the frequency of septation in its8-1 mutants. BE49385A, an inhibitor of MCD4 and Pig-n, also increased the septation index of the wild-type cell. Os...	G2P02996 - MFF - 614785.0 - HGNC:24858 - C2ORF33; GL004 - MFF-related encephalopathy due to defective mitochondrial and peroxisomal fission - 617086 - nan - biallelic_autosomal - nan - definitive - absent gene product - nan - loss of function - inferred	0
  Its8, a fission yeast homolog of Mcd4 and Pig-n, is involved in GPI anchor synthesis and shares an essential function with calcineurin in cytokinesis. In fission yeast, calcineurin is required for cytokinesis and ion homeostasis; however, most of its physiological roles remain obscure. To identify genes that share an essential function with calcineurin, we screened for mutations that confer sensitivity to the calcineurin inhibitor FK506 and high temperature and isolated the mutant its8-1. its8(+) encodes a homolog of the budding yeast MCD4 and human Pig-n that are involved in glycosylphosphatidylinositol (GPI) anchor synthesis. Consistently, reduced inositol labeling of proteins suggested impaired GPI anchor synthesis in its8-1 mutants. The temperature upshift induced a further decrease in inositol labeling and caused dramatic increases in the frequency of septation in its8-1 mutants. BE49385A, an inhibitor of MCD4 and Pig-n, also increased the septation index of the wild-type cell. Os...	G2P01166 - CERT1 - 604677.0 - HGNC:2205 - CERT; COL4A3BP; GPBP; STARD11 - CERT1-related intellectual disability - 616351 - nan - monoallelic_autosomal - restricted mutation set - definitive - altered gene product structure - nan - gain of function - inferred	0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 3.879271032713091e-05
• num_train_epochs: 2
• warmup_ratio: 0.1

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 3.879271032713091e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 2
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	anno_test_average_precision
-1	-1	-	0.6910
0.2254	500	0.0908	-
0.4509	1000	0.0266	-
0.6763	1500	0.0207	-
0.9017	2000	0.0126	-
1.1271	2500	0.0082	-
1.3526	3000	0.0098	-
1.5780	3500	0.0072	-
1.8034	4000	0.0098	-
-1	-1	-	0.3290

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.7.1+cu126
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}