inference-net/Aella-Nemotron-12B

Paper-Summarizer-Nemotron-12B

A fine-tuned Nemotron-12B model specialized for generating structured summaries of scientific research papers in standardized JSON format with superior throughput.

Model Description

This model is part of Project AELLA, developed in collaboration with LAION and Wynd Labs to democratize access to scientific knowledge by creating structured summaries of research papers at scale.

Base Model: NVIDIA Nemotron 12B (Hybrid Mamba-Transformer) Training Data: 110,000 curated research papers Performance: Achieves 71.3% accuracy on QA evaluation Throughput: 2.25× faster than Qwen3-14B variant

This generates comprehensive structured summaries in a JSON format. The papers are either classified as SCIENTIFIC_TEXT, PARTIAL_SCIENTIFIC_TEXT, or NON_SCIENTIFIC_TEXT. The fields extracted are key research elements such as methodology, results, claims, and limitations.

The model supports papers up to 131K tokens and is optimized for large-scale batch processing with high throughput (0.97 requests/sec).

Usage

Serving the Model

Note: This model requires a custom chat template for proper reasoning token handling.

vllm serve inference-net/Paper-Summarizer-Nemotron-12B \
  --port 8000 \
  --host 0.0.0.0 \
  --trust-remote-code \
  --data-parallel-size 1 \
  --tensor-parallel-size 1 \
  --max-num-seqs 32 \
  --max-model-len 131072 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching \
  --enable-chunked-prefill \
  --chat-template "{%- set ns = namespace(enable_thinking=true) %}{%- for message in messages -%}{%- set content = message['content'] -%}{%- if message['role'] == 'user' or message['role'] == 'system' -%}{%- if '/think' in content -%}{%- set ns.enable_thinking = true -%}{%- elif '/no_think' in content -%}{%- set ns.enable_thinking = false -%}{%- endif -%}{%- endif -%}{%- endfor -%}{%- if messages[0]['role'] != 'system' -%}{%- set ns.non_tool_system_content = '' -%}{{- '<SPECIAL_10>System\n' -}}{%- else -%}{%- set ns.non_tool_system_content = messages[0]['content'].replace('/think', '').replace('/no_think', '').strip() -%}{{- '<SPECIAL_10>System\n' + ns.non_tool_system_content }}{%- endif -%}{%- if tools -%}{%- if ns.non_tool_system_content is defined and ns.non_tool_system_content != '' -%}{{- '\n\n' -}}{%- endif -%}{{- 'You can use the following tools to assist the user if required:' -}}{{- '\n<AVAILABLE_TOOLS>[' -}}{%- for tool in tools -%}{{- (tool.function if tool.function is defined else tool) | tojson -}}{{- ', ' if not loop.last else '' -}}{%- endfor -%}{{- ']</AVAILABLE_TOOLS>\n\n' -}}{{- 'If you decide to call any tool(s), use the following format:\n' -}}{{- '<TOOLCALL>[{{\"name\": \"tool_name1\", \"arguments\": \"tool_args1\"}}, ' -}}{{- '{{\"name\": \"tool_name2\", \"arguments\": \"tool_args2\"}}]</TOOLCALL>\n\n' -}}{{- 'The user will execute tool-calls and return responses from tool(s) in this format:\n' -}}{{- '<TOOL_RESPONSE>[{{\"tool_response1\"}}, {{\"tool_response2\"}}]</TOOL_RESPONSE>\n\n' -}}{{- 'Based on the tool responses, you can call additional tools if needed, correct tool calls if any errors are found, or just respond to the user.' -}}{%- endif -%}{{- '\n' -}}{%- set messages = messages[1:] if messages[0]['role'] == 'system' else messages -%}{%- if messages[-1]['role'] == 'assistant' -%}{%- set ns.last_turn_assistant_content = messages[-1]['content'].strip() -%}{%- set messages = messages[:-1] -%}{%- endif -%}{%- for message in messages %}{%- set content = message['content'] %}{%- if message['role'] == 'user' -%}{{- '<SPECIAL_11>User\n' + content.replace('/think', '').replace('/no_think', '').strip() + '\n' }}{%- elif message['role'] == 'tool' -%}{%- if loop.first or (messages[loop.index0 - 1].role != 'tool') -%}{{- '<SPECIAL_11>User\n' + '<TOOL_RESPONSE>[' }}{%- endif -%}{{- message['content'] -}}{{- ', ' if not loop.last and (messages[loop.index0 + 1].role == 'tool') else '' -}}{%- if loop.last or (messages[loop.index0 + 1].role != 'tool') -%}{{- ']</TOOL_RESPONSE>\n' -}}{%- endif -%}{%- elif message['role'] == 'assistant' -%}{%- if '</think>' in content -%}{%- set content = content.split('</think>')[1].strip() %}{%- endif -%}{{- '<SPECIAL_11>Assistant\n' + content.strip() }}{%- if message.tool_calls -%}{%- if content.strip() != '' -%}{{- '\n\n' -}}{%- endif -%}{{- '<TOOLCALL>[' -}}{%- for call in message.tool_calls -%}{%- set fn = call.function if call.function is defined else call -%}{{- '{\"name\": \"' + fn.name + '\", \"arguments\": ' -}}{%- if fn.arguments is string -%}{{- fn.arguments -}}{%- else -%}{{- fn.arguments | tojson -}}{%- endif -%}{{- '}' + (', ' if not loop.last else '') -}}{%- endfor -%}{{- ']</TOOLCALL>' -}}{%- endif -%}{{- '\n<SPECIAL_12>\n' -}}{%- endif -%}{%- endfor -%}{%- if add_generation_prompt -%}{{- '<SPECIAL_11>Assistant\n' -}}{%- if ns.enable_thinking is defined and ns.enable_thinking is false -%}{{- '<think></think>' -}}{%- else -%}{{- '<think>\n' -}}{%- endif -%}{%- if ns.last_turn_assistant_content is defined and ns.last_turn_assistant_content != '' -%}{{- ns.last_turn_assistant_content -}}{%- endif -%}{%- else -%}{%- if ns.last_turn_assistant_content is defined and ns.last_turn_assistant_content != '' -%}{{- '<SPECIAL_11>Assistant\n' -}}{%- if ns.enable_thinking is defined and ns.enable_thinking is false -%}{{- '<think></think>' -}}{%- else -%}{{- '<think>\n' -}}{%- endif -%}{{- ns.last_turn_assistant_content -}}{%- if continue_final_message is defined -%}{%- if continue_final_message is false -%}{{- '\n<SPECIAL_12>\n' -}}{%- endif -%}{%- else -%}{{- '\n<SPECIAL_12>\n' -}}{%- endif -%}{%- endif -%}{%- endif -%}"

Making Requests

import requests

# System prompt (required)
system_prompt = """[Insert the full system prompt from the prompt.txt file -
see the full prompt in the model repository]"""

# User prompt: the paper text to summarize
paper_text = """
Title: Your Paper Title
Authors: Author 1, Author 2
Abstract: ...
[Full paper content]
"""

# API request
response = requests.post(
    "http://localhost:8000/v1/chat/completions",
    json={
        "model": "inference-net/Paper-Summarizer-Nemotron-12B",
        "messages": [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": paper_text}
        ],
        "temperature": 0.2
    },
    timeout=600
)

result = response.json()
# Note: Response may include reasoning tokens wrapped in <think></think>
# These are automatically stripped by the chat template
summary = result["choices"][0]["message"]["content"]
print(summary)

System Prompt

The model requires the same system prompt as the Qwen3-14B variant. The prompt instructs the model to:

1. Classify the text as SCIENTIFIC_TEXT, PARTIAL_SCIENTIFIC_TEXT, or NON_SCIENTIFIC_TEXT
2. Extract structured information including:
   • Title, authors, publication year
   • Research context and hypotheses
   • Methodological details
   • Key results with quantitative data
   • Claims with supporting evidence
   • Limitations and ethical considerations

The full system prompt is available in the model repository's prompt.txt file.

Output Format

The model outputs a single valid JSON object with this structure:

{
  "article_classification": "SCIENTIFIC_TEXT",
  "reason": null,
  "summary": {
    "title": "",
    "authors": "",
    "publication_year": null,
    "field_subfield": "",
    "executive_summary": "",
    "research_context": "",
    "methodological_details": "",
    "key_results": "",
    "claims": [...],
    "contradictions_and_limitations": "",
    ...
  }
}

Performance

LLM-as-a-Judge Evaluation

• Score: 4.095/5.0
• Comparison: Slightly behind Qwen3-14B (4.207) but still high quality

QA Dataset Evaluation

• Accuracy: 71.3%
• Comparison: Strong performance, suitable for batch processing

Throughput (8×H200 node)

• Requests/sec: 0.97 (2.25× faster than Qwen3-14B)
• Input Tokens/sec: 16,943.69
• Output Tokens/sec: 4,880.76
• Single Request Tokens/sec: 76.17

Cost Efficiency

• Processing 100M papers: ~$45,000 (vs $100,000 for Qwen3-14B, $5M+ for GPT-5)
• Ideal for: Large-scale batch processing where throughput matters

Training Details

• Training Set: 100,000 papers (same as Qwen3-14B)
• Validation Set: 10,000 papers
• Average Paper Length: 81,334 characters
• Architecture: Hybrid Mamba-Transformer for high throughput
• Training Approach: Post-training on summaries generated by frontier models

When to Use This Model

Choose Nemotron-12B if:

• Processing large batches (100K+ papers)
• Throughput and cost are primary concerns
• Accuracy in the 70-75% range is acceptable
• Running on GPU infrastructure with parallel processing

Choose Qwen3-14B if:

• Need highest possible accuracy (73.9% vs 71.3%)
• Processing smaller batches or single papers
• Quality is more important than speed

Limitations

• May generate subtle factual errors (hallucinations) for fine-grained details
• Context limit (131K tokens) may truncate extremely long documents
• Unified schema may not capture all domain-specific nuances
• Summaries are research aids, not replacements for primary sources in high-stakes scenarios
• Slightly lower accuracy than Qwen3-14B variant

Related Resources

• Paper Visualization Website: https://laion.inference.net
• Visualization Repository: https://github.com/context-labs/laion-data-explorer
• Alexandria Paper: https://arxiv.org/abs/2502.19413
• Qwen3-14B Variant: inference-net/Paper-Summarizer-Qwen3-14B

License

[License information to be added]

Acknowledgments

This work was made possible through collaboration with:

• LAION
• Wynd Labs
• Inference.net
• NVIDIA (base Nemotron architecture)
• Contributors to bethgelab, PeS2o, Common Pile, and OpenAlex