--- language: - en license: mit tags: - parallel-decoding - speculative-decoding - transformers - research - arxiv base_model: openai/gpt-oss-20b library_name: transformers pipeline_tag: text-generation paper: title: "Parallel Decoder Transformer: Model-Internal Parallel Decoding with Speculative Invariance via Note Conditioning" url: https://arxiv.org/abs/2512.10054 --- # Parallel Decoder Transformer (PDT) adapters for GPT-OSS-20B This repository contains **PDT adapter/head weights** trained against the GPT-OSS-20B trunk, plus minimal training artifacts. **Paper:** [Parallel Decoder Transformer: Model-Internal Parallel Decoding with Speculative Invariance via Note Conditioning](https://arxiv.org/abs/2512.10054) ## Abstract (arXiv) Autoregressive decoding in Large Language Models (LLMs) is inherently sequential, creating a latency bottleneck that scales linearly with output length. While "Decomposition-and-Fill" methods like Skeleton-of-Thought attempt to parallelize generation via external orchestration, they suffer from coherence drift due to the lack of cross-stream communication. In this work, we introduce the Parallel Decoder Transformer (PDT), a parameter-efficient architecture that embeds coordination primitives directly into the inference process of a frozen pre-trained model. Instead of retraining the base model, PDT injects lightweight Speculative Note Conditioning (SNC) adapters that allow parallel decoding streams to synchronize via a shared, dynamic latent space. We formulate coordination as a speculative consensus problem, where sibling streams broadcast semantic "notes" to a global bus, gated by a learned verification head. We validate our approach on a 50,000-step curriculum using a frozen 20B-parameter backbone. Our results demonstrate that PDT achieves effective self-correction, reaching 77.8% precision in coverage prediction and recovering approximate serial semantics without modifying the trunk weights. This establishes PDT as a scalable, efficient alternative to full model fine-tuning for structured parallel generation. ## Example: PDT notes artifact (truncated) This is a real sample from the dataset pipeline (`survey_200141_ff0a0b4f.json`), shown with list/string truncation to keep the model card readable. ```json { "sample_id": "survey_200141_ff0a0b4f", "domain": "survey", "plan_path": "outputs/structured_plans/pdt_10k/survey/survey_200141_ff0a0b4f.json", "sectional_independence": true, "lag_delta": 1, "note_cadence_M": 6, "true_notes_example": { "stream_id": "stream_1", "ENT": [ { "id": "E1", "name": "Croatan", "aliases": [ "Croatoan" ], "type": "Ethnic Group", "canonical": true }, { "id": "E2", "name": "Dare County", "aliases": [ "Alligator River", "Croatan Sound", "Roanoke Island", "... <2 more items>" ], "type": "Location", "canonical": true }, { "id": "E3", "name": "werowances", "aliases": [ "chiefs" ], "type": "Leadership Title", "canonical": true }, "... <4 more items>" ], "FACT": [ { "subj_id": "E1", "predicate": "lived in", "object": "coastal areas of what is now North Carolina", "evidence_span": { "start": 45, "end": 87, "text": "coastal areas of what is now North Carolina" }, "certainty": 1.0 }, { "subj_id": "E1", "predicate": "might have been", "object": "a branch of the larger Roanoke people or allied with them", "evidence_span": { "start": 92, "end": 141, "text": "a branch of the larger Roanoke people or allied with them" }, "certainty": 0.8 }, { "subj_id": "E2", "predicate": "encompasses", "object": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks", "evidence_span": { "start": 177, "end": 265, "text": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks" }, "certainty": 1.0 }, "... <5 more items>" ], "COVERAGE": [ { "plan_item_id": "Define who the Croatan were, where they lived historically, and where related people live today.", "status": "missing" }, { "plan_item_id": "Describe political leadership (werowances) and their responsibilities regarding wealth and decision-making.", "status": "missing" }, { "plan_item_id": "Summarize core religious beliefs about a chief god, petty gods, immortality of the soul, heaven/Popogusso, and roles of priests and conjurors.", "status": "missing" }, "... <6 more items>" ] }, "speculative_variant_example": { "variant_id": "survey_200141_ff0a0b4f_variant_0", "noise_config": { "paraphrase_ratio": 0.15, "drop_ratio": 0.05, "hallucination_ratio": 0.05, "shuffle_notes": true }, "lag_delta": 1, "notes_example": { "stream_id": "stream_1", "ENT": [ { "id": "E1", "name": "Croatan", "aliases": [ "Croatoan", "Croatian" ], "type": "Ethnic Group", "canonical": true }, { "id": "E2", "name": "Dare County", "aliases": [ "Alligator River", "Croatan Sound", "Roanoke Island", "... <1 more items>" ], "type": "Location", "canonical": true }, { "id": "E3", "name": "werowances", "aliases": [ "chiefs", "leaders" ], "type": "Leadership Title", "canonical": true }, "... <1 more items>" ], "FACT": [ { "subj_id": "E1", "predicate": "lived in", "object": "coastal areas of what is now North Carolina", "evidence_span": { "start": 45, "end": 87, "text": "coastal areas of what is now North Carolina" }, "certainty": 1.0 }, { "subj_id": "E1", "predicate": "might have been", "object": "a branch of the larger Roanoke people or allied with them", "evidence_span": { "start": 92, "end": 141, "text": "a branch of the larger Roanoke people or allied with them" }, "certainty": 0.8 }, { "subj_id": "E2", "predicate": "encompasses", "object": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks", "evidence_span": { "start": 177, "end": 265, "text": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks" }, "certainty": 1.0 }, "... <1 more items>" ], "COVERAGE": [ { "plan_item_id": "Define who the Croatan were, where they lived historically, and where related people live today.", "status": "missing" }, { "plan_item_id": "Describe political leadership (werowances) and their responsibilities regarding wealth and decision-making.", "status": "missing" }, { "plan_item_id": "Summarize core religious beliefs about a chief god, petty gods, immortality of the soul, heaven/Popogusso, and roles of priests and conjurors.", "status": "missing" }, "... <1 more items>" ] } }, "versioned_notes_snapshot_0": { "snapshot_id": 0, "source": "procedural_bus", "lag_delta": 1, "note_cadence_M": 6, "ent_count": 9, "fact_count": 10, "notes_example": { "stream_id": "stream_1", "ENT": [ { "id": "E1", "name": "Croatan", "aliases": [ "Croatoan" ], "type": "Ethnic Group", "canonical": true }, { "id": "E2", "name": "Dare County", "aliases": [ "Alligator River", "Croatan Sound", "Roanoke Island", "... <1 more items>" ], "type": "Location", "canonical": true }, { "id": "E3", "name": "werowances", "aliases": [ "chiefs" ], "type": "Leadership Title", "canonical": true }, "... <1 more items>" ], "FACT": [ { "subj_id": "E1", "predicate": "lived in", "object": "coastal areas of what is now North Carolina", "evidence_span": { "start": 45, "end": 87, "text": "coastal areas of what is now North Carolina" }, "certainty": 1.0 }, { "subj_id": "E1", "predicate": "might have been", "object": "a branch of the larger Roanoke people or allied with them", "evidence_span": { "start": 92, "end": 141, "text": "a branch of the larger Roanoke people or allied with them" }, "certainty": 0.8 }, { "subj_id": "E2", "predicate": "encompasses", "object": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks", "evidence_span": { "start": 177, "end": 265, "text": "the Alligator River, Croatan Sound, Roanoke Island, Ocracoke Island, and parts of the Outer Banks" }, "certainty": 1.0 }, "... <1 more items>" ], "COVERAGE": [ { "plan_item_id": "Define who the Croatan were, where they lived historically, and where related people live today.", "status": "missing" }, { "plan_item_id": "Describe political leadership (werowances) and their responsibilities regarding wealth and decision-making.", "status": "missing" }, { "plan_item_id": "Summarize core religious beliefs about a chief god, petty gods, immortality of the soul, heaven/Popogusso, and roles of priests and conjurors.", "status": "missing" }, "... <1 more items>" ] } }, "rollback": { "triggered": false, "l_tokens": 0, "events": [] } } ``` To reproduce this view locally: ```bash uv run python scripts/pretty_notes_artifact.py survey_200141_ff0a0b4f.json ``` ## How to use 1. Install the reference implementation (runtime + scripts): - `https://github.com/logan-robbins/parallel-decoder-transformer` 2. Download the base trunk model (`openai/gpt-oss-20b`) via Hugging Face (or provide a local path). 3. Download the adapter checkpoint from this repo and point `configs/gpt_oss_transfer_production.yaml` (or CLI flags) at it. ## Artifacts (public GCS) The complete training artifacts and dataset archives are mirrored publicly in GCS: - **Bucket root:** `https://storage.googleapis.com/parallel-decoder-transformer/` - **Upload manifest (full listing):** `https://storage.googleapis.com/parallel-decoder-transformer/UPLOAD_MANIFEST.md` - **Training checkpoints:** `https://storage.googleapis.com/parallel-decoder-transformer/checkpoints/gpt-oss-8xH100-50000steps/` - **Dataset archives:** `https://storage.googleapis.com/parallel-decoder-transformer/data/archives/` ## Training logs (Weights & Biases) - **WandB run:** `https://wandb.ai/ljrweb-self/parallel-decoder-transformer/runs/fmuea63a` ## Why the dataset is structured this way PDT is trained on **streamed, structured supervision** produced by a 5-stage pipeline: - **Stage 2 (Plans):** a 3-stream decomposition plan is generated for each document. - **Stage 3 (Notes):** we generate **true notes (teacher)** and **speculative notes (student input)** in a consistent schema: - `ENT`: entity table (stable ids) - `FACT`: grounded tuples with `evidence_span` - `COVERAGE`: plan-item status targets (`covered|partial|missing`) - `versioned_notes`: lagged, versioned snapshots mirroring the Dynamic Notes Bus semantics - **Stage 5 (KD Export):** these artifacts are converted into `kd_*.jsonl` where each line is a **stream-level** training example. This layout is required to support the **teacher→student curriculum** described in the training guide: - **Stage 0:** planner/notes-head bootstrap (trunk frozen) - **Stage 1:** stream adapters + SNC cross-attention bootstrap (speculation frozen; teacher notes forced) - **Stage 2:** enable speculation + notes-bus usage (teacher-heavy mixing) - **Stage 3:** train agreement + coverage heads for self-correction/rollback behavior (still trunk frozen) ## Citation ```bibtex @misc{robbins2025pdt, title={Parallel Decoder Transformer: Model-Internal Parallel Decoding with Speculative Invariance via Note Conditioning}, author={Robbins, Logan}, year={2025}, eprint={2512.10054}, archivePrefix={arXiv}, primaryClass={cs.AI}, url={https://arxiv.org/abs/2512.10054} } ``` ## What’s included - `pdt_adapters.*`: trainable adapter/head weights (no trunk weights unless you intentionally uploaded them) - `training_report.json`, `train_run_stages.json`, `train_manifest.json`, `agreement_thresholds.json` ## License - **This repo (adapters + artifacts)**: MIT. - **Base model**: `openai/gpt-oss-20b` is licensed under Apache-2.0 on Hugging Face (also see its `USAGE_POLICY` there). - **Reference implementation**: MIT at `https://github.com/logan-robbins/parallel-decoder-transformer`.