Raziel1234/PTT-1-110M

PTT-110M: Advanced Text Transformer

📌 Overview

PTT-110M(Pre-Trained Trasnfromer-110M) is a modern, transformer-based language model with 110 million parameters. It is trained on a high-quality dataset with long sequences of 100,000 tokens per line, capturing deep and complex language patterns.

This model is designed for:

• Context-aware text completion and continuation
• Understanding and generating structured and unstructured text
• Rapid experimentation with advanced transformer capabilities
• Maintaining coherence over long sequences

PTT-110M leverages a 2048-token context window, making it capable of handling intricate text structures and nuanced language dependencies.

---

⚙️ Model Details

Feature	Specification
Model Type	Transformer (Decoder-only)
Parameters	110M
Context Window	2048 tokens per sequence
Dataset	5B tokens of high-quality text
Sequence Length in Dataset	100,000 tokens per line
License	MIT

---

📂 Dataset & Preprocessing

• Sources: Wikipedia, forums, articles, community discussions
• Cleaning: Non-text elements removed; text curated for high quality

---

🚀 Training

• Optimized for learning complex language patterns from large, structured sequences
• Focused on generalization over memorization
• Designed for scalable experimentation and downstream fine-tuning

---

PTT Url endpoint

• this url: https://rnevo2016--ptt-chat-app-chat-endpoint.modal.run is the endpoint of the model deploy. is better easy to infrence or fine tuning.

🛠 Usage

import requests

endpoint_url = "https://rnevo2016--ptt-chat-app-chat-endpoint.modal.run"

print("PTT Chat Interactive! type 'exit' to quit.\n")

DEFAULT_MAX_LENGTH = 200 # or 2048 if you want
DEFAULT_TEMPERATURE = 1.0
DEFAULT_TOP_P = 0.95

while True:
    prompt = input("You: ")

    if prompt.lower() in ["exit", "quit"]:
        print("Session ended.")
        break

    payload = {
        "prompt": prompt,
        "max_length": DEFAULT_MAX_LENGTH,
        "temperature": DEFAULT_TEMPERATURE,
        "top_p": DEFAULT_TOP_P
    }

    try:
        resp = requests.post(endpoint_url, json=payload)
        if resp.status_code == 200:
            data = resp.json()
            print("PTT:", data.get("response", "[No response]"))
        else:
            print("Error:", resp.status_code, resp.text)
    except Exception as e:
        print("Connection error:", str(e))