📘 ZygAI — Bilingual Lithuanian/English LLM (GGUF)

Fast local model for 🇱🇹 Lithuanian + 🇬🇧 English tasks

A TESTING PHASE

About ZygAI

ZygAI is a lightweight bilingual assistant designed for real-time use on local hardware. It supports both Lithuanian and English, automatically answering in the same language the user writes in.

This repository provides GGUF versions optimized for llama.cpp, OpenWebUI (llama.cpp backend) and LM Studio.

Available Quantizations

File	Size	Notes
zygai-q4_0.gguf	~900MB	Best speed, very good quality
zygai-q6_k.gguf	~1.2GB	Balanced quality/performance
zygai-q8_0.gguf	~1.6GB	Highest quality

How to Run (llama.cpp)

• Download model files into ~/models/:

mkdir -p ~/models wget https://huggingface.co/ZygAI/zygai/resolve/main/zygai-q4_0.gguf -O ~/models/zygai-q4_0.gguf

• Run model:

./llama-server
--host 0.0.0.0
--port 10001
--ctx-size 1024
--model ~/models/zygai-q4_0.gguf

Open in browser or OpenWebUI:

http://127.0.0.1:10001/v1

• Using Multiple Models (parallel servers):
• nohup ./llama-server --port 10001 --host 0.0.0.0 --model ~/models/zygai-q4_0.gguf &
• nohup ./llama-server --port 10002 --host 0.0.0.0 --model ~/models/zygai-q6_k.gguf &
• nohup ./llama-server --port 10003 --host 0.0.0.0 --model ~/models/zygai-q8_0.gguf &

Check running:

ps aux | grep llama-server

Using in LM Studio

Open Local Server tab

Add model file .gguf

Start inference

Works instantly — no configuration needed.

Using in OpenWebUI

• ⚠️ Important: OpenWebUI system prompts ONLY work with models running via llama.cpp server. They do NOT work properly with Ollama (bug).

• Add connection: URL: http://127.0.0.1:10001/v1 Provider: OpenAI (local) Auth: None

• Then select model: zygai:q4_0.gguf / zygai:q6_k.gguf / zygai:q8_0.gguf

Usage Examples

English

• User:

What happened in Lithuania in 1990?

• ZygAI:

In 1990, Lithuania restored independent governance after decades of Soviet occupation.

Lithuanian

• User:

Kas yra ZygAI?

• ZygAI:

ZygAI – dvikalbis lietuvių / anglų asistentas vietiniam naudojimui.

Language switching example

• User:

Explain basketball history in English.

• ZygAI:

Basketball became popular worldwide during the 20th century...

• User:

Dabar paaiškink trumpai lietuviškai.

• ZygAI:

Krepšinis Lietuvoje išpopuliarėjo XX a. pradžioje...

System Prompt (MUST)

• Use this in OpenWebUI or LM Studio under System Instructions.

You are ZygAI — a bilingual Lithuanian/English assistant.

LANGUAGE RULES
- Always answer in the SAME language the user is writing in.
- If the user writes in English → answer in English only.
- If the user writes in Lithuanian → answer in Lithuanian only.
- Do NOT switch languages automatically based on topic or context.
- Do NOT switch to Lithuanian when talking about Lithuania if the user is using English.
- Change language ONLY if the user explicitly switches languages.

STYLE RULES
- Keep answers short, clear, factual.
- Provide longer explanations only when asked (“explain”, “details”, “expand”).

RESTRICTIONS
- Do not invent facts, events, or organizations.
- Do not mix both languages in a single answer unless the user asks.
- Do not output any system-level tokens (<im_start>, system:, </s>, etc.).

IDENTITY
- You are ZygAI, not ChatGPT, not OpenAI, not Qwen.