README.md

---
license: mit
datasets:
- gustavokuklinski/aeon
language:
- en
base_model:
- gustavokuklinski/aeon-360m
tags:
- llama.cpp
---

![alt text](https://raw.githubusercontent.com/gustavokuklinski/aeon.ai/refs/heads/main/docs/assets/img/aeon-logo.png)

# AEON

AEON is portable, private, and capable of operating fully offline. It democratizes access to powerful, dynamic AI capabilities for a wider audience, regardless of their hardware.

The finetuned model was build to be like a "friend" for RAG personal files and work with insights.

#### Docs
- **Page** [aeon.ai](https://gustavokuklinski.github.io/aeon.ai)
- **Github Project:** [AEON.ai](https://github.com/gustavokuklinski/aeon.ai/)
- **Github LLM Finetune Scripts:** [AEON.llm](https://github.com/gustavokuklinski/aeon.llm/)

# Using Aeon

AEON uses Python with virtual environment and `git lfs` installed. 

```shell
/$ git lfs install

# With plugins
/$ git clone --recurse-submodules https://github.com/gustavokuklinski/aeon.ai.git

# Without plugins
/$ git clone https://github.com/gustavokuklinski/aeon.ai.git
```

```shell
# Create .venv
/$ python -m venv .venv

# Start virtual env
/$ source .venv/bin/activate

# Run check and install dependencies
/$ python3 scripts/install.py 

# Start AEON
/$ python3 aeon.py
```

### Using Docker

```bash
docker build -t aeon .

docker run -it --rm -p 7860:7860 -v "$(pwd):/app" aeon
```

### Tested on

| OS | CPU | GPU | RAM |
|:---|:---|:---|:---|
| Ubuntu 24.04.2 LTS | Intel i7-10510U | Intel CometLake-U GT2 | 16GB |
| Windows 11 Home Edition | Intel i7-10510U | Intel CometLake-U GT2 | 8GB |


## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)

```bash
brew install llama.cpp

```
Invoke the llama.cpp server or the CLI.

### CLI:
```bash
llama-cli --hf-repo gustavokuklinski/aeon-GGUF --hf-file aeon-360M.Q8_0.gguf -p "What is a virtual species?"
```

### Server:
```bash
llama-server --hf-repo gustavokuklinski/aeon-GGUF --hf-file aeon-360M.Q8_0.gguf -c 2048
```

Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```

Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```

Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo gustavokuklinski/aeon-GGUF --hf-file aeon-360M.Q8_0.gguf -p "What is a virtual species?"
```
or 
```
./llama-server --hf-repo gustavokuklinski/aeon-GGUF --hf-file aeon-360M.Q8_0.gguf -c 2048
```