Spaces:
Build error
Build error
repo config
Browse files
README.md
CHANGED
|
@@ -1,276 +1,9 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
<div align="center">
|
| 12 |
-
<a href="https://discord.gg/e4AYNnFg2F">
|
| 13 |
-
<img src="https://img.shields.io/badge/discord-join%20chat-blue.svg?style=for-the-badge" alt="Join our Discord" height="20">
|
| 14 |
-
</a>
|
| 15 |
-
<a href="https://twitter.com/agishaun">
|
| 16 |
-
<img alt="Twitter Follow" src="https://img.shields.io/twitter/follow/agishaun?style=for-the-badge" height="20">
|
| 17 |
-
<a href="https://github.com/Shaunwei/RealChar">
|
| 18 |
-
<img alt="GitHub" src="https://img.shields.io/github/stars/Shaunwei/RealChar?style=for-the-badge&color=gold" height="20">
|
| 19 |
-
</a>
|
| 20 |
-
<a href="https://github.com/Shaunwei/RealChar/commits/main">
|
| 21 |
-
<img alt="GitHub" src="https://img.shields.io/github/last-commit/Shaunwei/RealChar/main?style=for-the-badge" height="20">
|
| 22 |
-
</a>
|
| 23 |
-
<a href="https://github.com/Shaunwei/RealChar/blob/main/README.md" target="_blank">
|
| 24 |
-
<img src="https://img.shields.io/static/v1?label=license&message=MIT&color=green&style=for-the-badge" alt="License" height="20">
|
| 25 |
-
</a>
|
| 26 |
-
<a href="https://hub.docker.com/repository/docker/shaunly/real_char/general" target="_blank">
|
| 27 |
-
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/shaunly/real_char?style=for-the-badge" height="20">
|
| 28 |
-
</a>
|
| 29 |
-
</div>
|
| 30 |
-
|
| 31 |
-
## β¨ Demo
|
| 32 |
-
Try our site at [RealChar.ai](https://realchar.ai/)
|
| 33 |
-
|
| 34 |
-
(We are also beta-testing our iOS mobile appπ±! Sign up [here](https://testflight.apple.com/join/JA6p9sZQ))
|
| 35 |
-
|
| 36 |
-
### Demo 1 - with AI Elon about cage fight!
|
| 37 |
-
|
| 38 |
-
https://github.com/Shaunwei/RealChar/assets/5101573/5de0b023-6cf3-4947-84cb-596f429d109e
|
| 39 |
-
|
| 40 |
-
### Demo 2 - with AI Raiden about AI and "real" memory
|
| 41 |
-
|
| 42 |
-
https://github.com/Shaunwei/RealChar/assets/5101573/62a1f3d1-1166-4254-9119-97647be52c42
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
__Demo settings: Web, GPT4, ElevenLabs with voice clone, Chroma, Google Speech to Text__
|
| 47 |
-
|
| 48 |
-
## π― Key Features
|
| 49 |
-
- **Easy to use**: No coding required to create your own AI character.
|
| 50 |
-
- **Customizable**: You can customize your AI character's personality, background, and even voice
|
| 51 |
-
- **Realtime**: Talk to or message your AI character in realtime
|
| 52 |
-
- **Multi-Platform**: You can talk to your AI character on web, terminal and mobile(Yes. we open source our mobile app)
|
| 53 |
-
- **Most up-to-date AI**: We use the most up-to-date AI technology to power your AI character, including OpenAI, Anthropic Claude 2, Chroma, Whisper, ElevenLabs, etc.
|
| 54 |
-
- **Modular**: You can easily swap out different modules to customize your flow. Less opinionated, more flexible. Great project to start your AI Engineering journey.
|
| 55 |
-
|
| 56 |
-
## π¬ Tech stack
|
| 57 |
-
<div align="center">
|
| 58 |
-
<img src="https://storage.googleapis.com/assistly/static/realchar/techstack.png" alt="RealChar-tech-stack" width="100%" style="padding: 20px"/>
|
| 59 |
-
</div>
|
| 60 |
-
|
| 61 |
-
- β
**Web**: [Vanilla JS](http://vanilla-js.com/), [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)
|
| 62 |
-
- β
**Mobile**: [Swift](https://developer.apple.com/swift/), [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)
|
| 63 |
-
- β
**Backend**: [FastAPI](https://fastapi.tiangolo.com/), [SQLite](https://www.sqlite.org/index.html), [Docker](https://www.docker.com/)
|
| 64 |
-
- β
**Data Ingestion**: [LlamaIndex](https://www.llamaindex.ai/), [Chroma](https://www.trychroma.com/)
|
| 65 |
-
- β
**LLM Orchestration**: [LangChain](https://langchain.com/), [Chroma](https://www.trychroma.com/)
|
| 66 |
-
- β
**LLM**: [OpenAI GPT3.5/4](https://platform.openai.com/docs/api-reference/chat), [Anthropic Claude 2](https://docs.anthropic.com/claude/docs/getting-started-with-claude)
|
| 67 |
-
- β
**Speech to Text**: [Local Whisper](https://github.com/openai/whisper), [OpenAI Whisper API](https://platform.openai.com/docs/api-reference/audio), [Google Speech to Text](https://cloud.google.com/speech-to-text/docs#docs)
|
| 68 |
-
- β
**Text to Speech**: [ElevenLabs](https://beta.elevenlabs.io/)
|
| 69 |
-
- β
**Voice Clone**: [ElevenLabs](https://beta.elevenlabs.io/voice-lab)
|
| 70 |
-
|
| 71 |
-
## π Comparison with existing products
|
| 72 |
-
<div align="center">
|
| 73 |
-
<img src="https://storage.googleapis.com/assistly/static/realchar/compare.png">
|
| 74 |
-
</div>
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
## π¨βπ Prerequisites
|
| 78 |
-
|
| 79 |
-
Before you begin setting up this project, please ensure you have completed the following tasks:
|
| 80 |
-
|
| 81 |
-
### 0. Setup Tutorial
|
| 82 |
-
|
| 83 |
-
- [Tutorial - YouTuBe](https://www.youtube.com/watch?v=Q16ZH3kJWxw)
|
| 84 |
-
|
| 85 |
-
### 1. LLM - OpenAI API Token
|
| 86 |
-
<details><summary>πclick me</summary>
|
| 87 |
-
This application utilizes the OpenAI API to access its powerful language model capabilities. In order to use the OpenAI API, you will need to obtain an API token.
|
| 88 |
-
|
| 89 |
-
To get your OpenAI API token, follow these steps:
|
| 90 |
-
|
| 91 |
-
1. Go to the [OpenAI website](https://beta.openai.com/signup/) and sign up for an account if you haven't already.
|
| 92 |
-
2. Once you're logged in, navigate to the [API keys page](https://beta.openai.com/account/api-keys).
|
| 93 |
-
3. Generate a new API key by clicking on the "Create API Key" button.
|
| 94 |
-
4. Copy the API key and store it safely.
|
| 95 |
-
5. Add the API key to your environment variable, e.g. `export OPENAI_API_KEY=<your API key>`
|
| 96 |
-
|
| 97 |
-
(Optional) To use Azure OpenAI API instead, refer to the following section:
|
| 98 |
-
|
| 99 |
-
1. Set API type
|
| 100 |
-
`export OPENAI_API_TYPE=azure`
|
| 101 |
-
|
| 102 |
-
If you want to use the earlier version `2023-03-15-preview`:
|
| 103 |
-
|
| 104 |
-
`export OPENAI_API_VERSION=2023-03-15-preview`
|
| 105 |
-
|
| 106 |
-
2. To set the base URL for your Azure OpenAI resource.
|
| 107 |
-
You can find this in the Azure portal under your Azure OpenAI resource.
|
| 108 |
-
|
| 109 |
-
`export OPENAI_API_BASE=https://your-base-url.openai.azure.com`
|
| 110 |
-
|
| 111 |
-
3. To set the OpenAI model deployment name for your Azure OpenAI resource.
|
| 112 |
-
|
| 113 |
-
`export OPENAI_API_MODEL_DEPLOYMENT_NAME=gpt-35-turbo-16k`
|
| 114 |
-
|
| 115 |
-
4. To set the OpenAIEmbeddings model deployment name for your Azure OpenAI resource.
|
| 116 |
-
|
| 117 |
-
`export OPENAI_API_EMBEDDING_DEPLOYMENT_NAME=text-embedding-ada-002`
|
| 118 |
-
|
| 119 |
-
</details>
|
| 120 |
-
|
| 121 |
-
### 1.1 (Optional) Prepare LLM - Anthropic(Claude 2) API Token
|
| 122 |
-
<details><summary>πclick me</summary>
|
| 123 |
-
|
| 124 |
-
To get your Anthropic API token, follow these steps:
|
| 125 |
-
|
| 126 |
-
1. Go to the [Anthropic website](https://docs.anthropic.com/claude/docs/getting-started-with-claude) and sign up for an account if you haven't already.
|
| 127 |
-
2. Once you're logged in, navigate to the [API keys page](https://console.anthropic.com/account/keys).
|
| 128 |
-
3. Generate a new API key by clicking on the "Create Key" button.
|
| 129 |
-
4. Copy the API key and store it safely.
|
| 130 |
-
5. Add the API key to your environment variable, e.g. `export ANTHROPIC_API_KEY=<your API key>`
|
| 131 |
-
</details>
|
| 132 |
-
|
| 133 |
-
### 2. (Optional) Prepare Speech to Text - Google Cloud API
|
| 134 |
-
<details><summary>πclick me</summary>
|
| 135 |
-
|
| 136 |
-
To get your Google Cloud API credentials.json, follow these steps:
|
| 137 |
-
|
| 138 |
-
1. Go to the [GCP website](https://cloud.google.com/speech-to-text/docs/before-you-begin) and sign up for an account if you haven't already.
|
| 139 |
-
2. Follow the guide to create a project and enable Speech to Text API
|
| 140 |
-
3. Put `google_credentials.json` in the root folder of this project. Check [GCP website](https://cloud.google.com/speech-to-text/docs/before-you-begin#set_your_authentication_environment_variable)
|
| 141 |
-
4. Change `SPEECH_TO_TEXT_USE` to use `GOOGLE` in your `.env` file
|
| 142 |
-
</details>
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
### 3. Prepare Text to Speech - ElevenLabs API Key
|
| 146 |
-
<details><summary>πclick me</summary>
|
| 147 |
-
1. Creating an ElevenLabs Account
|
| 148 |
-
Visit [ElevenLabs](https://beta.elevenlabs.io/) to create an account. You'll need this to access the text to speech and voice cloning features.
|
| 149 |
-
|
| 150 |
-
2. In your Profile Setting, you can get an API Key. Save it in a safe place.
|
| 151 |
-
|
| 152 |
-
3. Set API key in your .env file:
|
| 153 |
-
```
|
| 154 |
-
ELEVEN_LABS_API_KEY=<api key>
|
| 155 |
-
```
|
| 156 |
-
</details>
|
| 157 |
-
|
| 158 |
-
## πΏ Installation via Python
|
| 159 |
-
- **Step 1**. Clone the repo
|
| 160 |
-
```sh
|
| 161 |
-
git clone https://github.com/Shaunwei/RealChar.git && cd RealChar
|
| 162 |
-
```
|
| 163 |
-
- **Step 2**. Install requirements
|
| 164 |
-
- Install [portaudio](https://people.csail.mit.edu/hubert/pyaudio/) and [ffmpeg](https://ffmpeg.org/download.html) for audio
|
| 165 |
-
```sh
|
| 166 |
-
# for mac
|
| 167 |
-
brew install portaudio
|
| 168 |
-
brew install ffmpeg
|
| 169 |
-
```
|
| 170 |
-
```sh
|
| 171 |
-
# for ubuntu
|
| 172 |
-
sudo apt update
|
| 173 |
-
sudo apt install portaudio19-dev
|
| 174 |
-
sudo apt install ffmpeg
|
| 175 |
-
```
|
| 176 |
-
- Then install all python requirements
|
| 177 |
-
```sh
|
| 178 |
-
pip install -r requirements.txt
|
| 179 |
-
```
|
| 180 |
-
- **Step 3**. Create an empty [sqlite](https://www.sqlite.org/index.html) database if you have not done so before
|
| 181 |
-
```sh
|
| 182 |
-
sqlite3 test.db "VACUUM;"
|
| 183 |
-
```
|
| 184 |
-
- **Step 4**. Run db upgrade
|
| 185 |
-
```sh
|
| 186 |
-
alembic upgrade head
|
| 187 |
-
```
|
| 188 |
-
- **Step 5**. Setup `.env`: update API keys and select module
|
| 189 |
-
```sh
|
| 190 |
-
cp .env.example .env
|
| 191 |
-
```
|
| 192 |
-
- **Step 6**. Run server with `cli.py` or use uvicorn directly
|
| 193 |
-
```sh
|
| 194 |
-
python cli.py run-uvicorn
|
| 195 |
-
# or
|
| 196 |
-
uvicorn realtime_ai_character.main:app
|
| 197 |
-
```
|
| 198 |
-
- **Step 7**. Run client:
|
| 199 |
-
- Use **GPT4** for better conversation and **Wear headphone** for best audio(avoid echo)
|
| 200 |
-
- There are two ways to access the web client:
|
| 201 |
-
- **Option 1**: Open your web browser and navigate to http://localhost:8000 (NOT 0.0.0.0:8000)
|
| 202 |
-
- **Option 2**: Running the client in React.
|
| 203 |
-
```sh
|
| 204 |
-
cd client/web
|
| 205 |
-
npm install
|
| 206 |
-
npm start
|
| 207 |
-
```
|
| 208 |
-
After running these commands, a local development server will start, and your default web browser will open a new tab/window pointing to this server (usually http://localhost:3000).
|
| 209 |
-
- (Optional) Terminal client: Run the following command in your terminal
|
| 210 |
-
```sh
|
| 211 |
-
python client/cli.py
|
| 212 |
-
```
|
| 213 |
-
- (Optional) mobile client: open `client/mobile/ios/rac/rac.xcodeproj/project.pbxproj` in Xcode and run the app
|
| 214 |
-
- **Step 8**. Select one character to talk to, then start talking
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
## (Optional) π Installation via Docker
|
| 218 |
-
<details><summary>πclick me</summary>
|
| 219 |
-
|
| 220 |
-
1. Docker image: you can use our docker image directly
|
| 221 |
-
```sh
|
| 222 |
-
docker pull shaunly/real_char:latest
|
| 223 |
-
```
|
| 224 |
-
(Or you want build yourself) Build docker image
|
| 225 |
-
```sh
|
| 226 |
-
python cli.py docker-build
|
| 227 |
-
```
|
| 228 |
-
If you have issues with docker (especially on a non-Linux machine), please refer to https://docs.docker.com/get-docker/ (installation) and https://docs.docker.com/desktop/troubleshoot/overview/ (troubleshooting).
|
| 229 |
-
2. Run docker image with `.env` file
|
| 230 |
-
```sh
|
| 231 |
-
python cli.py docker-run
|
| 232 |
-
```
|
| 233 |
-
|
| 234 |
-
3. Go to http://localhost:8000 (NOT 0.0.0.0:8000) to start talking or use terminal client
|
| 235 |
-
```sh
|
| 236 |
-
python client/cli.py
|
| 237 |
-
```
|
| 238 |
-
|
| 239 |
-
</details>
|
| 240 |
-
|
| 241 |
-
<br/>
|
| 242 |
-
|
| 243 |
-
## π! LangSmith integration
|
| 244 |
-
<details><summary>πclick me</summary>
|
| 245 |
-
|
| 246 |
-
If you have access to LangSmith, you can edit these environment variables to enable:
|
| 247 |
-
```
|
| 248 |
-
LANGCHAIN_TRACING_V2=false # default off
|
| 249 |
-
LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
|
| 250 |
-
LANGCHAIN_API_KEY=YOUR_LANGCHAIN_API_KEY
|
| 251 |
-
LANGCHAIN_PROJECT=YOUR_LANGCHAIN_PROJECT
|
| 252 |
-
```
|
| 253 |
-
And it should work out of the box.
|
| 254 |
-
|
| 255 |
-
</details>
|
| 256 |
-
|
| 257 |
-
<br/>
|
| 258 |
-
|
| 259 |
-
## π Roadmap
|
| 260 |
-
- [x] Launch v0.0.1 and build a community
|
| 261 |
-
- [x] Move away from Vanilla JS
|
| 262 |
-
- [x] Launch mobile app (iOS TestFlight Beta link: https://testflight.apple.com/join/JA6p9sZQ)
|
| 263 |
-
- [ ] Add authentication for customization
|
| 264 |
-
- [ ] Allow selecting different LLM
|
| 265 |
-
- [ ] Add ability to add community characters
|
| 266 |
-
|
| 267 |
-
## π«Ά Contribute to RealChar
|
| 268 |
-
Please check out our [Contribution Guide](contribute.md)!
|
| 269 |
-
|
| 270 |
-
## πͺ Contributors
|
| 271 |
-
<a href="https://github.com/Shaunwei/RealChar">
|
| 272 |
-
<img src="https://contrib.rocks/image?repo=Shaunwei/RealChar" />
|
| 273 |
-
</a>
|
| 274 |
-
|
| 275 |
-
## π² Community
|
| 276 |
-
- Join us on [Discord](https://discord.gg/e4AYNnFg2F)
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: RealChar - Your Realtime AI Character
|
| 3 |
+
emoji: ποΈπ€
|
| 4 |
+
colorFrom: blue
|
| 5 |
+
colorTo: indigo
|
| 6 |
+
sdk: docker
|
| 7 |
+
sdk_version: 1.24.0
|
| 8 |
+
pinned: true
|
| 9 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|