jo1042/litellm-vector-store
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

More datatypes

Browse Source
This commit is contained in:
root
2026-04-29 09:11:46 +00:00
parent ef55253cbd
commit 965e900743
7 changed files with 519 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

200
README.md
View File

@@ -8,7 +8,8 @@ A vector store service built on top of [LiteLLM](https://github.com/BerriAI/lite
- 🗄️ **Vector Store** powered by PostgreSQL + pgvector
- 🔍 **Semantic Search** with optional Reranking
- 🤖 **RAG Endpoint** - Search + LLM in one request
- 📄 **File Upload** - PDF, DOCX, TXT, Markdown
- 📄 **File Upload** - PDF, DOCX, TXT, Markdown, Excel, CSV, PowerPoint, HTML, E-Mail, JSON
- 🖼️ **Image Support** - Upload images via Vision LLM (JPG, PNG, GIF, WebP, TIFF)
- 🧩 **OpenAI-compatible API** - works with existing OpenAI SDKs
- 👥 **Multi-User** - Store permissions per user
- 🖥️ **Admin UI** - Manage users, stores and permissions
@@ -20,20 +21,22 @@ A vector store service built on top of [LiteLLM](https://github.com/BerriAI/lite
Client (API Key)
│
▼
LiteLLM Proxy ──────────────────────┐
│ │
▼ ▼
Vector Store API Embedding Models
│ (via LiteLLM)
▼
PostgreSQL + pgvector
LiteLLM Proxy ──────────────────────────────┐
│ │
▼ ▼
Vector Store API LiteLLM Models
│ ┌──────────────────┐
▼ │ Embedding Models │
PostgreSQL + pgvector │ Vision Models │
│ LLM Models │
└──────────────────┘
```
## Requirements
- Kubernetes Cluster
- PostgreSQL with pgvector extension
- LiteLLM Proxy (deployed)
- PostgreSQL with pgvector extension (already deployed)
- LiteLLM Proxy (already deployed)
- Container Registry
## Quick Start
@@ -106,6 +109,7 @@ EOF
### 3. Configure
```bash
# Create secrets
kubectl create secret generic vector-api-secrets \
--namespace vector-store \
--from-literal=DATABASE_URL="postgresql://vecuser:pass@postgres:5432/vectordb" \
@@ -124,16 +128,17 @@ data:
ADMIN_USER_IDS: "your-admin-user-id"
API_URL: "https://api.your-domain.com"
EMBEDDING_MODEL: "your-embedding-model"
VISION_MODEL: "openai/gpt-4o-mini"
```
### 4. Build & Deploy
```bash
# API
# Build & push API
docker build -t your-registry/vector-store-api:1.0.0 .
docker push your-registry/vector-store-api:1.0.0
# Admin UI
# Build & push Admin UI
docker build \
-t your-registry/vector-store-admin:1.0.0 \
./ui
@@ -165,6 +170,7 @@ litellm-vector-store/
│ │ └── openai_compat.py # OpenAI-compatible API
│ └── utils/
│ ├── chunking.py # Text chunking
│ ├── image_processor.py # Vision LLM integration
│ └── stats.py # Usage tracking
├── ui/ # React Admin UI
│ ├── src/
@@ -216,6 +222,10 @@ Authorization: Bearer sk-your-api-key
| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/v1/models` | List all models |
| `GET` | `/v1/embeddings/models` | List embedding models |
| `GET` | `/v1/vision/models` | List vision models |
| `POST` | `/v1/embeddings` | Create embeddings |
| `POST` | `/v1/vector_stores` | Create store |
| `GET` | `/v1/vector_stores` | List stores |
| `GET` | `/v1/vector_stores/{id}` | Get store |
@@ -223,21 +233,21 @@ Authorization: Bearer sk-your-api-key
| `POST` | `/v1/vector_stores/{id}/files` | Add texts |
| `GET` | `/v1/vector_stores/{id}/files` | List files |
| `DELETE` | `/v1/vector_stores/{id}/files/{file_id}` | Delete file |
| `POST` | `/v1/vector_stores/{id}/upload` | Upload file |
| `POST` | `/v1/vector_stores/{id}/search` | Search |
| `POST` | `/v1/vector_stores/{id}/upload` | Upload file or image |
| `POST` | `/v1/vector_stores/{id}/search` | Semantic search |
| `POST` | `/v1/vector_stores/{id}/rag` | RAG query |
| `POST` | `/v1/embeddings` | Create embeddings |
| `GET` | `/v1/embeddings/models` | List embedding models |
| `GET` | `/v1/models` | List all models |
### Example
### Examples
#### Store anlegen & Datei hochladen
```python
import httpx
client = httpx.Client(
base_url="https://api.your-domain.com/v1",
headers={"Authorization": "Bearer sk-your-key"}
headers={"Authorization": "Bearer sk-your-key"},
timeout=120.0
)
# Create store
@@ -246,27 +256,129 @@ store = client.post(
json={"name": "My Knowledge Base"}
).json()
# Upload file
# Upload document
with open("document.pdf", "rb") as f:
client.post(
f"/vector_stores/{store['id']}/upload",
files={"file": f}
)
# Upload image (with default vision model)
with open("screenshot.png", "rb") as f:
client.post(
f"/vector_stores/{store['id']}/upload",
files={"file": f}
)
# Upload image (with custom vision model)
with open("diagram.png", "rb") as f:
client.post(
f"/vector_stores/{store['id']}/upload",
files={"file": f},
data={
"vision_model": "openai/gpt-4o",
"vision_prompt": "Explain this diagram in detail."
}
)
# Search
results = client.post(
f"/vector_stores/{store['id']}/search",
json={"query": "What is FastAPI?", "top_k": 3}
json={
"query": "What is FastAPI?",
"top_k": 3,
"rerank": True
}
).json()
# RAG
answer = client.post(
f"/vector_stores/{store['id']}/rag",
json={"query": "What is FastAPI?"}
json={
"query": "What is FastAPI?",
"model": "openai/gpt-4o-mini",
"rerank": True
}
).json()
print(answer["answer"])
```
#### JavaScript / TypeScript
```javascript
const API_KEY = "sk-your-api-key";
const BASE_URL = "https://api.your-domain.com/v1";
const HEADERS = {
"Authorization": `Bearer ${API_KEY}`,
"Content-Type": "application/json"
};
// Create store
const store = await fetch(`${BASE_URL}/vector_stores`, {
method: "POST",
headers: HEADERS,
body: JSON.stringify({ name: "My Store" })
}).then(r => r.json());
// Search
const results = await fetch(
`${BASE_URL}/vector_stores/${store.id}/search`, {
method: "POST",
headers: HEADERS,
body: JSON.stringify({
query: "What is FastAPI?",
top_k: 3,
rerank: true
})
}).then(r => r.json());
// RAG
const answer = await fetch(
`${BASE_URL}/vector_stores/${store.id}/rag`, {
method: "POST",
headers: HEADERS,
body: JSON.stringify({
query: "What is FastAPI?"
})
}).then(r => r.json());
console.log(answer.answer);
```
#### curl
```bash
# Create store
curl -X POST https://api.your-domain.com/v1/vector_stores \
-H "Authorization: Bearer sk-your-key" \
-H "Content-Type: application/json" \
-d '{"name": "My Store"}'
# Upload document
curl -X POST https://api.your-domain.com/v1/vector_stores/{store_id}/upload \
-H "Authorization: Bearer sk-your-key" \
-F "file=@document.pdf"
# Upload image with custom vision model
curl -X POST https://api.your-domain.com/v1/vector_stores/{store_id}/upload \
-H "Authorization: Bearer sk-your-key" \
-F "file=@diagram.png" \
-F "vision_model=openai/gpt-4o" \
-F "vision_prompt=Explain this diagram in detail."
# Search
curl -X POST https://api.your-domain.com/v1/vector_stores/{store_id}/search \
-H "Authorization: Bearer sk-your-key" \
-H "Content-Type: application/json" \
-d '{"query": "What is FastAPI?", "top_k": 3, "rerank": true}'
# RAG
curl -X POST https://api.your-domain.com/v1/vector_stores/{store_id}/rag \
-H "Authorization: Bearer sk-your-key" \
-H "Content-Type: application/json" \
-d '{"query": "What is FastAPI?", "model": "openai/gpt-4o-mini"}'
```
## Configuration Reference
### Environment Variables
@@ -278,15 +390,54 @@ print(answer["answer"])
| `LITELLM_MASTER_KEY` | ✅ | — | LiteLLM master key |
| `ADMIN_USER_IDS` | ✅ | — | Comma-separated admin user IDs |
| `EMBEDDING_MODEL` | ❌ | `text-embedding-ada-002` | Default embedding model |
| `VISION_MODEL` | ❌ | `openai/gpt-4o-mini` | Default vision model |
### Upload Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file` | file | — | File to upload |
| `chunk_size` | int | 512 | Characters per chunk |
| `chunk_overlap` | int | 50 | Overlap between chunks |
| `vision_model` | string | Config default | Vision model for images |
| `vision_prompt` | string | Auto | Custom prompt for vision model |
### Search Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `query` | string | — | Search query |
| `top_k` | int | 5 | Number of results (max. 50) |
| `rerank` | bool | false | Enable reranking |
| `rerank_model` | string | Auto | Custom rerank model |
### RAG Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `query` | string | — | Question |
| `model` | string | cosair/gemma4:31b | LLM model |
| `top_k` | int | 5 | Context documents |
| `rerank` | bool | false | Enable reranking |
| `system_prompt` | string | Auto | Custom system prompt |
| `messages` | array | [] | Chat history |
### Supported File Formats
| Format | Extension | Notes |
|--------|-----------|-------|
| Text | `.txt` | UTF-8 encoded |
| Markdown | `.md` | Standard Markdown |
| PDF | `.pdf` | Text PDFs only, no scans |
| Word | `.docx` | Microsoft Word 2007+ |
| Markdown | `.md` | Standard Markdown |
| Excel | `.xlsx` | All sheets extracted |
| CSV | `.csv` | All columns extracted |
| PowerPoint | `.pptx` | All slides extracted |
| HTML | `.html` `.htm` | Scripts/styles removed |
| Outlook Mail | `.msg` | Including headers |
| E-Mail | `.eml` | Including headers |
| JSON | `.json` | Pretty printed |
| Image | `.jpg` `.jpeg` `.png` `.gif` `.webp` `.tiff` | Via Vision LLM |
### Limits
@@ -320,6 +471,8 @@ DATABASE_URL="postgresql://..." \
LITELLM_PROXY_URL="http://..." \
LITELLM_MASTER_KEY="sk-..." \
ADMIN_USER_IDS="your-id" \
EMBEDDING_MODEL="your-model" \
VISION_MODEL="openai/gpt-4o-mini" \
uvicorn app.main:app --reload
# Run UI locally
@@ -336,6 +489,7 @@ VITE_API_URL=http://localhost:8000 npm run dev
| **Database** | PostgreSQL 16 + pgvector |
| **Auth** | LiteLLM Key Management |
| **Embeddings** | Via LiteLLM Proxy |
| **Vision** | Via LiteLLM Vision Models |
| **Admin UI** | React + TypeScript + Tailwind CSS |
| **Container** | Docker + Kubernetes |
| **Ingress** | NGINX Ingress Controller |