# litellm-vector-store

A vector store service built on top of [LiteLLM](https://github.com/BerriAI/litellm) and [pgvector](https://github.com/pgvector/pgvector), providing an OpenAI-compatible API for semantic search, document storage and Retrieval Augmented Generation (RAG).

## Features

- 🔐 **Authentication** via LiteLLM API Keys
- 🗄️ **Vector Store** powered by PostgreSQL + pgvector
- 🔍 **Semantic Search** with optional Reranking
- 🤖 **RAG Endpoint** - Search + LLM in one request
- 📄 **File Upload** - PDF, DOCX, TXT, Markdown
- 🧩 **OpenAI-compatible API** - works with existing OpenAI SDKs
- 👥 **Multi-User** - Store permissions per user
- 🖥️ **Admin UI** - Manage users, stores and permissions
- 📊 **Usage Tracking** - Track requests per user

## Architecture

```
Client (API Key)
      │
      ▼
LiteLLM Proxy ──────────────────────┐
      │                             │
      ▼                             ▼
Vector Store API            Embedding Models
      │                         (via LiteLLM)
      ▼
PostgreSQL + pgvector
```

## Requirements

- Kubernetes Cluster
- PostgreSQL with pgvector extension
- LiteLLM Proxy (deployed)
- Container Registry

## Quick Start

### 1. Clone Repository

```bash
git clone https://github.com/your-org/litellm-vector-store.git
cd litellm-vector-store
```

### 2. Database Setup

```bash
kubectl exec -it <postgres-pod> -n <namespace> \
  -- psql -U postgres -d vectordb << 'EOF'

CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

CREATE TABLE IF NOT EXISTS vector_stores (
    id            UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name          VARCHAR(255) NOT NULL,
    owner_user_id VARCHAR(255) NOT NULL,
    created_at    TIMESTAMP DEFAULT NOW()
);

CREATE TABLE IF NOT EXISTS documents (
    id         UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    store_id   UUID REFERENCES vector_stores(id) ON DELETE CASCADE,
    content    TEXT NOT NULL,
    metadata   JSONB DEFAULT '{}',
    embedding  vector(1024),
    created_at TIMESTAMP DEFAULT NOW()
);

CREATE TABLE IF NOT EXISTS store_permissions (
    store_id   UUID REFERENCES vector_stores(id) ON DELETE CASCADE,
    user_id    VARCHAR(255) NOT NULL,
    permission VARCHAR(50) DEFAULT 'read',
    PRIMARY KEY (store_id, user_id)
);

CREATE TABLE IF NOT EXISTS usage_stats (
    id         UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id    VARCHAR(255) NOT NULL,
    store_id   UUID REFERENCES vector_stores(id) ON DELETE SET NULL,
    action     VARCHAR(50) NOT NULL,
    tokens     INT DEFAULT 0,
    duration   FLOAT DEFAULT 0,
    created_at TIMESTAMP DEFAULT NOW()
);

CREATE INDEX IF NOT EXISTS idx_documents_store
    ON documents(store_id);
CREATE INDEX IF NOT EXISTS idx_documents_embedding
    ON documents USING ivfflat (embedding vector_cosine_ops)
    WITH (lists = 100);
CREATE INDEX IF NOT EXISTS idx_usage_user
    ON usage_stats(user_id);
CREATE INDEX IF NOT EXISTS idx_usage_created
    ON usage_stats(created_at);

GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO vecuser;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO vecuser;

EOF
```

### 3. Configure

```bash
kubectl create secret generic vector-api-secrets \
  --namespace vector-store \
  --from-literal=DATABASE_URL="postgresql://vecuser:pass@postgres:5432/vectordb" \
  --from-literal=LITELLM_MASTER_KEY="sk-master-key"
```

```yaml
# k8s/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: vector-store-config
  namespace: vector-store
data:
  LITELLM_PROXY_URL: "http://litellm.<namespace>.svc.cluster.local:4000"
  ADMIN_USER_IDS:    "your-admin-user-id"
  API_URL:           "https://api.your-domain.com"
  EMBEDDING_MODEL:   "your-embedding-model"
```

### 4. Build & Deploy

```bash
# API
docker build -t your-registry/vector-store-api:1.0.0 .
docker push your-registry/vector-store-api:1.0.0

# Admin UI
docker build \
  -t your-registry/vector-store-admin:1.0.0 \
  ./ui
docker push your-registry/vector-store-admin:1.0.0

# Deploy
kubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/configmap.yaml
kubectl apply -f k8s/secrets.yaml
kubectl apply -f k8s/vector-api/
kubectl apply -f k8s/admin-ui/
kubectl apply -f k8s/ingress-api.yaml
kubectl apply -f k8s/ingress-ui.yaml
```

## Project Structure

```
litellm-vector-store/
├── app/                          # FastAPI Backend
│   ├── main.py                   # Application entry point
│   ├── auth.py                   # LiteLLM authentication
│   ├── database.py               # PostgreSQL connection
│   ├── models.py                 # Pydantic models
│   ├── routers/
│   │   ├── stores.py             # Vector store CRUD
│   │   ├── documents.py          # Document management
│   │   ├── admin.py              # Admin endpoints
│   │   └── openai_compat.py      # OpenAI-compatible API
│   └── utils/
│       ├── chunking.py           # Text chunking
│       └── stats.py              # Usage tracking
├── ui/                           # React Admin UI
│   ├── src/
│   │   ├── pages/
│   │   │   ├── Login.tsx
│   │   │   ├── Dashboard.tsx
│   │   │   ├── Users.tsx
│   │   │   └── Stores.tsx
│   │   ├── components/
│   │   │   ├── Layout.tsx
│   │   │   └── PermissionModal.tsx
│   │   └── api/
│   │       └── client.ts
│   └── Dockerfile
├── k8s/                          # Kubernetes manifests
│   ├── namespace.yaml
│   ├── configmap.yaml
│   ├── secrets.yaml
│   ├── vector-api/
│   │   ├── deployment.yaml
│   │   └── service.yaml
│   ├── admin-ui/
│   │   ├── deployment.yaml
│   │   └── service.yaml
│   ├── ingress-api.yaml
│   └── ingress-ui.yaml
├── scripts/
│   └── init.sql                  # Database initialization
├── Dockerfile
├── requirements.txt
└── README.md
```

## API Reference

### Base URL

```
https://api.your-domain.com/v1
```

### Authentication

```
Authorization: Bearer sk-your-api-key
```

### Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/v1/vector_stores` | Create store |
| `GET` | `/v1/vector_stores` | List stores |
| `GET` | `/v1/vector_stores/{id}` | Get store |
| `DELETE` | `/v1/vector_stores/{id}` | Delete store |
| `POST` | `/v1/vector_stores/{id}/files` | Add texts |
| `GET` | `/v1/vector_stores/{id}/files` | List files |
| `DELETE` | `/v1/vector_stores/{id}/files/{file_id}` | Delete file |
| `POST` | `/v1/vector_stores/{id}/upload` | Upload file |
| `POST` | `/v1/vector_stores/{id}/search` | Search |
| `POST` | `/v1/vector_stores/{id}/rag` | RAG query |
| `POST` | `/v1/embeddings` | Create embeddings |
| `GET` | `/v1/embeddings/models` | List embedding models |
| `GET` | `/v1/models` | List all models |

### Example

```python
import httpx

client = httpx.Client(
    base_url="https://api.your-domain.com/v1",
    headers={"Authorization": "Bearer sk-your-key"}
)

# Create store
store = client.post(
    "/vector_stores",
    json={"name": "My Knowledge Base"}
).json()

# Upload file
with open("document.pdf", "rb") as f:
    client.post(
        f"/vector_stores/{store['id']}/upload",
        files={"file": f}
    )

# Search
results = client.post(
    f"/vector_stores/{store['id']}/search",
    json={"query": "What is FastAPI?", "top_k": 3}
).json()

# RAG
answer = client.post(
    f"/vector_stores/{store['id']}/rag",
    json={"query": "What is FastAPI?"}
).json()
print(answer["answer"])
```

## Configuration Reference

### Environment Variables

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `DATABASE_URL` | ✅ | — | PostgreSQL connection URL |
| `LITELLM_PROXY_URL` | ✅ | — | LiteLLM proxy URL |
| `LITELLM_MASTER_KEY` | ✅ | — | LiteLLM master key |
| `ADMIN_USER_IDS` | ✅ | — | Comma-separated admin user IDs |
| `EMBEDDING_MODEL` | ❌ | `text-embedding-ada-002` | Default embedding model |

### Supported File Formats

| Format | Extension | Notes |
|--------|-----------|-------|
| Text | `.txt` | UTF-8 encoded |
| PDF | `.pdf` | Text PDFs only, no scans |
| Word | `.docx` | Microsoft Word 2007+ |
| Markdown | `.md` | Standard Markdown |

### Limits

| Limit | Value |
|-------|-------|
| Max file size | 256 MB |
| Max search results | 50 |
| Request timeout | 600 seconds |
| Default chunk size | 512 characters |
| Default chunk overlap | 50 characters |

## Admin UI

The Admin UI is available at `https://admin.your-domain.com`.

Login with your Admin API Key to:

- 📊 View usage statistics
- 👥 Manage users and their stores
- 🔑 Rotate API keys
- 🔒 Grant/revoke store permissions

## Development

```bash
# Install dependencies
pip install -r requirements.txt

# Run locally
DATABASE_URL="postgresql://..." \
LITELLM_PROXY_URL="http://..." \
LITELLM_MASTER_KEY="sk-..." \
ADMIN_USER_IDS="your-id" \
uvicorn app.main:app --reload

# Run UI locally
cd ui
npm install
VITE_API_URL=http://localhost:8000 npm run dev
```

## Tech Stack

| Component | Technology |
|-----------|-----------|
| **API** | FastAPI + Python 3.12 |
| **Database** | PostgreSQL 16 + pgvector |
| **Auth** | LiteLLM Key Management |
| **Embeddings** | Via LiteLLM Proxy |
| **Admin UI** | React + TypeScript + Tailwind CSS |
| **Container** | Docker + Kubernetes |
| **Ingress** | NGINX Ingress Controller |
| **TLS** | cert-manager + Let's Encrypt |

## License

MIT License - see [LICENSE](LICENSE) for details.

## Contributing

1. Fork the repository
2. Create your feature branch (`git checkout -b feature/my-feature`)
3. Commit your changes (`git commit -m 'Add my feature'`)
4. Push to the branch (`git push origin feature/my-feature`)
5. Open a Pull Request