diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 8aecc51..a4ef079 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,296 +1,155 @@ -# Chat Server Architecture - Complete Reference +# Reliquary Server Architecture - Complete Reference -## Project Overview +## 1. Project Overview & The Reliquary Protocol +Reliquary is a self-hosted, highly optimized Discord alternative for private communities. It operates on the **Reliquary Protocol**, a decentralized identity and routing model. -Self-hosted Discord alternative with decentralized identity. Each user has a "home server" where their identity lives, and can join other "guest servers" seamlessly using cryptographic signatures. - -## Core Architecture Decisions - -### Home Server Model -- **NO identity server** - just use one regular server as "home" -- **NO federation protocol** - servers don't talk to each other -- **Client does everything** - syncs, cross-server requests, profile updates - -### Identity System -- Current implementation: secp256k1 keypair/signatures via `crypton` EC -- Target (planned): Ed25519 migration -- Private key = proof of identity -- Public key stored on home server at `/.well-known/users/:username` -- userId format: `username@serverhost` - -### Single Sign-On Flow -1. User registers/logs in ONCE on home server -2. Gets JWT token + encrypted keypair -3. To join other servers: sign a message with private key -4. Other server verifies signature against home server's public key -5. No password ever sent to guest servers +* **Desktop-First & Native Performance:** The primary target is Desktop and Web, followed by Android (using foreground services for background connectivity). iOS and centralized Push Notifications are explicitly out of scope for the MVP to avoid centralized infrastructure. +* **No Server-to-Server Federation:** Servers don't talk to each other. The client handles all cross-server fetching and syncing. +* **Cryptographic Identity:** Users have a "Home Server" storing their public key (`/.well-known/users/:username`). They join "Guest Servers" using cryptographic signatures. No passwords are ever sent to Guest Servers. --- -## What's Implemented +## 2. Current State (What is already implemented) -### ✅ Done -- User model with keypair, home server reference -- Channel and Message models -- Crypto service (keypair generation, signing, verification) -- JWT authentication middleware -- Register handler (creates keypair) -- Login handler (returns encrypted keypair for device sync) -- Join server handler (signature-based verification with identity checks) -- HTTP client for cross-server public key fetching -- Public endpoints for cross-server identity verification -- Channel CRUD -- Message CRUD with author caching +### ✅ Implemented Features +* **Identity System:** secp256k1 keypair/signatures via `crypton` EC (Targeting Ed25519 migration later). +* **Auth Flow:** Single Sign-On on Home Server (returns JWT + encrypted keypair). +* **Cross-Server Join Flow:** Signature-based verification. Guest servers fetch public keys from Home Servers via HTTP. +* **Middleware:** JWT authentication middleware. +* **Basic CRUD:** Channels and Messages (Text only) with author caching. -### ❌ Not Implemented -- WebSocket gateway for real-time events -- Notification system (mentions/replies/unread state) -- Push delivery pipeline (FCM/APNs/WebPush device tokens + dispatch) -- Profile sync (push updates to all servers) -- Sticker packs (future feature) -- File uploads -- Permissions system (roles exist but not enforced) -- Message editing/deletion permissions +### 💾 Current Database Models (ObjectBox) +* **User:** `id`, `userId` ("alice@home.com"), `publicKey`, `passwordHash` (home only), etc. +* **Channel:** `id`, `name`, `createdAt`. +* **Message:** `id`, `channelId`, `authorLocalId`, `authorUserId`, `content` (currently simple text), `createdAt`. + +### 🌐 Current REST APIs +* `POST /auth/register`, `POST /auth/login`, `POST /auth/join` +* `GET /.well-known/users/:username`, `GET /.well-known/verify/:userId` +* `GET /api/me`, `GET /api/channels`, `POST /api/channels`, `GET /api/channels/:id` +* `GET /api/messages`, `POST /api/messages`, `PUT /api/messages/:id`, `DELETE /api/messages/:id` --- -## Database Models +## 3. Active Implementation Target: The Component & File System +*⚠️ THIS IS THE IMMEDIATE NEXT STEP TO BE CODED. ⚠️* + +We are migrating from simple text strings to a **Component-Based Message Structure** (Reliquary Components) to support rich media, link embeds, and collapsible UIs natively. + +### 3.1 UX Flow: Staged Uploads & Dummy Messages (Discord-Style) +To preserve server storage and bandwidth while maintaining a fast, non-blocking UI, Reliquary uses a **Staged Upload Pipeline**. + +1. **Stage (Local Client Action):** User selects multiple files. The client displays local thumbnails in the input area. The user can drag to reorder, toggle `spoiler` modifiers on individual files or the whole gallery, and add text. *No network requests are made during this phase.* +2. **Commit (Local Client Action):** The user hits "Send". The client clears the input box so the user can keep typing, and inserts a "Dummy Message" into the local chat history. This dummy displays a file upload progress bar. +3. **Upload (Network Action):** The client initiates `POST /api/uploads` for the files. +4. **Publish (Network Action):** Once the server responds with the `FileId`s and dimensions, the client constructs the final `componentsJson` tree and calls `POST /api/messages`. +5. **Resolve:** The server processes the message and broadcasts it back via WebSocket. The client silently swaps the "Dummy Message" with the finalized, permanent message. + + +### 3.2 Updated Database Models (To be modified/created) ```dart -// User - works for both HOME and GUEST servers -@Entity() -class User { - int id; - String userId; // "alice@homeserver.com" - String username; - String? displayName, avatarUrl, bio; - - String publicKey; // Ed25519, always stored - - // HOME SERVER ONLY: - String? passwordHash; - String? encryptedPrivateKey; // For device sync - String? keyPairSalt; - - bool isLocal; // true = this is home server - String? homeServer; // if guest, where is home? - List roles; // Server-specific - DateTime createdAt, lastSyncedAt; -} - -@Entity() -class Channel { - int id; - String name; - String? description; - DateTime createdAt; -} - @Entity() class Message { int id; int channelId; - - // Author info (includes cross-server fetch capability) - int authorLocalId; - String authorUserId; // "alice@home.server" - String? authorHomeServer; - - // Cached snapshot at message time - String authorUsername; - String? authorAvatarUrl; - - String content; - DateTime createdAt, editedAt; + + // 1. Search & Notifications (Server Generated) + // Server concatenates all text from components upon ingestion. + String content; + + // 2. The Component Tree (Source of Truth for UI) + // JSON string representing List + String componentsJson; + + // ... existing author and date fields ... } @Entity() -class Notification { +class File { int id; - int recipientLocalId; - String recipientUserId; // "alice@home.server" - - String type; // "mention", "reply", "system" - String title; - String body; - - int? channelId; - int? messageId; - - bool isRead; + int uploaderId; + String path; // Relative storage path (/uploads/...) + String mimeType; + int sizeBytes; + + // Safety Net Garbage Collection + // Because uploads and message creation are two separate API calls, + // if the client crashes in between step 3 and 4, the file is orphaned. + // A simple daily worker will delete files where isAttached == false and age > 24h. + bool isAttached; // False until used in a POST /api/messages DateTime createdAt; - DateTime? readAt; } ``` ---- +### 3.3 The Reliquary Component Schema (JSON) +All components stored in `componentsJson` must follow this base schema to prevent client crashes and layout shifts. -## API Endpoints - -### Public (No Auth) -``` -POST /auth/register → Create account on HOME server (generates keypair) -POST /auth/login → Login to HOME server (returns keypair backup) -POST /auth/join → Join THIS server with signature (no password) - -GET /.well-known/users/:username → Public profile + public key -GET /.well-known/verify/:userId → Verify user exists -GET /.well-known/server → Server info +**Base Schema:** +```json +{ + "type": "string", // "text", "gallery", "code", "embed", "collapsible" + "modifiers":, // e.g., (client applies blur overlay) + "fallback": "string", // Plain text for older clients / notifications + "data": { } // Type-specific payload +} ``` -### Protected (JWT Required) -``` -GET /api/me → Current user profile - -GET /api/channels → List channels -POST /api/channels → Create channel -GET /api/channels/:id → Get channel - -GET /api/messages?channelId=X → Get messages -POST /api/messages → Send message -PUT /api/messages/:id → Edit message -DELETE /api/messages/:id → Delete message - -GET /api/notifications → List notifications -POST /api/notifications/read → Mark notifications read - -POST /api/push/devices → Register device push token -DELETE /api/push/devices/:id → Remove push token +**Type: Text** +```json +{ + "type": "text", + "modifiers":[], + "fallback": "Hello world", + "data": { "content": "Hello **world**", "format": "markdown" } +} ``` ---- - -## Realtime & Notifications (Planned MVP) - -- Add WebSocket endpoint (e.g. `/gateway`) authenticated by JWT. -- Keep REST as source of truth; WebSocket is for event fanout. -- On message create/edit/delete: - 1. Write to DB - 2. Emit gateway event to subscribed clients - 3. Generate notification records for mention/reply targets -- If recipient is online, deliver notification over gateway immediately. -- If recipient is offline and has device token, enqueue push dispatch. -- Reconnect strategy: client backfills missed messages using - `GET /api/messages?channelId=X&after=lastSeenMessageId`. -- Full Discord-style resume/replay is optional and can be added later. - ---- - -## Key Files - -``` -lib/ -├── server.dart # Entry point, routes, ObjectBox init -├── models.dart # User, Channel, Message entities -├── objectbox.g.dart # Generated -│ -├── config/config.dart # Server host, JWT secret, port -│ -└── src/ - ├── auth/ - │ ├── crypto_service.dart # Keypair, signing, encryption - │ └── auth_middleware.dart # JWT verification - │ - ├── handlers/ - │ ├── auth_handlers.dart # register, login, join, me - │ ├── public_handlers.dart # .well-known/* - │ ├── channel_handlers.dart - │ └── message_handlers.dart - │ - └── utils/extensions.dart # Request/Response helpers +**Type: Gallery (Images/Video)** +*Must include aspect ratio or width/height to prevent Cumulative Layout Shift (CLS).* +```json +{ + "type": "gallery", + "modifiers": [], + "fallback": "", + "data": { + "layout": "grid", + "items": + } +} ``` ---- - -## Join Server Flow (Implemented) - -``` -CLIENT GUEST SERVER HOME SERVER - │ │ │ - │ POST /auth/join │ │ - │ {userId, homeServer, timestamp, │ │ - │ signature} │ │ - │─────────────────────────────────────►│ │ - │ │ │ - │ │ GET /.well-known/users/ │ - │ │ alice │ - │ │────────────────────────►│ - │ │ │ - │ │ {publicKey, ...} │ - │ │◄────────────────────────│ - │ │ │ - │ │ verify signature │ - │ │ create shadow account │ - │ │ │ - │ {token, localId} │ │ - │◄─────────────────────────────────────│ │ +**Type: Collapsible (Expandable Tile)** +```json +{ + "type": "collapsible", + "modifiers": [], + "fallback": "", + "data": { + "summary": "Server Logs (Click to expand)", + "children":, "data": { "language": "bash", "content": "Error 500" } } + ] + } +} ``` -The join flow now: -- Fetches identity from home server over HTTP (`/.well-known/users/:username`) -- Verifies claimed `userId` matches home-server response -- Verifies signature before issuing token (including already-joined users) +### 3.4 API Changes Required +**1. New Endpoint: File Upload** +* `POST /api/uploads` (multipart/form-data) +* Returns: `{ "id": 101, "url": "/files/101.jpg", "width": 1920, "height": 1080 }` + +**2. Modify Endpoint: Send Message** +* `POST /api/messages` +* Payload now expects: `{ "channelId": 1, "components": }` +* Server logic required: + 1. Parse `components`. + 2. Extract all readable text to populate the `content` database column. + 3. Validate any `fileIds` inside galleries. Mark those `File` entities as `isAttached = true`. + 4. Save `Message` entity with `componentsJson`. --- -## Tech Stack - -- **Framework**: Relic 1.0 (Dart web server) -- **Database**: ObjectBox -- **Auth**: JWT + secp256k1 signatures (current), Ed25519 (planned) -- **Crypto**: crypton package (EC) -- **Password**: bcrypt - ---- - -## Design Decisions Rationale - -| Decision | Why | -|----------|-----| -| No identity server | Avoids hosting another service, one server = home | -| No server-to-server | Eliminates federation complexity, client handles everything | -| Keypair identity | Allows passwordless joins, proves identity cryptographically | -| Message author caching | Messages readable even if home server is down | -| userId format `user@server` | Simple, familiar, self-documenting | - ---- - -## Constraints - -- User CANNOT quit home server (that's where identity lives) -- User CANNOT change home server easily (migration = future feature) -- Home server MUST be online to join NEW servers -- Home server offline = can still use already-joined servers (cached keys) - ---- - -## Future Features (Out of Scope for MVP) - -1. **Sticker Packs** - Telegram-style, live on community servers, clients cache -2. **Home Server Migration** - Sign migration certificate, chain of trust -3. **File Uploads** - With cross-server URLs -4. **Message Forwarding** - Embed with link to original -5. **Permissions** - Role-based access to channels -6. **WebSocket** - Real-time message delivery -7. **Advanced Notification Rules** - Mute/priority/snooze/digests - ---- - -## Config to Change - -```dart -// lib/config/config.dart -static const String serverHost = 'your-actual-domain.com'; // CHANGE THIS -static const String jwtSecret = 'use-env-variable'; // CHANGE THIS -``` - ---- - -## Next Steps - -1. Add WebSocket gateway (`/gateway`) with JWT auth + channel subscriptions -2. Add message history cursor pagination (`before` + `after`) -3. Implement notifications (DB model + generation rules + unread counters) -4. Add push token registration + push dispatch worker -5. Add profile sync (client pushes to all servers on change) -6. Add proper error handling, replay protection, and logging +## 4. Future Roadmap (Do NOT implement yet) +* **WebSocket Gateway:** Real-time event fanout (`/gateway`) with channel subscriptions. +* **Embedder Background Worker:** A server-side queue that intercepts links in messages, fetches OpenGraph/yt-dlp data, and appends an `embed` component asynchronously. +* **Garbage Collector:** A daily worker to purge unattached files from the disk.