Major architecture update: stable identity, component validation, media indexing, and protocol docs
This commit is contained in:
Foxyz committed
1 parent
ce7c97d13a
commit
01c110bbe1
16 files changed
+2718
-468
No files matched your search
+137
-147
@@ -1,172 +1,162 @@
|
||||
# Reliquary Architecture & Protocol Reference
|
||||
# Reliquary Server Architecture
|
||||
|
||||
## 1. Project Overview
|
||||
**Reliquary** is a self-hosted, high-performance chat platform operating on the **Reliquary Protocol**.
|
||||
* **Philosophy:** Desktop-first, native performance, decentralized identity.
|
||||
* **Routing:** No server-to-server federation. The Client handles all routing.
|
||||
* **Identity:** Cryptographic keypairs (secp256k1). User ID: `user@home.server`.
|
||||
* **Push Notifications:** Explicitly out of scope for MVP. Desktop relies on background WebSockets; Android relies on Foreground Services.
|
||||
## 1. Overview
|
||||
Reliquary is a self-hosted chat server with:
|
||||
- JWT auth
|
||||
- ObjectBox persistence
|
||||
- WebSocket gateway fanout
|
||||
- Component-based messages
|
||||
- Decoupled upload pipeline (upload first, message commit anytime)
|
||||
|
||||
---
|
||||
## 2. Identity and Roles (Current)
|
||||
- `userId` is an immutable opaque identifier (host-independent).
|
||||
- `handle` is derived metadata: `@username@home`.
|
||||
- `ChatUser.roles` controls authorization (`admin`, `member`, etc.).
|
||||
- `GET /api/me` returns `roles` so the client can gate admin UI.
|
||||
|
||||
## 2. Current System State
|
||||
*The following features are implemented or have a fixed specification.*
|
||||
### Runtime Config Split
|
||||
- `server.bind_host` + `server.bind_port`: where Relic listens locally.
|
||||
- `server.public_base_url`: canonical external URL used for identity/handles/cross-server metadata.
|
||||
- This avoids conflating local bind settings with public identity/routing.
|
||||
|
||||
### ✅ Core Features
|
||||
* **Auth:** JWT-based. Register/Login returns encrypted keypair.
|
||||
* **Join Flow:** Cryptographic signature verification.
|
||||
* **REST API:** Basic CRUD for Channels and Text-only Messages.
|
||||
* **Gateway:** WebSocket server for event fanout (Specification defined in Section 4).
|
||||
### First Admin Bootstrap
|
||||
- Config key: `server.first_setup` in `config.yaml`.
|
||||
- When `first_setup: true`, the first registered local user gets `admin`.
|
||||
- Later registrations remain `member`.
|
||||
|
||||
### 💾 Database Models (ObjectBox)
|
||||
* **User:** `userId`, `publicKey`, `passwordHash` (home only).
|
||||
* **Channel:** `id`, `name`, `createdAt`.
|
||||
* **Message:** `id`, `channelId`, `authorUserId`, `content`, `createdAt`.
|
||||
## 3. Storage Model
|
||||
|
||||
---
|
||||
### 3.1 Entities
|
||||
- `ChatUser`: identity/profile/auth data.
|
||||
- `ChatChannel`: channel metadata.
|
||||
- `ChatMessage`: channel message, search text, and `componentsJson`.
|
||||
- `ChatFile`: canonical media object for:
|
||||
- message attachments
|
||||
- user avatar media
|
||||
- server icon/banner media
|
||||
- `ServerSettings`: singleton row for mutable server presentation settings.
|
||||
|
||||
## 3. Active Implementation Target: Component & File System
|
||||
*⚠️ THIS SECTION DESCRIBES NEW FEATURES TO BE CODED ⚠️*
|
||||
### 3.2 ChatFile Fields (Important)
|
||||
- `attachmentId` (unique external ID; API-facing)
|
||||
- `originalFileName` (preserved filename metadata)
|
||||
- `storedFileName` (physical disk key)
|
||||
- `sha256`
|
||||
- `mimeType`, `sizeBytes`, `width`, `height`
|
||||
- `usageType`: `message | avatar | server_icon | server_banner`
|
||||
- `status`: `pending | ready | failed`
|
||||
|
||||
We are migrating from plain text to **Reliquary Components**. This allows rich media, embeds, and collapsible UIs.
|
||||
## 4. Upload and Message Flow
|
||||
|
||||
### 3.1 UX Flow: Staged Uploads (Discord-Style)
|
||||
To ensure instant UI feedback and save server bandwidth:
|
||||
1. **Stage (Client):** User picks files. Client shows local previews. User reorders/adds spoilers. *No network request yet.*
|
||||
2. **Commit (Client):** User hits "Send". Client adds a "Dummy Message" to the UI with a progress bar.
|
||||
3. **Upload (Network):** Client POSTs files to `/api/uploads`.
|
||||
4. **Publish (Network):** Client gets `FileId`s, builds the `componentsJson`, and POSTs to `/api/messages`.
|
||||
5. **Resolve (Network):** Server broadcasts the new message via Gateway. Client swaps "Dummy" for "Real".
|
||||
### 4.1 Recommended Client Flow (Implemented Server Contract)
|
||||
1. Client stages attachment in composer/component editor.
|
||||
2. Client calls `POST /api/uploads/init` to create pending upload.
|
||||
3. Client uploads bytes to `PUT /api/uploads/:uploadId`.
|
||||
4. Client sends `POST /api/messages` with components referencing `uploadId`/`attachmentId`.
|
||||
5. If upload completes after message create, server emits `message.updated`.
|
||||
|
||||
### 3.2 New/Modified Database Models
|
||||
```dart
|
||||
@Entity()
|
||||
class Message {
|
||||
// ... existing fields ...
|
||||
|
||||
// Search Index (Server concatenates text from all components)
|
||||
String content;
|
||||
### 4.2 Why This Flow
|
||||
- No chronological jumping for text.
|
||||
- Immediate local optimistic UI.
|
||||
- Progressive media readiness via gateway updates.
|
||||
- Server keeps authoritative metadata and storage consistency.
|
||||
|
||||
// Source of Truth (JSON List<Component>)
|
||||
String componentsJson;
|
||||
}
|
||||
## 5. API Surface
|
||||
|
||||
@Entity()
|
||||
class File {
|
||||
int id;
|
||||
int uploaderId;
|
||||
String path; // /uploads/uuid.jpg
|
||||
String mimeType;
|
||||
int width; // CRITICAL for layout
|
||||
int height;
|
||||
int sizeBytes;
|
||||
|
||||
bool isAttached; // False until linked to a Message
|
||||
DateTime createdAt;
|
||||
}
|
||||
```
|
||||
### 5.0 Identity Contract
|
||||
- Registration returns:
|
||||
- `userId` (opaque stable ID)
|
||||
- `username`
|
||||
- `handle` (computed, mutable as home host changes)
|
||||
- Cross-server join request now expects:
|
||||
- `userId`
|
||||
- `username`
|
||||
- `homeServer`
|
||||
- `timestamp`
|
||||
- `signature` over `userId:timestamp`
|
||||
|
||||
### 3.3 Component JSON Schema
|
||||
All items in `componentsJson` must follow this structure.
|
||||
### 5.1 Uploads and Attachments
|
||||
- `POST /api/uploads/init` (auth)
|
||||
- `PUT /api/uploads/:uploadId` (auth)
|
||||
- `POST /api/uploads` (auth, legacy single-step compatibility)
|
||||
- `GET /attachments/:attachmentId` (public immutable media URL)
|
||||
- `GET /api/channels/:channelId/media` (auth, gallery index feed)
|
||||
|
||||
### 5.2 Profile and Server Media
|
||||
- `PATCH /api/me` (auth): update profile text fields.
|
||||
- `PUT /api/me/avatar` (auth): bind uploaded media as avatar.
|
||||
- `GET /avatars/:userId` (public alias for current avatar).
|
||||
- `GET /icons/:serverId` (public alias for current server icon).
|
||||
- `GET /banners/:serverId` (public alias for current server banner).
|
||||
|
||||
### 5.3 Server Settings
|
||||
- `GET /api/server/settings` (auth)
|
||||
- `PATCH /api/server/settings` (admin)
|
||||
- `PUT /api/server/icon` (admin)
|
||||
- `PUT /api/server/banner` (admin)
|
||||
|
||||
## 6. Gateway Behavior
|
||||
- Existing events:
|
||||
- `message.created`
|
||||
- `message.updated`
|
||||
- `message.deleted`
|
||||
- For upload-driven updates:
|
||||
- A message can be created with media items in `uploading`.
|
||||
- When upload becomes ready, server rewrites affected components and emits `message.updated`.
|
||||
|
||||
## 7. Component Media Item Contract (Client)
|
||||
Each media item in a component should carry stable IDs and upload state:
|
||||
|
||||
**Base Wrapper:**
|
||||
```json
|
||||
{
|
||||
"type": "text | gallery | collapsible | embed",
|
||||
"modifiers":, // "spoiler" applies blur
|
||||
"fallback": "String for push notifs",
|
||||
"data": { ... }
|
||||
"itemId": "local_or_stable_id",
|
||||
"source": "local_upload|external_url",
|
||||
"uploadId": "pending_or_ready_attachment_id",
|
||||
"attachmentId": "same_as_uploadId_when_known",
|
||||
"status": "uploading|ready|failed",
|
||||
"url": "/attachments/<attachmentId>",
|
||||
"originalFileName": "image.png",
|
||||
"mimeType": "image/webp",
|
||||
"sizeBytes": 12345,
|
||||
"width": 1280,
|
||||
"height": 720,
|
||||
"thumbHash": "optional"
|
||||
}
|
||||
```
|
||||
|
||||
**Gallery Component (Images/Video):**
|
||||
```json
|
||||
{
|
||||
"type": "gallery",
|
||||
"modifiers": [],
|
||||
"fallback": "Image",
|
||||
"data": {
|
||||
"items":
|
||||
}
|
||||
}
|
||||
```
|
||||
### 7.1 Validation Rules (Server)
|
||||
- Every component node must include:
|
||||
- `id` (string)
|
||||
- `type` (string)
|
||||
- `typeVersion` (positive int)
|
||||
- `data` (object)
|
||||
- Nested components are supported under `data.children` with depth/size limits.
|
||||
- Media item rules:
|
||||
- `source=local_upload` => `attachmentId` required.
|
||||
- `source=external_url` => valid `http/https` `url` required.
|
||||
- Invalid component payloads are rejected server-side.
|
||||
|
||||
---
|
||||
## 8. Client Implementation Notes
|
||||
|
||||
## 4. Gateway Protocol Specification (WebSockets)
|
||||
*The server implements a WebSocket Gateway at `/gateway`.*
|
||||
### 8.1 Staging and Transform Choice
|
||||
On file pick/drop, choose transform mode before upload starts:
|
||||
- `Compressed WebP` (recommended default)
|
||||
- `WebP`
|
||||
- `Original`
|
||||
|
||||
### Connection & Auth
|
||||
* **URL:** `ws://host:port/gateway`
|
||||
* **Auth:** `?token=<jwt>` (Query param) or `Authorization: Bearer <jwt>` (Header).
|
||||
* **OpCode: Ready** (Server -> Client on connect):
|
||||
```json
|
||||
{ "op": "ready", "connectionId": "gw_1", "userId": "alice@home" }
|
||||
```
|
||||
If user changes mode for a staged file:
|
||||
- cancel in-flight upload
|
||||
- reprocess
|
||||
- re-upload
|
||||
|
||||
### Client -> Server Ops
|
||||
**Subscribe:**
|
||||
```json
|
||||
{ "op": "subscribe", "channelIds": [12, 13] }
|
||||
```
|
||||
**Unsubscribe:**
|
||||
```json
|
||||
{ "op": "unsubscribe", "channelIds": [12] }
|
||||
```
|
||||
**Heartbeat:**
|
||||
```json
|
||||
{ "op": "ping" }
|
||||
```
|
||||
### 8.2 Metadata Trust
|
||||
Client may send metadata hints, but server metadata is canonical.
|
||||
Client should refresh with upload response values.
|
||||
|
||||
### Server -> Client Ops (Events)
|
||||
**Message Created:**
|
||||
```json
|
||||
{
|
||||
"op": "event",
|
||||
"event": "message.created",
|
||||
"channelId": 12,
|
||||
"message": {
|
||||
"id": 101,
|
||||
"content": "Legacy text or fallback",
|
||||
"componentsJson": "[...]",
|
||||
"authorId": "alice@home",
|
||||
"createdAt": "..."
|
||||
}
|
||||
}
|
||||
```
|
||||
*(Also supports `message.updated` and `message.deleted`)*
|
||||
### 8.3 Caching
|
||||
- `/attachments/:attachmentId`: immutable and long-cache.
|
||||
- `/avatars`, `/icons`, `/banners`: mutable aliases; revalidate frequently.
|
||||
|
||||
---
|
||||
|
||||
## 5. API Reference: Pagination & Uploads
|
||||
|
||||
### GET /api/messages
|
||||
Used for initial load and history backfilling.
|
||||
* **Params:**
|
||||
* `channelId` (required)
|
||||
* `limit` (default 50)
|
||||
* `before` (Message ID) -> Load older messages.
|
||||
* `after` (Message ID) -> Load newer messages (catch-up).
|
||||
|
||||
* **Response:**
|
||||
```json
|
||||
{
|
||||
"messages": [],
|
||||
"hasMore": true,
|
||||
"nextBefore": 51, // Cursor for older
|
||||
"nextAfter": 100 // Cursor for newer
|
||||
}
|
||||
```
|
||||
|
||||
### POST /api/uploads (New)
|
||||
* **Format:** `multipart/form-data`
|
||||
* **Response:**
|
||||
```json
|
||||
{
|
||||
"id": 505,
|
||||
"url": "/uploads/505.png",
|
||||
"width": 1920,
|
||||
"height": 1080,
|
||||
"mimeType": "image/png"
|
||||
}
|
||||
```
|
||||
## 9. AVIF and JPEG XL Policy
|
||||
- MVP default: WebP (best compatibility/perf tradeoff).
|
||||
- AVIF: optional client mode if target platform decode support is verified.
|
||||
- JPEG XL: defer for now due to uneven ecosystem support.
|
||||
Reference in new issue
Block a user