Foxyz/sync_replay_project
1
0 Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 13 Wiki Activity

Docs rewrite: current-state docs, HANDOFF retired

Browse Source 
README for the repo page; HOW-IT-WORKS overview; ARCHITECTURE,
PROTOCOL, DEVELOPMENT checked against the code; ROADMAP is backlog
only (isolate/dedicated server, sync fixes, rendering+shaders,
Windows build, Jellyfin epic). HANDOFF.md and mpv-http-contract.md
folded in and deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
foxyzandClaude Opus 5.5 committed 2026-10-07 00:42:28 +03:00
1 parent aeff087c7d
commit d8a0729029
9 files changed
+680 -687
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/ARCHITECTURE.md
+216 -68
View File
@@ -1,84 +1,232 @@
# ARCHITECTURE
# Architecture
One binary, three modes. `packages/server` (Relic) is embedded, not a separate product.
Plain overview first: [HOW-IT-WORKS.md](HOW-IT-WORKS.md). Wire format:
[PROTOCOL.md](PROTOCOL.md). Plans: [ROADMAP.md](ROADMAP.md).
- **Client**: GUI Join, sync WS client (`web_socket_channel` in
`apps/desktop`, NOT `packages/core` — core stays Flutter-free)
+ mpv. No server code runs.
- **Server+Client (Host)**: GUI Host starts `packages/server` in-process, binds port, self-connects via loopback. Host opens ports, runs traffic. Clients open nothing outbound-only.
- **Headless**: `--headless`, same server package, no window/player init.
## Packages
Server side: **Relic** (HTTP + built-in WS). Client side WS: **`web_socket_channel`**
(Relic has no client; server WS handling stays in Relic).
| Path | What | Rule |
|---|---|---|
| `packages/core` | protocol messages, sync engine (`RoomEngine`), constants, config, Range parsing, logging | pure Dart, no Flutter |
| `packages/server` | Relic HTTP + WebSocket server (`RoomServer`, `RoomHub`, `Session`, media endpoint), `bin/server.dart` | pure Dart, no Flutter |
| `apps/desktop` | Flutter app: UI, `SyncClient`, `SessionCoordinator`, `PlaybackController` (media_kit/mpv), updater | the only shipped binary |
| `tool` | harnesses: `range_logger`, `shaper`, `fake_viewer`, `e2e_check` | dev only |
| `deploy` | systemd unit for the standalone server (stale until the dedicated server exists) | |
Media order (locked): links first (shared http(s) URL, Syncplay-style sync),
file relay after (server-proxied bytes for the whitelist case).
Guard: `packages/core/test/flutter_free_test.dart` fails if anything under
`core/lib` imports Flutter, `dart:ui` or `signals_flutter`; CI runs
analyze + tests for core and server on every push. Server has no such test but
the same rule; it is checked by CI building without Flutter. The client
WebSocket (`web_socket_channel`) lives in `apps/desktop`; Relic has no client.
## Media paths (links vs file relay)
## Server lifecycle
Two ways to watch, same sync engine, same Range endpoint:
Today (in-process): `SessionState.host()` creates a `RoomServer`, starts it
on the chosen port (bound to all IPv4 interfaces), then the app joins it over
`127.0.0.1` like any other client. Leaving or closing the window stops it.
`RoomServer` runs a 1 s heartbeat: stale sweep, closed-socket sweep, silent
socket drop, room + feed broadcast. There is one session, id `default`.
1. **Links (primary, shipped):** host shares an http(s) media URL
(e.g. a Jellyfin Download URL). Viewers stream it directly; the
server is sync-authority only. The URL travels as `media.url`
in room broadcasts and as welcome `mediaUrl` (set to the shared
URL when the server holds no file). This
covers most sessions — proxying is usually unnecessary.
2. **File relay (secondary, local-file half shipped):** host serves
bytes from their own machine through the server's token-gated
Range endpoint `GET /m/<sid>/<token>/<name>`. Viewers (and the
host via loopback) play the server URL. This is the fallback for
the whitelist case: viewers cannot reach the source, but the
server/host can (e.g. Jellyfin allowlists the host, not the
viewers). No server storage — bytes stream file→socket, config +
logs only, never to disk.
`--headless` does not work: the Flutter binary needs a display and exits with
"cannot open display" on a headless box. `packages/server/bin/server.dart` is
the only working standalone server (flags `--port`, `--secret`, `--name`,
`--media`, `--advertise-host`, `--config`; default port 8080).
Backends behind `MediaSource` (`packages/core/lib/src/media_source.dart`):
`FileMediaSource` (local file — SHIPPED as an abstraction, but the
live route serves `SessionMedia.file` directly via `file.openRead`;
nothing in `packages/` references `FileMediaSource` today);
`HttpSource` (server fetches a remote URL and re-serves it —
DEFERRED, Phase 8). `resolveMediaRequest` stays
framework-agnostic so both ride the same tested Range table.
Planned: hosting from the app runs the server on a Dart isolate (ports and
sockets stay in the same process, UI stays responsive); the dedicated server
is a separate `dart compile exe` of `packages/server`. See [ROADMAP.md](ROADMAP.md).
## Deferred: remote-host upload tunnel (NOT active protocol)
Passwords: the default secret is empty, which means an open server (a
warning is logged). Otherwise one shared secret, compared at hello.
The host is the member that connected from a loopback address; it cannot be
kicked. Kick = revoke token, send `kicked`, close; the
person may rejoin at once with a new token.
If a VPS tier ever needs a *remote* host to push bytes to a server
it cannot serve from disk, the reserved design is the brief's WSS
tunnel: host dials out, JSON control frames
(`hello`/`welcome`+mediaUrl, `open`{id,start,end}, `ack`, `cancel`,
`eof`/`err`) plus binary `[u32 BE reqId][payload]` chunks —
chunk 256 KiB, credit window 8 (≈2 MiB in flight), max 16 concurrent
`open`, relay acks only after socket flush (real backpressure).
`RelayConstants.chunkBytes/creditWindowChunks/maxConcurrentOpen`
are reserved for this. No code exists; direct-serve covers v1
(host IS the server, opens ports itself).
## Sync model
On file relay, host playback goes through loopback like everyone
else's (`http://127.0.0.1:<port>/…`) — same code path as remote
viewers, so server bugs surface on the host screen first. (On
links the host plays the direct URL.) Deployment ladder,
cheapest first: host's own PC (LAN, or internet with port-forward) →
same binary `--headless` on a friend's always-on box → cheap VPS
(CGNAT escape; TLS/Caddy deferred with this tier).
Loosely modeled on Syncplay, with a different readiness design.
Sync: heavily Syncplay-inspired + original readiness design, NOT 1:1
(audit done; deviations: clock never moves from reports, no
pause-on-leave, force-for-all, quorum gate — see HANDOFF §2).
Additive namespaced extensions only, `proto` bump + `PROTOCOL.md` same-commit.
- **Transport** is `playing` or `held`. The server owns the clock:
`positionAt(now) = position + (playing ? now - serverTime : 0)`.
Reports from clients never move it. Only seek, swap and gate transitions
write `position`/`serverTime` and bump `eventId`.
- **Reports**: clients send `position`, `buffered`, `eventId` every 1 s. The
server stores `buffered` and liveness; a report with `eventId` older than
current has its position ignored.
- **Ready gate** (HELD to PLAYING): the in-set is non-empty, every in-set
member is ready, fresh and has `buffered >= 4 s` (`minBufferSecs`), and
quorum holds: in-set size >= 2, unless at most one member is fresh. So a
lone viewer plays alone; with a second viewer present one Space waits for the
other. Joining members start with `inSet=false, ready=false` and never block.
Disconnect removes from the set; swap clears everyone.
- **Pause**: while PLAYING, any in-set member who becomes not ready, under
4 s buffered or stale flips the server to HELD (feed action `pause` by `server`).
Recovery resumes by itself when the gate passes.
- **Seek**: allowed from any member in any transport; the server writes the
position and bumps `eventId`; flags are untouched. Clients diff each `room`
broadcast by `eventId` (`RoomDiff`): new event means seek to the computed
room position, transport change means play/pause. Between events: drift
over 5 s for more than 2 s seeks, ahead by more than 1.5 s sets rate 0.95
until level, otherwise nothing. Local seeks are detected by the coordinator
polling mpv position once per second (`_detectUserSeek`, jump over 2 s and
more than 2 s from the room) and sent as a `seek` intent; a retry
(`_checkSeekLanded`) re-applies a room seek that has not landed after 1.5 s.
Known roughness: see ROADMAP.
- **Liveness**: WS ping every 5 s; silent over 15 s means stale (out of the
gate, `stale` feed); over 30 s the socket is closed and the member removed.
Closed sockets are polled with `isClosed` every second: Relic's server-side
`onDone` does not fire for dead peers, so presence never relies on it.
- **Client policies**: no room broadcast for 5 s means connection lost, clear
the server state and reconnect every 1.5 s until cancelled (an epoch counter makes
a cancelled attempt stale) or kicked. After a reconnect the ready flag is
restored from the server's state. Cancel during setup aborts the join.
Closing the window leaves gracefully (2 s timeout).
- **Idle** (`IdleTracker`): measured from the later of the last input and the
moment playback stopped. While HELD and self ready: warning at 8 min,
auto-unready at 10 min, and never less than 2 min after the warning.
- **Deviations from Syncplay**: clock never follows the slowest watcher; no
pause-on-leave; anyone can force ready/unready (all or one); quorum gate;
the clock is server time, and remote clock offset is not corrected (clients
assume their clock matches the server's).
No server storage: config + logs only. Media never touches disk.
Additive wire changes only; bump `proto` and PROTOCOL.md in the same commit.
## Health reporting (standing rule)
## Media
Every failure must display. Nothing stalls silently.
Two sources, same sync.
- Server `/health` returns machine-readable status: ok, room, viewer
count, uptime, media, bytes served, version. The app reads it.
- App surfaces all failure states in UI with cause + what-to-do:
connecting/reconnecting, wrong secret (401), kicked (403), session
gone (410), host offline / buffering (503), starvation
("waiting for X — N.N s buffered"). Backed by a structured status
signal (severity + message), not bare strings.
- Errors log via `debugPrint` + stack trace at the boundary, then
convert to a displayable state. No swallowed exceptions.
- **Link**: the media field holds an http(s) URL. It travels as `media.url`
in `swap` and `room`, and as `mediaUrl` in `welcome`. Each app plays it
directly; the server is only the clock.
- **Host local file**: `RoomServer.setLocalFile` / `mediaPath` makes the file
the session media. `GET /m/<sid>/<token>/<name>` serves it with a Range
table, gated by the viewer's token. The server builds the URL per viewer
(`http://<host>:<port>/m/default/<token>/<name>`), so tokens never reach
other viewers. `<host>` is the authority the client dialled; `advertiseHost`
(default empty) only overrides it when set explicitly. When a `room`
broadcast has no `media.url`, the client builds the URL itself from the host
it connected to. Bytes stream file to socket and are counted as streamed
(`bytesServed`); nothing is stored. The host plays through loopback like
everyone else.
Range table (`resolveMediaRequest`, tested byte-exact):
| Request | Response |
|---|---|
| no Range | 200, `Content-Length` = size |
| `bytes=a-b` | 206, `Content-Range: bytes a-b/size` |
| `bytes=a-` | 206 to EOF |
| `bytes=-n` | 206 last n bytes |
| unsatisfiable | 416, `Content-Range: bytes */size` |
| host has no file | 503 + `Retry-After: 2` |
| bad / unknown token, kicked, unknown session | 401, 403, 410 |
Always `Accept-Ranges: bytes`; `Content-Type` from the extension
(`contentTypeForFile`). Relic hands the Range over as a typed `RangeHeader`;
the endpoint takes the first range and keeps the unit, so non-`bytes` units
still get 416.
What mpv does against it (recorded with `tool/range_logger.dart`): mp4 opens
once with no Range then streams `bytes=0-` to the end; mkv opens `bytes=0-`
twice (probe, playback), reads a small tail at `size-n` (cues, expressed
open-ended), then every seek is a fresh `bytes=N-` connection. A server
outage of a few seconds is ridden out from mpv's cache with no error. Longer
outages exhaust the cache and rely on reconnect (not chaos-tested yet).
mpv settings (`PlaybackController`): `Player` with `bufferSize` 64 MiB,
`libass` on; pinned via runtime property: `demuxer-readahead-secs=10`,
`cache-secs=10`, `stream-lavf-o=reconnect_streamed=1,reconnect_on_http_error=1,reconnect_delay_max=4`
(comma form; this mpv build rejects colon-joined). media_kit has no public
`setProperty`; the code calls it through a guarded dynamic call on
`player.platform`. Hardware decoding comes from Settings (auto, off, custom)
and is passed to the `VideoController`; env `SYNC_RELAY_HWDEC` (e.g. `no`)
overrides it. It is read per player, so a change applies on the next join or
Reload video.
Player lifecycle: the `Player` and its `VideoController` live for one
session: created on join, disposed on leave, replaced by `recreate()` for
"Reload video" (button or Ctrl+R; volume kept). `openUrl` opens with `play: false`; play/pause/seek/rate come only from
`RoomDiff` actions or the play gate (`localPlayAllowed`: self in-set and
ready). A room seek within 0.5 s of the current position is skipped. `leave()`
stops the player (otherwise audio keeps playing). On media change
(`mediaId` bump) the coordinator opens the new URL and `RoomDiff` seeks to 0,
holds and resets rate.
Frozen picture: `FrameWatchdog` samples about once a second. It hashes a
small capture of the compositor's centre (`ShownFrameProbe`) and suspects a
freeze when position advances while that hash stays constant for 3 s. Two
differing mpv `screenshot-raw` frames show the decoder is still producing
pictures, which confirms the freeze (a still scene is dismissed). A confirmed
freeze reloads the video with a toast. It fires once until the picture moves
again for 10 s, so it cannot loop. Seeks, pause, buffering and a hidden window
disarm it.
Never unmount GPU-textured widgets (avatar images) on a transport-gated
branch: teardown races mpv's frame upload and the window goes black. Keep them
mounted and skip painting (`Visibility` with `maintainState`).
## Desktop UI
One `HomeScreen` (`screens/home/`) with a nav rail: Home (profile chip, Join
card, Host card, up to 6 Recent server chips for one-click rejoin), Settings
(profile, hardware decoding, updates, shortcuts, shaders "coming soon"),
Integrations (Jellyfin "coming soon") and About. The update indicator sits on
Home. Settings persist in two Hive boxes: `connect_form` (last form and
recents; saved even when a join fails) and `settings`. Link media names are
derived from the URL path and never contain a query string.
Window chrome is frameless (`window_manager`): every screen brings its own
drag area and min/max/close buttons (`widgets/window_chrome_bar.dart`,
`window_buttons.dart`); the session drag layer is `window_drag_layer.dart`.
## Health reporting
Every failure must show in the UI with a cause and a next step; nothing
stalls silently. The app has a `HealthStatus` signal (severity + message)
for connecting, reconnecting, wrong password, version mismatch, kicked,
host offline, and starvation ("waiting for X - N.N s buffered").
Errors are logged with a stack trace at the boundary, then turned into a
displayable state. The server's `GET /health` returns JSON (`ok`, `room`,
`proto`, `viewers`, `bytesServed`, `uptimeSecs`); it is unauthenticated and
the app does not read it.
## Updater
`apps/desktop/lib/src/update/updater.dart` (no Flutter imports, unit tested).
Reads a manifest (`version`, `protocol`, `url`, optional `sha256`, `notes`)
from `updateManifestUrl`, compares versions, downloads the tarball, checks
sha256 when present, then extracts into a staging dir inside the install
folder (same filesystem, so moves are renames; the folder must be
user-writable). Each top-level entry is moved aside and the new one moved in,
with rollback on failure (writing the running executable fails with ETXTBSY;
rename works). A packaged install is detected by `data/` and `lib/` next to
the binary; anything else exits with "update manually". A manifest whose
protocol is newer than the local one is a required update. Windows apply
(`*.old` rename, swept on next start) exists but is untested. The version is
stamped with `--dart-define=APP_VERSION` (dev runs report `0.0.0-dev` and skip
checks). `watch_together --upgrade` (or the shipped `update.sh`) runs it from
the CLI; `UpdateController` runs the same code for Home's update indicator,
which opens the update card, and restarts the app ("Update and restart").
Settings has Check now and an auto-check toggle. The manifest is `updates/manifest.json` in
this repo, written by `tool/release.sh`.
## Decisions
- Desktop + mpv only, no web clients: exists because Jellyfin SyncPlay is bad.
- No transcoding: bandwidth is plentiful and the Jellyfin box should not be loaded.
- No roles, friends trust model: simplest thing that fits the use.
- Blank password = open server: the owner chooses between open and private.
- The host cannot be kicked: the server dies with the host anyway.
- Server ships inside the app (and, planned, as a separate compiled server):
a Flutter binary cannot run headless, so a dedicated server needs its own
build of `packages/server`.
- Server owns the clock; reports never move it: avoids ping-pong between clients.
- Quorum gate: a lone viewer must be able to watch; two or more must start together.
- Relic for the server, `web_socket_channel` for the client; packages are
chosen on merit, not locked.
- Plain `ws://`, no TLS: friends use LAN, VPN or a forwarded port; TLS
(e.g. a reverse proxy) is a documentation item.
- One session per server, id `default`: the map is keyed for later but unused.
- No server storage: config and logs only; media never touches the server disk.
- Avatars ride in the 1 Hz `room` broadcast (cap 256 KiB base64 per avatar):
simple, costs host uplink; hash-based serving is on the roadmap.
docs/DEVELOPMENT.md
+102
View File
@@ -0,0 +1,102 @@
# Development
Layout and rules: [ARCHITECTURE.md](ARCHITECTURE.md).
## Build
Needs the Flutter SDK (beta channel, Dart 3.13+) and, for the Linux desktop
build, the usual GTK toolchain (clang, cmake, ninja, pkg-config, libgtk-3-dev).
mpv libraries come from `media_kit_libs_video`.
```
cd apps/desktop && flutter pub get && flutter run -d linux
cd apps/desktop && flutter build linux --release --dart-define=APP_VERSION=<x.y.z>
tool/package_linux.sh [--skip-build] [--noupload] # dist/*.tar.gz
```
`APP_VERSION` defaults to `0.0.0-dev`; `package_linux.sh` stamps it from the
desktop pubspec. Setting `DISCORD_WEBHOOK_URL` makes the packager post the
tarball to Discord (never commit it).
## Dev loop
Run a server without the app, then point apps at it:
```
dart run packages/server/bin/server.dart --port 8080 --secret x [--media FILE]
```
Prints `READY port=... room=...` and stops on SIGINT. Then Join from one or
more apps at `127.0.0.1:8080`. Or just Host from the app. mpv env override:
`SYNC_RELAY_HWDEC` (wins over the Settings choice). `SYNC_RELAY_VO` has no
effect with embedded video: media_kit forces `vo=libmpv`.
Harnesses in `tool/` (own `pubspec.yaml`, `dart run tool/<name>.dart`):
`range_logger` (serve a file and log every Range request), `shaper`
(bandwidth cap, latency, `--kill-after` for outage tests), `fake_viewer`
and `e2e_check` (simulated viewers against a live server).
## Tests
- `cd packages/core && dart test`, `cd packages/server && dart test`.
- `cd apps/desktop && flutter test test/<file>`.
- `sync_playback_e2e_test.dart` and `coord_presence_test.dart` use real mpv;
the e2e one also expects `~/Videos/chiyo.mp4`. Run only tests related to
what you changed.
- CI (`.gitea/workflows/ci.yml`): analyze + test for core and server only.
Desktop tests are not in CI.
Traps:
- Never pump the session screen or media_kit `Video` in `flutter test`: the
runner hangs (kill the `flutter_tester` by PID). Test pure logic
(`RoomDiff`, coordinator without widgets) instead.
- Never depend on a server-side WebSocket `onDone`; assert presence with
deadline polling.
- `sub.asFuture()` can resolve before `onDone`; poll with a deadline.
- Tests must not assume a threat model beyond the friends trust basis.
## Releasing
User pushes tags; CI does the rest.
1. `tool/release.sh [patch|minor|major] [--dry-run]`: derives the next
version from the latest tag, bumps the three pubspecs in lockstep, writes
`updates/manifest.json` (asset URL is predictable), commits `Release vX.Y.Z`,
makes an annotated tag.
2. Review, then `git push --follow-tags`.
3. `.gitea/workflows/release.yml` builds the Linux tarball and runs
`tool/release_publish.sh vX.Y.Z`, which creates the release and uploads it.
By hand: `tool/package_linux.sh` then `tool/release_publish.sh vX.Y.Z`.
Tokens come from the environment (`RELEASE_TOKEN`, or Actions-provided
`GITHUB_TOKEN`/`GITEA_TOKEN`). The manifest has no `sha256` by default; the
updater skips the check when absent. Workflows live in `.gitea/workflows`
(the forge reads `.gitea` before `.github`).
## Gotchas
- Relic gives Range as a typed `RangeHeader`; convert from unit + first range.
- Relic server-side `onDone` does not fire for dead peers. Presence is the
`isClosed` sweep + `onError` forget + `lastSeen` timeouts.
- `web_socket_channel` 3.x: `channel.ready` errors are unhandled on refused
connections; swallow them and report through the stream's `onError`.
Streams are single-subscription: one listener per socket.
- media_kit has no public `Player.setProperty`; the code reaches mpv through a
dynamic call on `player.platform`. Cache size goes through
`PlayerConfiguration(bufferSize:)`.
- mpv reconnect flags: comma form works, colon form is rejected. Re-verify
through `setProperty` if touched.
- Overlays that are hidden but mounted keep focus (Space falls into a hidden
button). Use `ExcludeFocus` on closed subtrees and unfocus on close; reuse
the shared no-focus node for overlay buttons.
- `modifyVolumeOnScroll: false` on the session controls is load-bearing for
window drag (volume is the speaker button only). A `GestureDetector` below
media_kit's overlay loses the gesture arena; the drag layer is a raw pointer
observer on top.
- New screens must bring their own drag area and window buttons (frameless
window).
- Chat arrives twice from the server (see PROTOCOL.md); the client drops the
textless feed copy.
- Tab traversal is disabled on the session screen on purpose; Tab toggles the
side panel.
- After editing constants (ports, thresholds), grep to confirm the value landed.
docs/HOW-IT-WORKS.md
+76
View File
@@ -0,0 +1,76 @@
# How it works
A five-minute tour. Details: [ARCHITECTURE.md](ARCHITECTURE.md), wire format:
[PROTOCOL.md](PROTOCOL.md).
## The pieces
```
your PC friend's PC
+-------------------+ +-------------------+
| app (mpv inside) | | app (mpv inside) |
+---------+---------+ +---------+---------+
| sync (WebSocket) | sync (WebSocket)
v v
+----------------------------------------+
| server (who is ready, where are we) |
+----------------------------------------+
video bytes come from somewhere else:
- a link (Jellyfin, any http/https URL): each app fetches it directly
- a file on the host's disk: the server streams it (/m/... endpoint)
```
- **App**: the desktop window. It draws the video with mpv, shows who is
here, and sends what you do (ready, seek, chat).
- **Server**: keeps one shared clock and the list of people. It never sees
video frames, only "position 123 s, playing". There is exactly one server
and one watch session on it.
- **Jellyfin** (or any web host): where a link's video lives. The server does
not touch it today; each app downloads it itself.
## Ways to run
- **Host from the app**: the app you click Host in also runs the server.
Simplest. The server lives as long as that app.
- **Dedicated server**: a separate server program on an always-on box.
Planned, see [ROADMAP.md](ROADMAP.md).
Friends join either way with the Join card on Home.
## One session, start to finish
1. **Connect.** The app opens a WebSocket to the server and says hello
(version, password, name, avatar). Wrong password or version: refused with
a reason. Otherwise the server answers with a welcome and the current state.
2. **Catch up.** The app opens the current video, jumps to where the server
says we are, and buffers. A new arrival is not ready yet and never holds
anyone back.
3. **Ready gate.** Press Space to say you are ready. The server starts
playback only when everyone who is ready has at least 4 seconds buffered.
If two or more people are ready, one person alone cannot start it for the
others. A lone viewer can play alone.
4. **Play.** The server owns the clock. Each app compares its mpv position
to the clock: a small lead slows mpv a little (0.95x), a big gap (over 5 s)
jumps. If someone's buffer runs dry the server pauses everyone and resumes
when they have caught up.
5. **Seek.** Anyone can seek, anytime. The server moves the clock and
everybody follows. Seeking does not change anyone's ready flag.
6. **Swap.** Anyone can change the video by pasting another link. Everybody
stops at 0, ready flags clear, and the gate runs again.
7. **Leave.** Closing or leaving drops you from the server; the others keep
going. If you vanish without saying goodbye, you are marked stale after 15 s
(stops blocking) and removed after 30 s. The app reconnects by itself and
restores your ready flag.
There are no roles: anyone can seek, swap, force-ready others and kick
anyone except the host.
## Where video comes from
- **Link**: you paste an http(s) URL. Every app plays it directly with mpv.
Works for a Jellyfin direct-download link or any file on the web.
- **Local file (host only)**: you pick a file on your disk. The server
streams it to everyone, host included, over HTTP with range requests, each
with a per-person token. Nothing is stored or converted.
- No transcoding, ever. Quality is whatever the file is.
docs/PROTOCOL.md
+75 -102
View File
@@ -1,120 +1,93 @@
# PROTOCOL — v4 (Phase 3: sync engine + readiness)
# Protocol (proto = 4)
`proto = 4`. Bump history: v1 hello/state/event shapes; v2 `token` in
welcome; v3 `kicked`/`kick`; v4 readiness intents, room fan-out, feed,
public member ids (tokens never leave their owner — media URLs carry
only the requester's own token).
Wire format only. Behavior (gate, clock, timeouts) is in
[ARCHITECTURE.md](ARCHITECTURE.md). Source of truth:
`packages/core/lib/src/protocol.dart` and `packages/server/lib/src/room_hub.dart`.
## Transport vs readiness (the core split)
Any wire change bumps `RelayConstants.protoVersion` and edits this file in
the same commit. Changes are additive.
Room transport is PLAYING or HELD. Seeks move transport position only:
while PLAYING anyone seeks freely, everyone follows, catch-up buffering
is tolerated with no flag changes. Readiness gates only the
HELD→PLAYING transition. Starvation pauses transport without touching
flags; recovery auto-resumes on buffers, no Space needed.
## Transport
## Message catalog
- WebSocket at `/sync`, JSON text frames, each an object with a `t` field.
- The first client frame must be `hello` within 10 s. Server pings every 5 s.
- Match incoming frames by `t`, never by order: `welcome` can race `room`.
- Failures: server sends `{"t":"err","message":...}` and closes.
Messages: `bad hello: ...`, `proto mismatch: got X, want Y`, `wrong secret`,
`rejected: <reason>` (e.g. avatar too large), `bad message: ...` (a frame
with a non-finite number, e.g. NaN position), `bad swap media`,
`swap needs a url or the hosted file` (empty swap).
- `GET /health` returns JSON (`ok`, `room`, `proto`, `viewers`, `bytesServed`,
`uptimeSecs`; unauthenticated). `GET /m/<sid>/<token>/<name>`
serves the host's file (Range table in ARCHITECTURE.md).
Client→server intents (all post-hello text frames):
## Client to server
```
{"t":"ready","ready":true,"source":"manual|auto"} Space flip / idle timer
{"t":"force","target":"all|<memberId>","ready":false}
{"t":"seek","position":123.4} doSeek event, room jumps
{"t":"swap","media":{"name":..,"size":..,"duration":..,"url":..}}
{"t":"hello","proto":4,"secret":"","user":{"name":"","avatar":""},
"media":{"name":"","size":0,"duration":0,"url":null}}
{"t":"ready","ready":true,"source":"manual"} source: manual | auto
{"t":"force","target":"all"|"<memberId>","ready":true}
{"t":"seek","position":123.4}
{"t":"swap","media":{"name":"","size":0,"duration":0,"url":"..."}}
{"t":"report","position":123.1,"buffered":6.2,"eventId":41}
{"t":"chat","text":"…"}
{"t":"kick","target":"<memberId>","reason":"…"} v3, now by member id
{"t":"chat","text":"..."}
{"t":"kick","target":"<memberId>","reason":""}
```
Server→client:
Field notes:
- `hello.secret`: the server password; a server with an empty secret is open.
The member connecting from a loopback address is the host.
`hello.media` is carried but not used by the server today.
- `user.avatar`: base64 image, at most 256 KiB of characters. `name` is
trimmed, empty becomes `?`, cut to 32 characters.
- `chat.text` is cut to 500 characters.
- `swap.media.url` is the link; `name`, `size`, `duration` are informational.
- Frames before `welcome` (other than `hello`) are ignored; unparsable or
unknown frames are ignored silently.
## Server to client
```
{"t":"room",{eventId,transport:"playing"|"held",position,serverTime,
mediaId,media:{…}|null,
members:[{id,name,avatar,ready,inSet,buffered,stale}]}}
{"t":"feed",{actor,action,target?,reason?,at}} toasts + sheet log
{"t":"welcome",{token,memberId,mediaUrl?,state}} v2+4, state now a room
{"t":"kicked",{reason}} {"t":"err",{message}}
{"t":"welcome","token":"...","memberId":"m1","mediaUrl":"...","state":{<room>}}
{"t":"room","eventId":7,"transport":"playing"|"held","position":12.3,
"serverTime":"<ISO-8601 UTC>","mediaId":1,
"media":{name,size,duration,url?}|absent,
"members":[{"id","name","avatar","ready","inSet","buffered","stale"}]}
{"t":"feed","actor":"","action":"","target":"","reason":"","at":"<ISO-8601>"}
{"t":"chat","text":"...","from":"<name>"}
{"t":"kicked","reason":""}
{"t":"err","message":""}
```
`room` broadcasts at 1 Hz and on every change. Clients diff by
`eventId`: position jump ⇒ seek, transport change ⇒ play/pause.
`feed` actions: ready, unready, force-ready, force-unready, force-play,
force-hold, kick, join, leave, stale, swap, seek, pause, resume, chat —
every one carries actor + server timestamp.
Force-play-all is `force` with `target:"all",ready:true`; anyone not
ready Spaces back. Sounds key off feed actions.
Chat dual delivery: the server emits both `{"t":"chat","text","from"}`
(render this one — it carries the text) and a textless `feed` chat
entry (housekeeping echo). Clients must render exactly one row per
send; the reference client drops the textless feed entry.
- `welcome.token` is private to its owner and appears only in that member's
media URL. `mediaUrl` is omitted when there is no media; it is a built
`/m/...` URL for a host file (host part = what the client dialled), otherwise
the shared link.
`welcome.state` has the same shape as `room`.
- `room` is sent every second and on every change. `position` is the value at
`serverTime`; clients extrapolate while `transport` is `playing`.
`eventId` increases on seek, swap and transport changes.
`mediaId` increases on swap.
- `feed.action` values: `ready`, `unready`, `force-ready`, `force-unready`,
`force-play`, `force-hold`, `kick`, `join`, `leave`, `stale`, `swap`,
`seek`, `pause`, `resume`, `chat`. `target` and `reason` are optional;
`reason` carries `auto`/`manual` for ready and `silent too long` or
`no signal` for leave/stale.
- Chat is delivered twice: a `chat` frame with the text and a textless
`feed` with `action:"chat"`. Render the `chat` frame only; the reference
client drops the textless feed entry.
## Membership set (kills join-cascade + ghost-quorum)
## Join and reconnect
- Join: `inSet=false, ready=false`. Catch up in background (open media,
seek to room edge, buffer). Flipping ready joins the set.
- Unready while in set: stays in set, room re-evaluates → HELD.
- Disconnect: removed from set, never counts toward quorum.
- Swap: all sets cleared, all flags false.
- Gate to PLAYING: set non-empty AND every in-set member
ready + buffered ≥ 4 s + fresh (not stale) AND quorum: in-set
size ≥ 2, unless no more than one member is fresh. A lone
watcher can play alone; the moment a second watcher is present,
one Space waits for the other. Empty set ⇒ HELD.
`hello` then `welcome`, then the app opens `mediaUrl`, seeks to the room
position and buffers. A joining member is `ready=false, inSet=false` until
the first `ready`. A socket that dies loses its token; reconnect is a fresh
`hello`, after which the client re-sends its prior ready flag.
## Clock rules (Syncplay port)
## Reserved, not implemented
Server owns the advancing clock:
`positionAt(now) = position + (playing ? now-serverTime : 0)`.
Plain reports are informational and can never move it. Only seek, swap,
and gate transitions write position/serverTime/eventId. Reports tagged
with `eventId < current` have their position ignored (no ping-pong);
buffered is still accepted. Pause (HELD) freezes position immediately;
resume sets serverTime=now — laggards seek forward, never drag back.
Tiered client correction on room state: |drift| > 5 s ⇒ seek;
ahead > 1.5 s ⇒ 0.95× until level; inside deadzone ⇒ nothing.
## Join state machine
connect → hello → welcome → open media (server mediaUrl or shared
url) → seek to room edge → buffer → not-ready catch-up → Space →
in set → HELD→PLAYING when gate passes. Reconnect: re-hello (tokens
die with the socket), prior flag restored, buffer gate holds playback
until truly caught up.
## Swap state machine
anyone sends swap → HELD, mediaId++, clock reset (0/now), sets
cleared, flags false → all clients re-run Join → auto-resume when the
gate passes. No authority check in v1 (same trust basis as force).
## Join / auth (v2–v3, unchanged)
proto must equal server proto; secret is the room password; avatar
≤64 KiB; failures get `err` + close. `sid="default"`, map keyed for
later multi-session. `mediaUrl` carries the playable URL: a built
`/m/…` URL when the server holds bytes, otherwise the shared link.
## Media endpoint (Phase 1, unchanged)
`GET /m/<sid>/<token>/<name>` Range table. 410/401/403/503 gates.
## Heartbeat (Phase 2 baseline, feeds the gate)
5 s WS pings; per-token lastSeen on every frame; silent >15 s ⇒ out
of the gate (`stale` feed); silent >30 s ⇒ socket closed, member
removed (`leave`), client reconnects via re-hello. No limbo sockets.
Closed sockets are swept every second (`isClosed` poll — transport
`onDone` is unreliable, so presence never depends on it).
Client-side reconnect loop (fixed polite interval + manual escape)
is app work.
## Playlist (deferred, space reserved)
Not v1. Shape when it lands: server-held `queue:[media…]` alongside
the current media; anyone appends links (`queue-add`), anyone skips
(`queue-skip`); reaching EOF auto-advances exactly like a swap
(HELD, mediaId++, sets cleared, Join rerun, gate resumes). No code
until then — `mediaId` + swap already carry it.
Server-held queue (`queue-add`, `queue-skip`) would auto-advance like a swap.
`RelayConstants.chunkBytes`, `creditWindowChunks`, `maxConcurrentOpen` and the
`open`, `ack`, `cancel`, `eof`, `buffered`, `state`, `event` kinds in
`MessageKind` are unused leftovers.
docs/ROADMAP.md
+87 -77
View File
@@ -1,87 +1,97 @@
# ROADMAP — sync_relay_project (backlog tracker)
# Roadmap
This is the durable source of truth for what is built and what is next.
Chat is ephemeral; this file + `HANDOFF.md` (root) + `PROTOCOL.md` /
`ARCHITECTURE.md` carry state across sessions. Check an item off when it
lands, same commit as the code. Proto bumps still require PROTOCOL.md
same-commit.
Backlog and epics, in order. Done work lives in git history. Check an item
off by deleting it in the same commit as the code. Wire changes also edit
[PROTOCOL.md](PROTOCOL.md).
Model: direct-serve (host opens ports, clients open nothing). The
brief's old WSS host-upload tunnel is obsolete unless a VPS tier
needs remote-host upload. Server = Relic, client WS =
web_socket_channel. Protocol still v4.
## Backlog
## Built (phases 0–4 + hardening — done, details in git history)
1. **Isolate server + dedicated server.** Replace host detection by loopback
peer (breaks behind a same-machine reverse proxy or tunnel) with an explicit
host token. Run the server on a Dart isolate
when hosting from the app; build the dedicated server as a separate
`dart compile exe` of `packages/server` (a Flutter binary cannot be
headless); drop `--headless`; rename "room" to "server" in code
(`RoomServer`, `RoomHub`, `RoomBroadcast`, wire `t:'room'`, `room` in
`/health`, config defaults) with the proto bump that implies; refresh
`deploy/` and the packaging.
Scaffold + CI, mpv Range contract, sessions/tokens/media endpoint,
sync engine + quorum, desktop shell (SyncClient, RoomDiff, media_kit,
avatars, toasts+sheet, reconnect), presence hardening, frameless
chrome, chat/idle fixes, chat double-row fix, avatar 256 KiB cap,
Linux tarball packager + Discord upload card, updater v1
(`--upgrade`: manifest check, download, sha256, apply-over).
WAN-tested 2-client. Counts at last green: core 42, server 41,
desktop 17 + 7 updater.
2. **Sync fixes.**
- Jumpy seek. Root cause: the media_kit seek bar calls `player.seek`
continuously while dragging; the coordinator only notices through its
1 s position poll (`_detectUserSeek`), so it fights the server echo and
the seek-landed retry (`_checkSeekLanded`).
- End-of-file stall. The 4 s buffered gate cannot be met in the last 4 s of
a file, the server never learns the duration, and there is no finished
state. Add duration reporting and a finished state.
- Tiny read-ahead. Our own pins (`cache-secs=10`,
`demuxer-readahead-secs=10` in `playback_controller.dart`) cap it; want
about two minutes for large files.
- Remote clock offset is not corrected: clients assume their clock equals
the server's.
- Stale-ready. A ready member who went stale (no reports for 15 s) should
return to the ready set when reports resume; the client re-sends ready as
a workaround (remove it then).
- Ghost member after an unclean drop until `dropAfter`.
## Backlog (priority order, each one sitting-sized)
3. **Rendering, then shaders.** media_kit falls back to software rendering
on the dev machine. Suspect Impeller on Linux against media_kit's GL
texture path; investigate first, since shaders need hardware rendering and
it may also cause the video freeze. Then shaders: per-viewer local setting
using mpv `glsl-shaders`, shipping Anime4K presets (bloc97, MIT).
1. First real release: `tool/release.sh` (derives the version,
writes the manifest, no typing) → review →
`git push --follow-tags` → CI (`release.yml`) builds the
tarball and publishes the Forgejo release. Verifies the whole
`--upgrade` chain end to end. Manual fallback is
`tool/package_linux.sh` + `tool/release_publish.sh <tag>`.
2. Jumpy seek: live repro with timeline log first, then fix the
seek-land retry vs deadband/debounce fight.
3. EOF stall (bug, dogfood 09-12): playback stops just before the
end and gates HELD ("waiting for X (3.9s)"). Suspect the
buffered≥4s readiness gate is unsatisfiable near end of media.
4. Tiny buffer (bug, dogfood 09-12): 1–3 s observed; want ~2 min
for a 1.4 GB / 24 min file (intro skip + surviving short
drops). Investigate demuxer forward-queue props + 64 MB
`bufferSize`, then raise.
5. Recovery controls: reload-current-media button + restart-player
button for when playback wedges.
6. Link conveniences: Ctrl+V pastes clipboard into the media
field; session link swaps update the remembered media (today
only the connect screen persists); session shows the open link
with an open-in-browser button.
7. Time UI: remaining time until end + scrubber preview (target
time while scrubbing).
8. Shortcut parity pass with mpv / mpv.net.
9. Sounds swap-in: replace `NoopSfxService` (flutter_soloud preload,
Kenney/ObsydianX/akx sources in HANDOFF §7); keep seek/pause/
resume cues muted or near-silent.
10. `/health` auth + stop broadcasting the full media URL (api_key
leaks onto the wire).
11. File-join wart: welcome mediaUrl advertises 127.0.0.1, remote file
joiners fail-open then self-heal on first room broadcast.
12. Cache-exhaustion long outage via `tool/shaper.dart --kill-after`.
13. Phase 5 chaos suite (host drop 5 s / 30 s, viewer stall, 20-seek
storm) + OOM / stuck-room / drift-recovery assertions.
14. Windows zip in packaging + Discord card.
15. TLS/Caddy doc.
16. Chat history pane (schema exists, Activity log only for now).
17. Shaders: anime4k upscaling via mpv glsl-shaders + UI toggle.
18. Avatar caching + static serving (worth-it decision open):
serve avatar bytes once over HTTP by hash instead of
base64-in-every-room-broadcast.
19. Backfill LAN soak + 1.4 GB test details (record-only, user ran
both 09-12).
4. **Windows build + CI.** There is no Windows runner. Try
`ubuntu2610/flutter_build` (LLVM-MinGW + Wine `gen_snapshot`) on the Linux
runner, fallback a GitHub Actions mirror. The manifest needs per-platform
URLs and `release_publish.sh` must handle a second platform. The updater's
Windows apply path exists but is untested. Desktop tests are not in CI
either.
## Epics (multi-sitting, scheduled whole — not before the backlog
friction items above, except Jellyfin which is next feature)
5. **Jellyfin** (epic, below).
- File relay (original brief, second half): links shipped, relay
unbuilt. Split into sub-tasks when scheduled.
- Jellyfin native — NEXT feature.
- Updater expansion: in-app prompt, Windows helper (locked-exe
dance), auto-restart after apply.
- Playlist (`queue-add`/`queue-skip`, shape reserved in PROTOCOL.md),
sidecar subtitles, local-file fallback (viewer plays matching
size+duration file locally, still synced), `HttpSource` (host
pastes a direct-stream URL, server fetches + re-serves).
6. **Smaller items**
- Link conveniences: Ctrl+V pastes into the media field; a link swap in
session updates the remembered media; show the open link with an
open-in-browser button.
- Time UI: remaining time, scrubber preview.
- mpv / mpv.net shortcut parity.
- Sounds: replace `NoopSfxService` (cues already fire). Candidates:
Kenney Interface Sounds (CC0), ObsydianX Pack 1 (CC0), akx/Notifications
(CC0); playback via flutter_soloud. Keep seek/pause/resume near silent.
- Chaos suite: host drop 5 s / 30 s, viewer stall, 20-seek storm, long
outage with cache exhaustion (`tool/shaper.dart --kill-after`); assert no
OOM, no stuck session, drift recovers.
- Chat history pane (today: activity log only).
- Avatar caching: serve avatar bytes once by hash instead of base64 in
every `room` broadcast.
- The `room` broadcast carries the full media URL, which can include an
`api_key`.
- TLS: document a reverse proxy in front of the server.
- Playlist: `queue-add`, `queue-skip`, auto-advance like a swap.
- Sidecar subtitles.
## Deferred (needs explicit approval; transcode is never)
## Epic: Jellyfin
P2P/WebRTC, multi-session, Android. Transcode: NEVER (user decision
09-12 — not the age for it anymore).
- Each viewer logs in to Jellyfin themselves (Quick Connect or password).
- Several Jellyfin servers can be linked, keyed by `ServerId`.
- The sync server shares only `{ServerId, ItemId}`. Pasted Jellyfin links are
recognised and reduced to that pair.
- Viewers with an account direct-play `/Videos/{id}/stream?static=true`
with their own token and report Playing / Progress / Stopped, so watched
state and Continue Watching update per account.
- Guests without an account play through the sync server, which fetches with a
credential that stays server-side (the host's own or a configured one).
- Next episode: the successor in `/Shows/{seriesId}/Episodes` (NextUp means
first unwatched, not next-after-X); audio and subtitle track matching by language, then title,
then codec/channels.
- Library browser; anyone can put up items.
Spike facts (Jellyfin 10.11.11): `/Videos/{id}/stream?static=true&mediaSourceId=`
serves the original with byte ranges (206). PlaybackInfo gives `PlaySessionId`
and DirectPlay. All audio and subtitle tracks are embedded in the mkv. Quick
Connect is enabled but `/QuickConnect/Initiate` returned 400: investigate
(likely missing client/device fields in the auth header).
## Deferred
P2P, Android. Never: transcoding, web clients.
docs/mpv-http-contract.md
-70
View File
@@ -1,70 +0,0 @@
# mpv HTTP contract (Phase 1, verified against Relic endpoint)
Spike: `tool/range_logger.dart` (Relic, `mountMediaRoute` + `onLog` hook).
Player: stock mpv CLI headless (`--vo=null --ao=null`). Server logs
`METHOD path range=<h> -> status bytes=<n>`.
## Real traces
mp4, 570692 B, full play (`--frames=300`):
```text
get /m/file range=- -> 200 bytes=570692
get /m/file range=bytes=0- -> 206 bytes=570692
```
One no-Range open (mpv probes full length), then a single open-ended
stream to EOF. Small file fits demuxer cache; one connection does it all.
mkv (ffmpeg remux of same file), 562197 B, play + `--start=8` seek:
```text
get /m/file range=bytes=0- -> 206 bytes=562197
get /m/file range=bytes=0- -> 206 bytes=562197
get /m/file range=bytes=562093- -> 206 bytes=104
get /m/file range=bytes=243694- -> 206 bytes=318503
```
Two `bytes=0-` opens (demuxer probe + playback), a 104 B tail read at
`size-104` (Cues-at-EOF probe — mpv expresses it open-ended, not as
`bytes=-n`; the server answers both identically), then a mid-file open
at a cluster boundary for the seek/start position. Seeks always arrive
as a fresh `bytes=N-` connection, never as a mutation of the old one.
Outage: server SIGINT-killed 5 s mid-play (looping file, comma-form
reconnect flags), restarted. mpv kept playing from cache, no error, no
reconnect storm; no new Range request inside the post-restart window.
Short drops are ridden out from cache — matches the north-star case.
Cache-exhaustion reconnect (long drop) is a Phase 5 chaos assertion;
`tool/shaper.dart` (KB/s cap + latency + `--kill-after`) exists for it.
## Server behavior table (all byte-exact tested, 16 server tests)
| Request | Response |
|---|---|
| no Range | 200, exact `Content-Length`, full bytes |
| `bytes=a-b` | 206, `Content-Range: a-b/size`, exact length |
| `bytes=a-` | 206 open-ended to EOF, exact length |
| `bytes=-n` | 206 tail `size-n..EOF` (Cues probe) |
| unsatisfiable start | 416 + `Content-Range: bytes */size` |
| host offline | 503 + `Retry-After: 2` (mpv retries with reconnect flags) |
| bad token / kicked / gone | 401 / 403 / 410 |
Always: `Accept-Ranges: bytes`; `Content-Type` from extension
(mkv `video/x-matroska`, mp4 `video/mp4`, webm, avi, mov, ts, mka/mp3).
## Relic notes
- Range arrives as typed `RangeHeader` (`req.headers.range`), not a raw
string. Multi-range takes the first entry; unit is preserved so
non-`bytes` units still 416. `resolveMediaRequest` in
`packages/server` stays framework-agnostic and is the tested truth.
- Reconnect flag syntax is build-sensitive: this mpv rejected the
colon-joined `--stream-lavf-o=a=1:b=1` form and accepted commas.
The exact media_kit property set is re-verified in Phase 4 through
mpv `set_property`, not CLI flags.
## Remaining manual (human + media_kit, Phase 4)
1.4 GB real file through the endpoint with the pinned buffer props;
long-outage reconnect with cache exhaustion via shaper.