Docs rewrite: current-state docs, HANDOFF retired
README for the repo page; HOW-IT-WORKS overview; ARCHITECTURE, PROTOCOL, DEVELOPMENT checked against the code; ROADMAP is backlog only (isolate/dedicated server, sync fixes, rendering+shaders, Windows build, Jellyfin epic). HANDOFF.md and mpv-http-contract.md folded in and deleted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
foxyzandClaude Opus 5.5 committed
1 parent
aeff087c7d
commit
d8a0729029
9 files changed
+680
-687
No files matched your search
@@ -1,35 +1,42 @@
|
||||
# AGENTS.md — sync_relay_project
|
||||
# AGENTS.md
|
||||
|
||||
Project conventions for agents. Global interaction rules live in the
|
||||
user-level AGENTS.md; this file holds repo-specific facts only.
|
||||
Source of truth order: codebase > `HANDOFF.md` + `docs/ROADMAP.md` >
|
||||
chat history.
|
||||
Repo-specific rules. Global interaction rules live in the user-level
|
||||
AGENTS.md. Source of truth: the code, then the docs below.
|
||||
|
||||
## Docs
|
||||
|
||||
- [README.md](README.md): users. [docs/HOW-IT-WORKS.md](docs/HOW-IT-WORKS.md):
|
||||
plain tour. [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md): packages, sync,
|
||||
media, updater, decisions. [docs/PROTOCOL.md](docs/PROTOCOL.md): wire format.
|
||||
[docs/DEVELOPMENT.md](docs/DEVELOPMENT.md): build, test, release, gotchas.
|
||||
[docs/ROADMAP.md](docs/ROADMAP.md): backlog and epics.
|
||||
- Docs describe current truth only: no dates, no "fixed" notes, no phase
|
||||
logs (git has history, plans go in ROADMAP).
|
||||
- Each fact lives in exactly one file; others link to it.
|
||||
- Code and its docs change in the same commit. A wire change also bumps
|
||||
`RelayConstants.protoVersion` and edits PROTOCOL.md.
|
||||
- User-facing wording says "server" (there is only one), never "room".
|
||||
Code identifiers and wire names keep their real names until the rename
|
||||
in ROADMAP item 1.
|
||||
|
||||
## Wishlist drafts
|
||||
|
||||
The user drops raw idea/bug lists as drafts (gitignored, never
|
||||
committed: `docs/Wishlist.md`, `docs/drafts/`). Agent duty on sight:
|
||||
translate each entry into its proper place — `docs/ROADMAP.md`
|
||||
backlog (ordered, one sitting-sized item each) or Epics, real bugs
|
||||
with repro detail preserved — then confirm placement with the user
|
||||
and delete the draft. Drafts must never be committed, even partially.
|
||||
The user drops raw idea/bug lists as drafts (gitignored, never committed:
|
||||
`docs/Wishlist.md`, `docs/drafts/`). On sight, translate each entry into its
|
||||
place in `docs/ROADMAP.md` (ordered backlog item or epic; keep repro detail
|
||||
for real bugs), confirm placement with the user, then delete the draft.
|
||||
Drafts must never be committed, even partially.
|
||||
|
||||
## Commits
|
||||
|
||||
Few commits, one per topic/session — never per fix. Rewriting
|
||||
unpushed history (amend, squash) is expected, not exceptional.
|
||||
Commit code + its docs together. No `flutter analyze` unless asked
|
||||
or diagnosing; scoped `dart analyze` on touched files is the
|
||||
pre-commit check. Tests run only when the code under test changed.
|
||||
Few commits, one per topic/session, never per fix. Rewriting unpushed
|
||||
history (amend, squash) is normal. Commit code + its docs together. No
|
||||
`flutter analyze` unless asked or diagnosing; scoped `dart analyze` on touched
|
||||
files is the pre-commit check. Run tests only when the code under test changed.
|
||||
|
||||
## Releases (user pushes tags, CI does the rest)
|
||||
## Releases
|
||||
|
||||
Release flow: `tool/release.sh [patch|minor|major]` derives the
|
||||
version (no typing), bumps all pubspecs, commits, tags → review →
|
||||
`git push --follow-tags` → `.gitea/workflows/release.yml`
|
||||
builds and publishes the Forgejo release (manifest already written
|
||||
by release.sh — predictable asset URL, no CI roundtrip). `tool/release_publish.sh` is the same
|
||||
flow by hand. Workflows live in `.gitea/workflows` (this forge reads
|
||||
`.gitea` first, `.github` is ignored when it exists).
|
||||
Never commit secrets: tokens arrive via env (`DISCORD_WEBHOOK_URL`,
|
||||
`RELEASE_TOKEN`) or Actions-provided (`GITHUB_TOKEN/GITEA_TOKEN`).
|
||||
`tool/release.sh [patch|minor|major]` then `git push --follow-tags`; CI
|
||||
publishes. Details: [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md). Never commit
|
||||
secrets: tokens come from env (`DISCORD_WEBHOOK_URL`, `RELEASE_TOKEN`) or
|
||||
Actions (`GITHUB_TOKEN`/`GITEA_TOKEN`).
|
||||
-323
@@ -1,323 +0,0 @@
|
||||
# HANDOFF — sync_relay_project (watch-together app)
|
||||
|
||||
Date: 2026-09-12 (updated late session). Phases 0–4 + presence hardening
|
||||
+ UI redesigns done, all green (core 42, server 41, desktop 24).
|
||||
Phases retired as a tracker 09-12 — `docs/ROADMAP.md` is now a flat
|
||||
backlog (Built / Backlog / Epics / Deferred); phase text lives in
|
||||
git history.
|
||||
Black screen (§8) DIAGNOSED + FIXED (avatar-texture teardown race;
|
||||
Visibility keep-mounted) + VERIFIED, incl. over internet. Bisect
|
||||
screens (legacy/minimal/nosheet) REMOVED 09-12 — app.dart routes
|
||||
session straight to SessionScreen. Internet test PASSED (WAN port
|
||||
6969, 09-12); user reports 2-client LAN soak + 1.4 GB real-file
|
||||
test also done 09-12 (outcomes not yet recorded below). Tonight's
|
||||
fixes: server logging wired into app (`initLogging` + hello logs),
|
||||
Cancel aborts pending connect, first-join failure no longer
|
||||
retry-loops. Git repo INITIALIZED (commit 427d851, root
|
||||
.gitignore). Prior chat may be unreliable; this file plus `docs/`
|
||||
is the full state (`docs/ROADMAP.md` is now the phase tracker).
|
||||
|
||||
## 1. What this is
|
||||
|
||||
Private watch-together desktop app (Windows + Linux). Host serves a file
|
||||
or shared link from their own machine; 4–8 friends watch in sync.
|
||||
One binary, three modes: Client (Join), Server+Client (Host, opens
|
||||
ports, self-plays via loopback), `--headless` (server only, no window).
|
||||
North star: <1.5 s drift, seeks settle ~2 s, cache rides short drops,
|
||||
kick kills sync+video, no relay memory growth.
|
||||
|
||||
## 2. Locked architecture (do not reopen without the user)
|
||||
|
||||
- `packages/core`: protocol, sync engine, media abstractions. ZERO Flutter
|
||||
imports (enforced by test). `packages/server`: Relic-based room + media
|
||||
server, also zero Flutter imports. `apps/desktop`: the ONE Flutter
|
||||
binary embedding both. Direct-serve model: host opens ports, clients
|
||||
open nothing. NO host-upload tunnel (the brief's WSS chunk protocol is
|
||||
obsolete unless a VPS tier ever needs remote-host upload).
|
||||
- Server HTTP+WS: **Relic ^1.2.0**. Client WS: **web_socket_channel**.
|
||||
Server lib also uses **web_socket** for event types.
|
||||
- PACKAGES ARE NOT LOCKED (user decision 09-11, overrides the old brief
|
||||
list): pick on merit, no churn where the stack works. Current desktop
|
||||
additions: media_kit_video + media_kit_libs_video (required),
|
||||
file_picker (avatar/media picking), hive_ce + hive_ce_flutter
|
||||
(connect-form persistence), image (avatar shrink). signals + media_kit
|
||||
+ Relic stay (load-bearing).
|
||||
- Sync: heavily Syncplay-inspired + original readiness design, NOT 1:1.
|
||||
Reference clone (scratch, may be gone — re-clone shallow):
|
||||
`git clone --depth 1 https://github.com/Syncplay/syncplay.git /tmp/opencode/syncplay-ref`
|
||||
Audit already done; conscious deviations: clock never moves from
|
||||
reports (theirs follows slowest watcher), no pause-on-leave,
|
||||
force-for-all instead of operator-only setOthersReadiness, and
|
||||
QUORUM gate (09-12, user found solo-play meaningless): HELD→PLAYING
|
||||
needs in-set size ≥ 2 unless ≤1 member is fresh — lone watcher plays
|
||||
alone, second watcher present ⇒ one Space waits. Late joiners outside
|
||||
the set never block.
|
||||
- Single room password (one secret, persisted in Hive on the user's own
|
||||
box — accepted). No op roles (deferred: op flag + second hello
|
||||
secret). Kick = transient nudge: revoke + close + `kicked` reason,
|
||||
instant rejoin with fresh token.
|
||||
- Deferred, do not build: TLS/Caddy, Jellyfin native, updater
|
||||
expansion (v1 `--upgrade` SHIPPED 09-12 — see §9.12; in-app
|
||||
prompt + Windows helper still deferred), playlist (extension shape reserved in PROTOCOL.md), op roles, sounds
|
||||
(researched, wired, no assets — see §7).
|
||||
- Standing rules: health reporting (every failure displays, nothing
|
||||
stalls silently); proto bump + PROTOCOL.md same-commit on ANY
|
||||
wire change (still v4 — quorum/timeouts are server policy, no bump).
|
||||
|
||||
## 3. Repo map (root: Flutter/sync_relay_project/)
|
||||
|
||||
- `packages/core/lib/src/`: constants.dart (tuning: seek 300 ms,
|
||||
drift 5 s, slow 1.5 s @0.95, min_buffer 4 s, heartbeat 1 s, ping 5 s,
|
||||
avatar 256 KiB (base64 chars; wide headroom over any 128x128 PNG
|
||||
— raised 09-12 from 64 KiB, which rejected normal PNGs), name 32, chat 500; dropAfter lives as RoomEngine ctor
|
||||
default, now 30 s), errors.dart, logging.dart, config.dart,
|
||||
protocol.dart (v4 schemas), media_source.dart, sync_engine.dart
|
||||
(RoomEngine + quorum in evaluate's HELD branch; PLAYING branch
|
||||
unchanged).
|
||||
- `packages/server/lib/src/`: session.dart, media_endpoint.dart,
|
||||
room_hub.dart (forgetSocket, sweepClosedSockets via `isClosed` poll,
|
||||
closeAllSockets, onError→forgetSocket), room_server.dart (1 Hz
|
||||
heartbeat: sweepStale + sweepClosedSockets + dropSilentSockets +
|
||||
broadcast; stop() closes client sockets first). `bin/server.dart`.
|
||||
- `apps/desktop/lib/src/`: sync/ (sync_client.dart: hello/intents/
|
||||
fan-out/match-by-`t`; room_diff.dart: diff + debounce + play gate),
|
||||
playback/ (media_kit Player, 64 MB cache, pin props, 0.5 s seek
|
||||
deadband, stop() on leave, SYNC_RELAY_VO/HWDEC env overrides),
|
||||
session/ (session_coordinator.dart: THE brain — room→player,
|
||||
reports, scrub/nudge→seek intents incl. HELD, seek-land retry,
|
||||
idle 8/10 min, reconnect loop, 5 s silence timeout; health.dart),
|
||||
sfx/ (SfxKind/Cue/Service + NoopSfxService — sounds deferred),
|
||||
state/ (session_state.dart: route/mode/lastForm/prefs/host/join/
|
||||
leave; connect_prefs.dart), screens/ (welcome/connect/session),
|
||||
widgets/ (user_section.dart: ConnectProfileCard; member_avatar.dart:
|
||||
THE avatar — memoized, no bg on images, dark ring+fallback;
|
||||
dialogs.dart: kick confirm; media_input.dart: parseMediaInput).
|
||||
- `apps/desktop/test/`: room_diff_test (9), media_input_test (6),
|
||||
updater_test (7: version/manifest/layout + stub-server flows),
|
||||
sync_playback_e2e_test (quorum: solo Space holds, both ready plays;
|
||||
needs mpv + ~/Videos/chiyo.mp4), coord_presence_test (dead server
|
||||
clears + reconnects; needs mpv).
|
||||
- `tool/`: range_logger, shaper (chaos: KB/s cap, latency,
|
||||
--kill-after), fake_viewer, e2e_check (sim timeline, still ALL PASS
|
||||
post-quorum), package_linux.sh (release tarball + Discord card,
|
||||
stamps APP_VERSION).
|
||||
- `docs/`: PROTOCOL.md (v4 + quorum + 15/30 s timeouts + isClosed note,
|
||||
normative), ARCHITECTURE.md (+health rule), mpv-http-contract.md.
|
||||
- `deploy/`: systemd unit. Git repo initialized 09-12 (`git init`,
|
||||
root `.gitignore`, initial commit 427d851 covering phases 0–4 +
|
||||
tonight's connect/logging fixes). Bisection is possible again.
|
||||
|
||||
## 4. Protocol v4 + policy (see PROTOCOL.md for normative detail)
|
||||
|
||||
hello→welcome(token, memberId, mediaUrl?, room-state); intents:
|
||||
ready{ready,source}/force{target:all|memberId,ready}/seek/seek-swap/
|
||||
report{position,buffered,eventId}/chat/kick{memberId}. Broadcasts:
|
||||
room @1 Hz + on change (diff by eventId), feed{actor,action,…}
|
||||
(toasts/log/sounds), kicked{reason}, err. Media:
|
||||
GET /m/<sid>/<token>/<name> (200/206×3 shapes/416/401/403/410/503).
|
||||
Transport PLAYING/HELD; gate = non-empty set AND all in-set ready +
|
||||
buffered≥4 s + fresh AND quorum (in-set≥2 unless ≤1 fresh).
|
||||
Stale >15 s ⇒ out of gate; silent >30 s ⇒ closed + removed.
|
||||
Closed sockets swept every second; socket errors remove immediately.
|
||||
Client: 5 s room-silence ⇒ dead (clear room, warn, reconnect loop
|
||||
1.5 s polite infinite + manual cancel/retry, prior-flag restore).
|
||||
|
||||
## 5. Phase log (all exits met, all green as stated above)
|
||||
|
||||
- P0 scaffold + CI + Flutter-free tests; binary binds/logs/exits.
|
||||
- P1 Range table byte-exact, real mpv traces, shaper built. 503 tested.
|
||||
- P2 sessions: secret→128-bit token, 401/403/410 gates, kick kills WS+
|
||||
media, byte counters, fake_viewer ALL PASS live, proto→3.
|
||||
- P3 engine + readiness: engine tests (fake clock), e2e 15/15 sim,
|
||||
Syncplay audit. Sanity: NaN guards, caps, stale/leave feeds,
|
||||
1 Hz flush, token redaction.
|
||||
- P4 desktop shell→app: SyncClient framing, RoomDiff (eventId diff,
|
||||
slowdown latch 0.95×/0.1 reset, seek-land check), media_kit wiring
|
||||
(pinned buffer props, buffered from demuxer), Space=ready, avatars,
|
||||
toasts+sheet, health display, reconnect loop. Sounds WIRED ONLY
|
||||
(NoopSfxService) — see §7.
|
||||
- P5 presence+sync hardening (user-reported, all reproduced live
|
||||
first): stop() closes sockets; isClosed sweep (~1 s graceful);
|
||||
onError forget (~2–4 s kill -9, was 56 s ghost); drop 60→30 s;
|
||||
client silence-timeout + stuck-reconnect-flag fix + ready-future
|
||||
leak fix; seek race fixed (2 s same-event debounce both
|
||||
transports, edge-aware detector 1 s guard, 0.5 s apply deadband);
|
||||
HELD seeks send intents (paused frame moves for all); arrows
|
||||
nudge ∓10 s; local play gate (in-set+ready); autoplay off;
|
||||
leave() stops player (was leaking audio); kick confirm; Cancel
|
||||
connect; connect-form memory→Hive persistence; file/media
|
||||
Browse buttons.
|
||||
- P6 UI redesigns (two subagents, disjoint files — repeat this
|
||||
pattern): full-bleed video + overlays, controls never autohide
|
||||
(24 h hover) + Ready/Mark-everyone-ready IN the controls bar
|
||||
(intents only, zero local transport buttons), center avatars-only
|
||||
64 px, slide-in toasts, animated right sheet (users/log/chat/swap/
|
||||
Leave w/ confirm), health pill only-when-unwell, ESC closes/
|
||||
asks-leave, global focus-proof keys (Space/Tab/arrows/ESC/F,
|
||||
350 ms throttle, ignored while typing; Tab traversal disabled on
|
||||
the session screen 09-12 — game-style, Tab toggles the sheet), single media field
|
||||
(parseMediaInput + tests), avatar pipeline (128 px PNG, keeps
|
||||
transparency — JPEG did not; self-check measures encoded base64
|
||||
against RelayConstants, single source of truth), tap to
|
||||
pick, no path fields), Discord-like corner profile (connect only;
|
||||
welcome is title+Host/Join; edits apply next join — no rename
|
||||
intent on the wire), dialogs for terminal states.
|
||||
- UX decisions locked along the way: Space flips self-ready only;
|
||||
free seeks any transport (no flag churn); join catch-up never
|
||||
halts; idle 8-min warn + 10-min auto-unready; reconnect = re-hello
|
||||
with flag restore; kick → dialog + connect screen; fixed 1.5 s
|
||||
reconnect + cancel/retry; GMod toasts (social only) + side sheet;
|
||||
avatar fallback = initial circle; "waiting for X (n/m ready)".
|
||||
- Open user questions for later: warning-tone timing; bar width at
|
||||
minimum window; second-ESC on leave dialog (re-asks vs cancel);
|
||||
cancel-stops-host-server (kept: idle means no bound port).
|
||||
|
||||
## 6. Gotchas (learned the hard way)
|
||||
|
||||
1. Relic gives Range as TYPED RangeHeader — convert via unit+first.
|
||||
2. Relic server-side onDone DOES NOT fire for dead peers (proven
|
||||
live both directions: clean close AND kill -9 both ghosted to
|
||||
timeout). Presence = isClosed heartbeat sweep + socket-onError
|
||||
forget + lastSeen timeouts. NEVER depend on onDone. (Docs pattern
|
||||
confirms CloseReceived is a data event.)
|
||||
3. Welcome races room broadcasts — match WS messages by `t`, never
|
||||
by index.
|
||||
4. web_socket_channel streams are single-subscription — one listener
|
||||
per socket (test double-listens throw "already been listened to").
|
||||
5. `sub.asFuture()` can resolve before `onDone` — poll with
|
||||
deadlines in tests.
|
||||
6. Comma-form mpv reconnect flags work here; colon-form rejected
|
||||
(re-verify via set_property if touched).
|
||||
7. Numeric literals corrupted once by tooling — verify with rg after
|
||||
writes touching ports/consts.
|
||||
8. Tool `bash` `&`-backgrounding is unreliable; use nohup + log file
|
||||
in its OWN call, poll in later calls, kill via `fuser -k
|
||||
PORT/tcp` (never `pkill -f`). Chain small calls, never long
|
||||
`&`-chains (they hang the runner past timeout with zero output).
|
||||
9. Tests must not assume threat model beyond trust-basis.
|
||||
10. web_socket_channel 3.x: `channel.ready` errors unhandled on
|
||||
refused connections — eat it (`unawaited(ch.ready.then(...,
|
||||
onError: ...))`); stream onError is the reporting path.
|
||||
11. MediaKit has NO public Player.setProperty — runtime props go via
|
||||
guarded dynamic call on player.platform; cache size via public
|
||||
PlayerConfiguration(bufferSize:).
|
||||
12. NEVER `flutter test`-pump the session screen: media_kit Video
|
||||
hangs the runner (a hung flutter_tester must be kill -9'd by PID;
|
||||
pgrep carefully). Sync-path coverage lives in room_diff (pure),
|
||||
lifecycle (server, raw WS), coord_presence + e2e (real mpv,
|
||||
no widgets). Deleted session_overlays_test.dart for this reason.
|
||||
13. No `flutter analyze` unless asked (IDE covers it) — except
|
||||
deliberate per-change checks; kept clean throughout.
|
||||
14. New `MediaDesktopVideoControlsThemeData` per build is fine
|
||||
(stock guards subscriptions) but wasteful; leave it.
|
||||
15. Env overrides for mpv experiments (no rebuild to switch):
|
||||
SYNC_RELAY_VO (e.g. xv, gpu, gpu-next), SYNC_RELAY_HWDEC (e.g. no).
|
||||
16. Hidden-but-mounted overlays keep focus: closing the sheet left
|
||||
focus on an offstage button, so Space fell through (350 ms
|
||||
throttle hole) into the hidden control. Fix pattern: ExcludeFocus
|
||||
on the closed subtree + unfocus-on-close effect. Same reason new
|
||||
overlay buttons reuse the shared no-focus node.
|
||||
|
||||
## 7. Sounds (deferred, wiring ready)
|
||||
|
||||
Base: Kenney Interface Sounds (CC0, confirmation_* ladder for
|
||||
ascending-ready). Alt ladder: ObsydianX Pack 1 (CC0, avoid paid
|
||||
packs). Soft layer: akx/Notifications (CC0 branch). Playback pick:
|
||||
flutter_soloud (preload WAV, per-play pitch) — NOT added. Swap-in =
|
||||
replace NoopSfxService; cues already fire with readyIndex. URLs +
|
||||
mapping in 09-11 chat (unreliable — re-derive from sources above).
|
||||
|
||||
## 8. Black window while playing (09-12, FIXED)
|
||||
|
||||
Symptom was: room PLAYING ⇒ entire window black (audio + keys
|
||||
alive, unready restored, zero errors). Cause: the waiting-avatars
|
||||
center overlay unmounted its GPU textures (avatar Images) exactly
|
||||
at the HELD→PLAYING flip; teardown racing mpv's frame uploads on
|
||||
the shared GL context fails silently ⇒ black. Legacy chips never
|
||||
unmounted and minimal never mounted, which is why they survived.
|
||||
Fix: center overlay is Visibility(visible + maintainState +
|
||||
maintainAnimation + maintainSize) — avatars stay mounted, paint
|
||||
skipped when hidden. Verified by user with full tree, and holds
|
||||
over internet (WAN 2-client test, port 6969, late 09-12) — CLOSED.
|
||||
RULE: never unmount GPU-textured widgets on a transport-gated
|
||||
branch; keep mounted + skip paint. Bisect screens
|
||||
(legacy/minimal/nosheet + SYNC_RELAY_UI switch in app.dart)
|
||||
DELETED 09-12 (commit tracks this); no commented bisect lines
|
||||
remain in session_screen.dart.
|
||||
|
||||
## 9. Next (ordered, updated 09-12 late session)
|
||||
|
||||
1. §8 black screen — CLOSED (fix verified incl. over internet).
|
||||
2. Remove bisect flags — DONE 09-12 (files deleted, app.dart
|
||||
routes session directly; tree analyzes clean).
|
||||
3. 2-client LAN soak — user reports DONE 09-12 (details pending).
|
||||
4. 1.4 GB real-file media test — user reports DONE 09-12
|
||||
(details pending; api_key must never land in files).
|
||||
5.–8. MIGRATED 09-12 to `docs/ROADMAP.md` backlog (cache-exhaustion,
|
||||
sounds, file-join wart, /health) — the roadmap is now the single
|
||||
backlog; this section keeps session history + user-owned pendings
|
||||
only.
|
||||
9. FIXED 09-12: auto-unready fired while passively watching —
|
||||
the 10-min check keyed off `_lastInput`, refreshed only by
|
||||
explicit actions. Steady playback now counts as activity:
|
||||
warn + auto-unready are skipped while room transport is
|
||||
PLAYING (`session_coordinator.dart` idle block). HELD-and-idle
|
||||
still unreadies after 10 min.
|
||||
10. FIXED 09-12 (second attempt — first theory was wrong): one chat
|
||||
send rendered as TWO log rows. Root cause is server dual
|
||||
delivery, not double-send: `broadcastChat`
|
||||
(`room_hub.dart:69`) emits both a `ChatMessage` (with text,
|
||||
converted to a feed entry client-side) and a textless `feed`
|
||||
entry, and the client logged both. The blank-guard in
|
||||
`sendChat` was defense against a phantom. Real fix: client
|
||||
drops textless `chat` feed entries (`sync_client.dart` feed
|
||||
case) — one send, one row, one toast, one sfx. No wire
|
||||
change, still v4. Chat field also keeps focus after send
|
||||
(dedicated FocusNode, re-requested on submit/button).
|
||||
11. KNOWN (non-breaking, fix later — roadmap backlog #2): seeks feel
|
||||
jumpy (forward-back-forward) — suspect the seek-land retry
|
||||
(`_checkSeekLanded`, `session_coordinator.dart:429`)
|
||||
re-applying SeekAction against the 0.5 s deadband / 2 s
|
||||
debounce and the 1 Hz room echo. Needs a live repro with a
|
||||
timeline log before touching.
|
||||
12. Updater v1 SHIPPED 09-12: `watch-together --upgrade`
|
||||
(`apps/desktop/lib/src/update/updater.dart`, zero Flutter
|
||||
imports, 7 tests) — manifest check, download, sha256, apply
|
||||
over live install (extract-temp + delete-first move-over;
|
||||
ETXTBSY forbids writing the running exe, delete+rename is
|
||||
fine). Version stamped via `--dart-define=APP_VERSION`
|
||||
(wired in `tool/package_linux.sh`; dev runs report
|
||||
`0.0.0-dev`). Publishing is CI-driven: tag `vX.Y.Z` →
|
||||
`.gitea/workflows/release.yml` builds, creates the Forgejo
|
||||
release, uploads the tarball. Manifest is written locally by
|
||||
`release.sh` (predictable asset URL, no CI roundtrip);
|
||||
sha256 is optional — skipped with a note when absent
|
||||
(HTTPS + gzip CRC cover the realistic cases).
|
||||
Manual trigger only: bump desktop pubspec in a release commit
|
||||
first (script enforces tag == pubspec). No silent
|
||||
proto jumps: updater warns when manifest protocol != local.
|
||||
Environment: dart 3.14 beta, flutter 3.48 beta, mpv+ffmpeg present,
|
||||
~/Videos/chiyo.mp4 sample. Never kill the user's Syncplay mpv
|
||||
instance. Git repo at project root (init 09-12); commit code + docs
|
||||
together; prefer FEW commits (one per topic/session, not per fix)
|
||||
over bisect-granularity — user decision 09-12.
|
||||
|
||||
## 10. Window chrome — frameless, mpv-style (09-12, VERIFIED by user)
|
||||
|
||||
Native title bar gone everywhere (`window_manager ^0.5.2`,
|
||||
`TitleBarStyle.hidden` in main.dart GUI path only, headless
|
||||
untouched). New screens MUST bring their own drag + min/max/close
|
||||
or they ship immovable — reuse `widgets/window_buttons.dart` +
|
||||
`widgets/window_chrome_bar.dart` (welcome/connect pattern).
|
||||
Session: drag is `widgets/window_drag_layer.dart`, topmost raw-
|
||||
pointer observer inside the controls stack (a GestureDetector
|
||||
below media_kit's overlay LOSES the arena — do not revert to
|
||||
that); dead zones: bottom 120 px (seek stays exact), open sheet,
|
||||
leave dialog. `modifyVolumeOnScroll: false` is load-bearing for
|
||||
drag — side effect: volume is speaker-button-only, no scroll-
|
||||
volume. Maximize button = media_kit fullscreen toggle in
|
||||
session (no OS maximize there); NO aspect-ratio lock by user
|
||||
decision (custom UI lives outside the video frame). Close =
|
||||
graceful leave + stopServer, then window close. Min size
|
||||
160x90, no max. Verified on KDE Wayland (Arch): bar-drag,
|
||||
video-drag, fullscreen, free resize all good.
|
||||
@@ -1,28 +1,98 @@
|
||||
# sync_relay_project
|
||||
# watch-together
|
||||
|
||||
Watch-together desktop app (working title). One binary, three modes:
|
||||
Watch a video with friends, in sync, with the quality of a real mpv player.
|
||||
Paste a Jellyfin link, any direct video link, or pick a file on your own disk;
|
||||
everyone's play, pause and seek stay together.
|
||||
|
||||
- **Client** — GUI Join, sync WS + mpv. No server code runs.
|
||||
- **Server+Client (Host)** — GUI Host starts `packages/server` in-process,
|
||||
listens on a port, self-connects via loopback. Host must open ports.
|
||||
Clients never open ports.
|
||||
- **Headless** — `watch.exe --headless`, same server package, no window.
|
||||
Why not Jellyfin SyncPlay: it runs in the browser and its sync is poor.
|
||||
This is a desktop app that drives mpv.
|
||||
|
||||
v1 order: links support (shared http(s) media URL, Syncplay-style sync)
|
||||
first, file relay (server-proxied bytes, whitelist case) after.
|
||||
Jellyfin native + auto-update are deferred as their own undertakings.
|
||||
Name: `watch-together` is a placeholder and the final name is undecided
|
||||
(`sync-relay` is the working name). It appears in file names and the
|
||||
update manifest until that is settled.
|
||||
|
||||
Layout:
|
||||
- Desktop only (Linux now, Windows planned), always mpv. No web clients.
|
||||
- No transcoding. Everyone plays the original file.
|
||||
- Friends trust model: no accounts, no roles. A blank password means an open server.
|
||||
|
||||
```
|
||||
/packages/core protocol, sync engine, messages (pure Dart, no Flutter)
|
||||
/packages/server room + media HTTP endpoint (pure Dart, no Flutter)
|
||||
/apps/desktop Flutter app embedding packages/server (the ONE binary)
|
||||
/tool spikes + harnesses (range_logger, shaper, fake_viewer, e2e_check)
|
||||
/deploy systemd unit, notes
|
||||
/docs PROTOCOL.md, ARCHITECTURE.md, mpv-http-contract.md, ROADMAP.md
|
||||
```
|
||||
How it fits together: [docs/HOW-IT-WORKS.md](docs/HOW-IT-WORKS.md).
|
||||
|
||||
Phases + status: `docs/ROADMAP.md`. Session handoff: `HANDOFF.md` (root).
|
||||
## Install and update
|
||||
|
||||
Enforcement: `core` and `server` must have zero Flutter imports (tested).
|
||||
Download the latest `watch-together-<version>-linux-x64.tar.gz` from the
|
||||
Releases page of this repo, extract it, run `watch_together`.
|
||||
|
||||
Update from the app: when a newer version exists, an update indicator
|
||||
appears on Home and opens the update card. "Update and restart" downloads,
|
||||
verifies, applies in place and restarts; the install folder must be writable
|
||||
by you. Settings has Check now and an auto-check toggle. From a terminal:
|
||||
`watch_together --upgrade` or `update.sh` (shipped in the tarball). A
|
||||
release with a newer protocol is a required update: everyone on a server
|
||||
must run the same protocol version, and a mismatch is refused with a message
|
||||
telling you to update.
|
||||
|
||||
## Host from the app
|
||||
|
||||
Home, Host card. Pick a port, an optional password (blank = anyone can
|
||||
join), and optionally a video: a link or a file path. Your name and avatar
|
||||
come from your profile. The app starts a
|
||||
server inside itself and joins it. Friends need to reach your machine: open
|
||||
the TCP port on your router (or use a LAN or VPN address). Friends open nothing.
|
||||
|
||||
If you pick a local file, the server streams it to everyone from your disk.
|
||||
Nothing is uploaded or stored anywhere.
|
||||
|
||||
## Dedicated server
|
||||
|
||||
Being reworked: today the server is started by the app (host) and the
|
||||
standalone server is a dev tool. The planned dedicated server is a separate
|
||||
binary built from `packages/server`. See [docs/ROADMAP.md](docs/ROADMAP.md).
|
||||
|
||||
## Join
|
||||
|
||||
Home, Join card. Enter the host's address, port, and the password if there
|
||||
is one. Your name and avatar come from your profile (Home chip or Settings).
|
||||
Servers you got into appear as up to 6 Recent chips on Home: one click rejoins.
|
||||
|
||||
## Settings
|
||||
|
||||
Profile, hardware decoding (auto, off or custom; the env var
|
||||
`SYNC_RELAY_HWDEC` overrides it; applies on the next join or Reload video),
|
||||
updates, and the keyboard shortcuts (read-only). Integrations (Jellyfin) and
|
||||
shaders show "coming soon". About shows the version.
|
||||
|
||||
Keys in a session: Space = ready / unready, Tab = side panel (people, log,
|
||||
chat, change video), Left/Right = seek 10 s, F = fullscreen, Esc = close
|
||||
panel or ask to leave, Ctrl+R = reload video.
|
||||
|
||||
## Ports
|
||||
|
||||
| What | Default | Notes |
|
||||
|---|---|---|
|
||||
| Sync WebSocket and `/health` | TCP 8080 | plain `ws://`, no TLS |
|
||||
| Local file streaming | same port | path `/m/...`, needs a session token |
|
||||
|
||||
Only the host needs the port reachable.
|
||||
|
||||
## Jellyfin
|
||||
|
||||
Pasting a Jellyfin direct-play link works today like any link. Proper
|
||||
Jellyfin support (per-person login, watched status, library browsing) is
|
||||
planned: see [docs/ROADMAP.md](docs/ROADMAP.md).
|
||||
|
||||
## Video quality and shaders
|
||||
|
||||
Planned: Anime4K and other mpv shaders as a per-viewer setting. See
|
||||
[docs/ROADMAP.md](docs/ROADMAP.md).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- "host offline - check address and port": wrong address, or the port is not open.
|
||||
- "wrong server password": password mismatch.
|
||||
- "host runs a different version": update the app on both sides.
|
||||
- Picture frozen but sound fine: Ctrl+R reloads the video; the app also does it automatically.
|
||||
- Everyone waits at "waiting for X": X has not buffered 4 seconds yet or has not pressed Space.
|
||||
- No Windows build yet.
|
||||
|
||||
For contributors: [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md),
|
||||
[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), [docs/PROTOCOL.md](docs/PROTOCOL.md).
|
||||
+216
-68
@@ -1,84 +1,232 @@
|
||||
# ARCHITECTURE
|
||||
# Architecture
|
||||
|
||||
One binary, three modes. `packages/server` (Relic) is embedded, not a separate product.
|
||||
Plain overview first: [HOW-IT-WORKS.md](HOW-IT-WORKS.md). Wire format:
|
||||
[PROTOCOL.md](PROTOCOL.md). Plans: [ROADMAP.md](ROADMAP.md).
|
||||
|
||||
- **Client**: GUI Join, sync WS client (`web_socket_channel` in
|
||||
`apps/desktop`, NOT `packages/core` — core stays Flutter-free)
|
||||
+ mpv. No server code runs.
|
||||
- **Server+Client (Host)**: GUI Host starts `packages/server` in-process, binds port, self-connects via loopback. Host opens ports, runs traffic. Clients open nothing outbound-only.
|
||||
- **Headless**: `--headless`, same server package, no window/player init.
|
||||
## Packages
|
||||
|
||||
Server side: **Relic** (HTTP + built-in WS). Client side WS: **`web_socket_channel`**
|
||||
(Relic has no client; server WS handling stays in Relic).
|
||||
| Path | What | Rule |
|
||||
|---|---|---|
|
||||
| `packages/core` | protocol messages, sync engine (`RoomEngine`), constants, config, Range parsing, logging | pure Dart, no Flutter |
|
||||
| `packages/server` | Relic HTTP + WebSocket server (`RoomServer`, `RoomHub`, `Session`, media endpoint), `bin/server.dart` | pure Dart, no Flutter |
|
||||
| `apps/desktop` | Flutter app: UI, `SyncClient`, `SessionCoordinator`, `PlaybackController` (media_kit/mpv), updater | the only shipped binary |
|
||||
| `tool` | harnesses: `range_logger`, `shaper`, `fake_viewer`, `e2e_check` | dev only |
|
||||
| `deploy` | systemd unit for the standalone server (stale until the dedicated server exists) | |
|
||||
|
||||
Media order (locked): links first (shared http(s) URL, Syncplay-style sync),
|
||||
file relay after (server-proxied bytes for the whitelist case).
|
||||
Guard: `packages/core/test/flutter_free_test.dart` fails if anything under
|
||||
`core/lib` imports Flutter, `dart:ui` or `signals_flutter`; CI runs
|
||||
analyze + tests for core and server on every push. Server has no such test but
|
||||
the same rule; it is checked by CI building without Flutter. The client
|
||||
WebSocket (`web_socket_channel`) lives in `apps/desktop`; Relic has no client.
|
||||
|
||||
## Media paths (links vs file relay)
|
||||
## Server lifecycle
|
||||
|
||||
Two ways to watch, same sync engine, same Range endpoint:
|
||||
Today (in-process): `SessionState.host()` creates a `RoomServer`, starts it
|
||||
on the chosen port (bound to all IPv4 interfaces), then the app joins it over
|
||||
`127.0.0.1` like any other client. Leaving or closing the window stops it.
|
||||
`RoomServer` runs a 1 s heartbeat: stale sweep, closed-socket sweep, silent
|
||||
socket drop, room + feed broadcast. There is one session, id `default`.
|
||||
|
||||
1. **Links (primary, shipped):** host shares an http(s) media URL
|
||||
(e.g. a Jellyfin Download URL). Viewers stream it directly; the
|
||||
server is sync-authority only. The URL travels as `media.url`
|
||||
in room broadcasts and as welcome `mediaUrl` (set to the shared
|
||||
URL when the server holds no file). This
|
||||
covers most sessions — proxying is usually unnecessary.
|
||||
2. **File relay (secondary, local-file half shipped):** host serves
|
||||
bytes from their own machine through the server's token-gated
|
||||
Range endpoint `GET /m/<sid>/<token>/<name>`. Viewers (and the
|
||||
host via loopback) play the server URL. This is the fallback for
|
||||
the whitelist case: viewers cannot reach the source, but the
|
||||
server/host can (e.g. Jellyfin allowlists the host, not the
|
||||
viewers). No server storage — bytes stream file→socket, config +
|
||||
logs only, never to disk.
|
||||
`--headless` does not work: the Flutter binary needs a display and exits with
|
||||
"cannot open display" on a headless box. `packages/server/bin/server.dart` is
|
||||
the only working standalone server (flags `--port`, `--secret`, `--name`,
|
||||
`--media`, `--advertise-host`, `--config`; default port 8080).
|
||||
|
||||
Backends behind `MediaSource` (`packages/core/lib/src/media_source.dart`):
|
||||
`FileMediaSource` (local file — SHIPPED as an abstraction, but the
|
||||
live route serves `SessionMedia.file` directly via `file.openRead`;
|
||||
nothing in `packages/` references `FileMediaSource` today);
|
||||
`HttpSource` (server fetches a remote URL and re-serves it —
|
||||
DEFERRED, Phase 8). `resolveMediaRequest` stays
|
||||
framework-agnostic so both ride the same tested Range table.
|
||||
Planned: hosting from the app runs the server on a Dart isolate (ports and
|
||||
sockets stay in the same process, UI stays responsive); the dedicated server
|
||||
is a separate `dart compile exe` of `packages/server`. See [ROADMAP.md](ROADMAP.md).
|
||||
|
||||
## Deferred: remote-host upload tunnel (NOT active protocol)
|
||||
Passwords: the default secret is empty, which means an open server (a
|
||||
warning is logged). Otherwise one shared secret, compared at hello.
|
||||
The host is the member that connected from a loopback address; it cannot be
|
||||
kicked. Kick = revoke token, send `kicked`, close; the
|
||||
person may rejoin at once with a new token.
|
||||
|
||||
If a VPS tier ever needs a *remote* host to push bytes to a server
|
||||
it cannot serve from disk, the reserved design is the brief's WSS
|
||||
tunnel: host dials out, JSON control frames
|
||||
(`hello`/`welcome`+mediaUrl, `open`{id,start,end}, `ack`, `cancel`,
|
||||
`eof`/`err`) plus binary `[u32 BE reqId][payload]` chunks —
|
||||
chunk 256 KiB, credit window 8 (≈2 MiB in flight), max 16 concurrent
|
||||
`open`, relay acks only after socket flush (real backpressure).
|
||||
`RelayConstants.chunkBytes/creditWindowChunks/maxConcurrentOpen`
|
||||
are reserved for this. No code exists; direct-serve covers v1
|
||||
(host IS the server, opens ports itself).
|
||||
## Sync model
|
||||
|
||||
On file relay, host playback goes through loopback like everyone
|
||||
else's (`http://127.0.0.1:<port>/…`) — same code path as remote
|
||||
viewers, so server bugs surface on the host screen first. (On
|
||||
links the host plays the direct URL.) Deployment ladder,
|
||||
cheapest first: host's own PC (LAN, or internet with port-forward) →
|
||||
same binary `--headless` on a friend's always-on box → cheap VPS
|
||||
(CGNAT escape; TLS/Caddy deferred with this tier).
|
||||
Loosely modeled on Syncplay, with a different readiness design.
|
||||
|
||||
Sync: heavily Syncplay-inspired + original readiness design, NOT 1:1
|
||||
(audit done; deviations: clock never moves from reports, no
|
||||
pause-on-leave, force-for-all, quorum gate — see HANDOFF §2).
|
||||
Additive namespaced extensions only, `proto` bump + `PROTOCOL.md` same-commit.
|
||||
- **Transport** is `playing` or `held`. The server owns the clock:
|
||||
`positionAt(now) = position + (playing ? now - serverTime : 0)`.
|
||||
Reports from clients never move it. Only seek, swap and gate transitions
|
||||
write `position`/`serverTime` and bump `eventId`.
|
||||
- **Reports**: clients send `position`, `buffered`, `eventId` every 1 s. The
|
||||
server stores `buffered` and liveness; a report with `eventId` older than
|
||||
current has its position ignored.
|
||||
- **Ready gate** (HELD to PLAYING): the in-set is non-empty, every in-set
|
||||
member is ready, fresh and has `buffered >= 4 s` (`minBufferSecs`), and
|
||||
quorum holds: in-set size >= 2, unless at most one member is fresh. So a
|
||||
lone viewer plays alone; with a second viewer present one Space waits for the
|
||||
other. Joining members start with `inSet=false, ready=false` and never block.
|
||||
Disconnect removes from the set; swap clears everyone.
|
||||
- **Pause**: while PLAYING, any in-set member who becomes not ready, under
|
||||
4 s buffered or stale flips the server to HELD (feed action `pause` by `server`).
|
||||
Recovery resumes by itself when the gate passes.
|
||||
- **Seek**: allowed from any member in any transport; the server writes the
|
||||
position and bumps `eventId`; flags are untouched. Clients diff each `room`
|
||||
broadcast by `eventId` (`RoomDiff`): new event means seek to the computed
|
||||
room position, transport change means play/pause. Between events: drift
|
||||
over 5 s for more than 2 s seeks, ahead by more than 1.5 s sets rate 0.95
|
||||
until level, otherwise nothing. Local seeks are detected by the coordinator
|
||||
polling mpv position once per second (`_detectUserSeek`, jump over 2 s and
|
||||
more than 2 s from the room) and sent as a `seek` intent; a retry
|
||||
(`_checkSeekLanded`) re-applies a room seek that has not landed after 1.5 s.
|
||||
Known roughness: see ROADMAP.
|
||||
- **Liveness**: WS ping every 5 s; silent over 15 s means stale (out of the
|
||||
gate, `stale` feed); over 30 s the socket is closed and the member removed.
|
||||
Closed sockets are polled with `isClosed` every second: Relic's server-side
|
||||
`onDone` does not fire for dead peers, so presence never relies on it.
|
||||
- **Client policies**: no room broadcast for 5 s means connection lost, clear
|
||||
the server state and reconnect every 1.5 s until cancelled (an epoch counter makes
|
||||
a cancelled attempt stale) or kicked. After a reconnect the ready flag is
|
||||
restored from the server's state. Cancel during setup aborts the join.
|
||||
Closing the window leaves gracefully (2 s timeout).
|
||||
- **Idle** (`IdleTracker`): measured from the later of the last input and the
|
||||
moment playback stopped. While HELD and self ready: warning at 8 min,
|
||||
auto-unready at 10 min, and never less than 2 min after the warning.
|
||||
- **Deviations from Syncplay**: clock never follows the slowest watcher; no
|
||||
pause-on-leave; anyone can force ready/unready (all or one); quorum gate;
|
||||
the clock is server time, and remote clock offset is not corrected (clients
|
||||
assume their clock matches the server's).
|
||||
|
||||
No server storage: config + logs only. Media never touches disk.
|
||||
Additive wire changes only; bump `proto` and PROTOCOL.md in the same commit.
|
||||
|
||||
## Health reporting (standing rule)
|
||||
## Media
|
||||
|
||||
Every failure must display. Nothing stalls silently.
|
||||
Two sources, same sync.
|
||||
|
||||
- Server `/health` returns machine-readable status: ok, room, viewer
|
||||
count, uptime, media, bytes served, version. The app reads it.
|
||||
- App surfaces all failure states in UI with cause + what-to-do:
|
||||
connecting/reconnecting, wrong secret (401), kicked (403), session
|
||||
gone (410), host offline / buffering (503), starvation
|
||||
("waiting for X — N.N s buffered"). Backed by a structured status
|
||||
signal (severity + message), not bare strings.
|
||||
- Errors log via `debugPrint` + stack trace at the boundary, then
|
||||
convert to a displayable state. No swallowed exceptions.
|
||||
- **Link**: the media field holds an http(s) URL. It travels as `media.url`
|
||||
in `swap` and `room`, and as `mediaUrl` in `welcome`. Each app plays it
|
||||
directly; the server is only the clock.
|
||||
- **Host local file**: `RoomServer.setLocalFile` / `mediaPath` makes the file
|
||||
the session media. `GET /m/<sid>/<token>/<name>` serves it with a Range
|
||||
table, gated by the viewer's token. The server builds the URL per viewer
|
||||
(`http://<host>:<port>/m/default/<token>/<name>`), so tokens never reach
|
||||
other viewers. `<host>` is the authority the client dialled; `advertiseHost`
|
||||
(default empty) only overrides it when set explicitly. When a `room`
|
||||
broadcast has no `media.url`, the client builds the URL itself from the host
|
||||
it connected to. Bytes stream file to socket and are counted as streamed
|
||||
(`bytesServed`); nothing is stored. The host plays through loopback like
|
||||
everyone else.
|
||||
|
||||
Range table (`resolveMediaRequest`, tested byte-exact):
|
||||
|
||||
| Request | Response |
|
||||
|---|---|
|
||||
| no Range | 200, `Content-Length` = size |
|
||||
| `bytes=a-b` | 206, `Content-Range: bytes a-b/size` |
|
||||
| `bytes=a-` | 206 to EOF |
|
||||
| `bytes=-n` | 206 last n bytes |
|
||||
| unsatisfiable | 416, `Content-Range: bytes */size` |
|
||||
| host has no file | 503 + `Retry-After: 2` |
|
||||
| bad / unknown token, kicked, unknown session | 401, 403, 410 |
|
||||
|
||||
Always `Accept-Ranges: bytes`; `Content-Type` from the extension
|
||||
(`contentTypeForFile`). Relic hands the Range over as a typed `RangeHeader`;
|
||||
the endpoint takes the first range and keeps the unit, so non-`bytes` units
|
||||
still get 416.
|
||||
|
||||
What mpv does against it (recorded with `tool/range_logger.dart`): mp4 opens
|
||||
once with no Range then streams `bytes=0-` to the end; mkv opens `bytes=0-`
|
||||
twice (probe, playback), reads a small tail at `size-n` (cues, expressed
|
||||
open-ended), then every seek is a fresh `bytes=N-` connection. A server
|
||||
outage of a few seconds is ridden out from mpv's cache with no error. Longer
|
||||
outages exhaust the cache and rely on reconnect (not chaos-tested yet).
|
||||
|
||||
mpv settings (`PlaybackController`): `Player` with `bufferSize` 64 MiB,
|
||||
`libass` on; pinned via runtime property: `demuxer-readahead-secs=10`,
|
||||
`cache-secs=10`, `stream-lavf-o=reconnect_streamed=1,reconnect_on_http_error=1,reconnect_delay_max=4`
|
||||
(comma form; this mpv build rejects colon-joined). media_kit has no public
|
||||
`setProperty`; the code calls it through a guarded dynamic call on
|
||||
`player.platform`. Hardware decoding comes from Settings (auto, off, custom)
|
||||
and is passed to the `VideoController`; env `SYNC_RELAY_HWDEC` (e.g. `no`)
|
||||
overrides it. It is read per player, so a change applies on the next join or
|
||||
Reload video.
|
||||
|
||||
Player lifecycle: the `Player` and its `VideoController` live for one
|
||||
session: created on join, disposed on leave, replaced by `recreate()` for
|
||||
"Reload video" (button or Ctrl+R; volume kept). `openUrl` opens with `play: false`; play/pause/seek/rate come only from
|
||||
`RoomDiff` actions or the play gate (`localPlayAllowed`: self in-set and
|
||||
ready). A room seek within 0.5 s of the current position is skipped. `leave()`
|
||||
stops the player (otherwise audio keeps playing). On media change
|
||||
(`mediaId` bump) the coordinator opens the new URL and `RoomDiff` seeks to 0,
|
||||
holds and resets rate.
|
||||
|
||||
Frozen picture: `FrameWatchdog` samples about once a second. It hashes a
|
||||
small capture of the compositor's centre (`ShownFrameProbe`) and suspects a
|
||||
freeze when position advances while that hash stays constant for 3 s. Two
|
||||
differing mpv `screenshot-raw` frames show the decoder is still producing
|
||||
pictures, which confirms the freeze (a still scene is dismissed). A confirmed
|
||||
freeze reloads the video with a toast. It fires once until the picture moves
|
||||
again for 10 s, so it cannot loop. Seeks, pause, buffering and a hidden window
|
||||
disarm it.
|
||||
|
||||
Never unmount GPU-textured widgets (avatar images) on a transport-gated
|
||||
branch: teardown races mpv's frame upload and the window goes black. Keep them
|
||||
mounted and skip painting (`Visibility` with `maintainState`).
|
||||
|
||||
## Desktop UI
|
||||
|
||||
One `HomeScreen` (`screens/home/`) with a nav rail: Home (profile chip, Join
|
||||
card, Host card, up to 6 Recent server chips for one-click rejoin), Settings
|
||||
(profile, hardware decoding, updates, shortcuts, shaders "coming soon"),
|
||||
Integrations (Jellyfin "coming soon") and About. The update indicator sits on
|
||||
Home. Settings persist in two Hive boxes: `connect_form` (last form and
|
||||
recents; saved even when a join fails) and `settings`. Link media names are
|
||||
derived from the URL path and never contain a query string.
|
||||
|
||||
Window chrome is frameless (`window_manager`): every screen brings its own
|
||||
drag area and min/max/close buttons (`widgets/window_chrome_bar.dart`,
|
||||
`window_buttons.dart`); the session drag layer is `window_drag_layer.dart`.
|
||||
|
||||
## Health reporting
|
||||
|
||||
Every failure must show in the UI with a cause and a next step; nothing
|
||||
stalls silently. The app has a `HealthStatus` signal (severity + message)
|
||||
for connecting, reconnecting, wrong password, version mismatch, kicked,
|
||||
host offline, and starvation ("waiting for X - N.N s buffered").
|
||||
Errors are logged with a stack trace at the boundary, then turned into a
|
||||
displayable state. The server's `GET /health` returns JSON (`ok`, `room`,
|
||||
`proto`, `viewers`, `bytesServed`, `uptimeSecs`); it is unauthenticated and
|
||||
the app does not read it.
|
||||
|
||||
## Updater
|
||||
|
||||
`apps/desktop/lib/src/update/updater.dart` (no Flutter imports, unit tested).
|
||||
Reads a manifest (`version`, `protocol`, `url`, optional `sha256`, `notes`)
|
||||
from `updateManifestUrl`, compares versions, downloads the tarball, checks
|
||||
sha256 when present, then extracts into a staging dir inside the install
|
||||
folder (same filesystem, so moves are renames; the folder must be
|
||||
user-writable). Each top-level entry is moved aside and the new one moved in,
|
||||
with rollback on failure (writing the running executable fails with ETXTBSY;
|
||||
rename works). A packaged install is detected by `data/` and `lib/` next to
|
||||
the binary; anything else exits with "update manually". A manifest whose
|
||||
protocol is newer than the local one is a required update. Windows apply
|
||||
(`*.old` rename, swept on next start) exists but is untested. The version is
|
||||
stamped with `--dart-define=APP_VERSION` (dev runs report `0.0.0-dev` and skip
|
||||
checks). `watch_together --upgrade` (or the shipped `update.sh`) runs it from
|
||||
the CLI; `UpdateController` runs the same code for Home's update indicator,
|
||||
which opens the update card, and restarts the app ("Update and restart").
|
||||
Settings has Check now and an auto-check toggle. The manifest is `updates/manifest.json` in
|
||||
this repo, written by `tool/release.sh`.
|
||||
|
||||
## Decisions
|
||||
|
||||
- Desktop + mpv only, no web clients: exists because Jellyfin SyncPlay is bad.
|
||||
- No transcoding: bandwidth is plentiful and the Jellyfin box should not be loaded.
|
||||
- No roles, friends trust model: simplest thing that fits the use.
|
||||
- Blank password = open server: the owner chooses between open and private.
|
||||
- The host cannot be kicked: the server dies with the host anyway.
|
||||
- Server ships inside the app (and, planned, as a separate compiled server):
|
||||
a Flutter binary cannot run headless, so a dedicated server needs its own
|
||||
build of `packages/server`.
|
||||
- Server owns the clock; reports never move it: avoids ping-pong between clients.
|
||||
- Quorum gate: a lone viewer must be able to watch; two or more must start together.
|
||||
- Relic for the server, `web_socket_channel` for the client; packages are
|
||||
chosen on merit, not locked.
|
||||
- Plain `ws://`, no TLS: friends use LAN, VPN or a forwarded port; TLS
|
||||
(e.g. a reverse proxy) is a documentation item.
|
||||
- One session per server, id `default`: the map is keyed for later but unused.
|
||||
- No server storage: config and logs only; media never touches the server disk.
|
||||
- Avatars ride in the 1 Hz `room` broadcast (cap 256 KiB base64 per avatar):
|
||||
simple, costs host uplink; hash-based serving is on the roadmap.
|
||||
@@ -0,0 +1,102 @@
|
||||
# Development
|
||||
|
||||
Layout and rules: [ARCHITECTURE.md](ARCHITECTURE.md).
|
||||
|
||||
## Build
|
||||
|
||||
Needs the Flutter SDK (beta channel, Dart 3.13+) and, for the Linux desktop
|
||||
build, the usual GTK toolchain (clang, cmake, ninja, pkg-config, libgtk-3-dev).
|
||||
mpv libraries come from `media_kit_libs_video`.
|
||||
|
||||
```
|
||||
cd apps/desktop && flutter pub get && flutter run -d linux
|
||||
cd apps/desktop && flutter build linux --release --dart-define=APP_VERSION=<x.y.z>
|
||||
tool/package_linux.sh [--skip-build] [--noupload] # dist/*.tar.gz
|
||||
```
|
||||
|
||||
`APP_VERSION` defaults to `0.0.0-dev`; `package_linux.sh` stamps it from the
|
||||
desktop pubspec. Setting `DISCORD_WEBHOOK_URL` makes the packager post the
|
||||
tarball to Discord (never commit it).
|
||||
|
||||
## Dev loop
|
||||
|
||||
Run a server without the app, then point apps at it:
|
||||
|
||||
```
|
||||
dart run packages/server/bin/server.dart --port 8080 --secret x [--media FILE]
|
||||
```
|
||||
|
||||
Prints `READY port=... room=...` and stops on SIGINT. Then Join from one or
|
||||
more apps at `127.0.0.1:8080`. Or just Host from the app. mpv env override:
|
||||
`SYNC_RELAY_HWDEC` (wins over the Settings choice). `SYNC_RELAY_VO` has no
|
||||
effect with embedded video: media_kit forces `vo=libmpv`.
|
||||
|
||||
Harnesses in `tool/` (own `pubspec.yaml`, `dart run tool/<name>.dart`):
|
||||
`range_logger` (serve a file and log every Range request), `shaper`
|
||||
(bandwidth cap, latency, `--kill-after` for outage tests), `fake_viewer`
|
||||
and `e2e_check` (simulated viewers against a live server).
|
||||
|
||||
## Tests
|
||||
|
||||
- `cd packages/core && dart test`, `cd packages/server && dart test`.
|
||||
- `cd apps/desktop && flutter test test/<file>`.
|
||||
- `sync_playback_e2e_test.dart` and `coord_presence_test.dart` use real mpv;
|
||||
the e2e one also expects `~/Videos/chiyo.mp4`. Run only tests related to
|
||||
what you changed.
|
||||
- CI (`.gitea/workflows/ci.yml`): analyze + test for core and server only.
|
||||
Desktop tests are not in CI.
|
||||
|
||||
Traps:
|
||||
- Never pump the session screen or media_kit `Video` in `flutter test`: the
|
||||
runner hangs (kill the `flutter_tester` by PID). Test pure logic
|
||||
(`RoomDiff`, coordinator without widgets) instead.
|
||||
- Never depend on a server-side WebSocket `onDone`; assert presence with
|
||||
deadline polling.
|
||||
- `sub.asFuture()` can resolve before `onDone`; poll with a deadline.
|
||||
- Tests must not assume a threat model beyond the friends trust basis.
|
||||
|
||||
## Releasing
|
||||
|
||||
User pushes tags; CI does the rest.
|
||||
|
||||
1. `tool/release.sh [patch|minor|major] [--dry-run]`: derives the next
|
||||
version from the latest tag, bumps the three pubspecs in lockstep, writes
|
||||
`updates/manifest.json` (asset URL is predictable), commits `Release vX.Y.Z`,
|
||||
makes an annotated tag.
|
||||
2. Review, then `git push --follow-tags`.
|
||||
3. `.gitea/workflows/release.yml` builds the Linux tarball and runs
|
||||
`tool/release_publish.sh vX.Y.Z`, which creates the release and uploads it.
|
||||
By hand: `tool/package_linux.sh` then `tool/release_publish.sh vX.Y.Z`.
|
||||
|
||||
Tokens come from the environment (`RELEASE_TOKEN`, or Actions-provided
|
||||
`GITHUB_TOKEN`/`GITEA_TOKEN`). The manifest has no `sha256` by default; the
|
||||
updater skips the check when absent. Workflows live in `.gitea/workflows`
|
||||
(the forge reads `.gitea` before `.github`).
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Relic gives Range as a typed `RangeHeader`; convert from unit + first range.
|
||||
- Relic server-side `onDone` does not fire for dead peers. Presence is the
|
||||
`isClosed` sweep + `onError` forget + `lastSeen` timeouts.
|
||||
- `web_socket_channel` 3.x: `channel.ready` errors are unhandled on refused
|
||||
connections; swallow them and report through the stream's `onError`.
|
||||
Streams are single-subscription: one listener per socket.
|
||||
- media_kit has no public `Player.setProperty`; the code reaches mpv through a
|
||||
dynamic call on `player.platform`. Cache size goes through
|
||||
`PlayerConfiguration(bufferSize:)`.
|
||||
- mpv reconnect flags: comma form works, colon form is rejected. Re-verify
|
||||
through `setProperty` if touched.
|
||||
- Overlays that are hidden but mounted keep focus (Space falls into a hidden
|
||||
button). Use `ExcludeFocus` on closed subtrees and unfocus on close; reuse
|
||||
the shared no-focus node for overlay buttons.
|
||||
- `modifyVolumeOnScroll: false` on the session controls is load-bearing for
|
||||
window drag (volume is the speaker button only). A `GestureDetector` below
|
||||
media_kit's overlay loses the gesture arena; the drag layer is a raw pointer
|
||||
observer on top.
|
||||
- New screens must bring their own drag area and window buttons (frameless
|
||||
window).
|
||||
- Chat arrives twice from the server (see PROTOCOL.md); the client drops the
|
||||
textless feed copy.
|
||||
- Tab traversal is disabled on the session screen on purpose; Tab toggles the
|
||||
side panel.
|
||||
- After editing constants (ports, thresholds), grep to confirm the value landed.
|
||||
@@ -0,0 +1,76 @@
|
||||
# How it works
|
||||
|
||||
A five-minute tour. Details: [ARCHITECTURE.md](ARCHITECTURE.md), wire format:
|
||||
[PROTOCOL.md](PROTOCOL.md).
|
||||
|
||||
## The pieces
|
||||
|
||||
```
|
||||
your PC friend's PC
|
||||
+-------------------+ +-------------------+
|
||||
| app (mpv inside) | | app (mpv inside) |
|
||||
+---------+---------+ +---------+---------+
|
||||
| sync (WebSocket) | sync (WebSocket)
|
||||
v v
|
||||
+----------------------------------------+
|
||||
| server (who is ready, where are we) |
|
||||
+----------------------------------------+
|
||||
|
||||
video bytes come from somewhere else:
|
||||
- a link (Jellyfin, any http/https URL): each app fetches it directly
|
||||
- a file on the host's disk: the server streams it (/m/... endpoint)
|
||||
```
|
||||
|
||||
- **App**: the desktop window. It draws the video with mpv, shows who is
|
||||
here, and sends what you do (ready, seek, chat).
|
||||
- **Server**: keeps one shared clock and the list of people. It never sees
|
||||
video frames, only "position 123 s, playing". There is exactly one server
|
||||
and one watch session on it.
|
||||
- **Jellyfin** (or any web host): where a link's video lives. The server does
|
||||
not touch it today; each app downloads it itself.
|
||||
|
||||
## Ways to run
|
||||
|
||||
- **Host from the app**: the app you click Host in also runs the server.
|
||||
Simplest. The server lives as long as that app.
|
||||
- **Dedicated server**: a separate server program on an always-on box.
|
||||
Planned, see [ROADMAP.md](ROADMAP.md).
|
||||
|
||||
Friends join either way with the Join card on Home.
|
||||
|
||||
## One session, start to finish
|
||||
|
||||
1. **Connect.** The app opens a WebSocket to the server and says hello
|
||||
(version, password, name, avatar). Wrong password or version: refused with
|
||||
a reason. Otherwise the server answers with a welcome and the current state.
|
||||
2. **Catch up.** The app opens the current video, jumps to where the server
|
||||
says we are, and buffers. A new arrival is not ready yet and never holds
|
||||
anyone back.
|
||||
3. **Ready gate.** Press Space to say you are ready. The server starts
|
||||
playback only when everyone who is ready has at least 4 seconds buffered.
|
||||
If two or more people are ready, one person alone cannot start it for the
|
||||
others. A lone viewer can play alone.
|
||||
4. **Play.** The server owns the clock. Each app compares its mpv position
|
||||
to the clock: a small lead slows mpv a little (0.95x), a big gap (over 5 s)
|
||||
jumps. If someone's buffer runs dry the server pauses everyone and resumes
|
||||
when they have caught up.
|
||||
5. **Seek.** Anyone can seek, anytime. The server moves the clock and
|
||||
everybody follows. Seeking does not change anyone's ready flag.
|
||||
6. **Swap.** Anyone can change the video by pasting another link. Everybody
|
||||
stops at 0, ready flags clear, and the gate runs again.
|
||||
7. **Leave.** Closing or leaving drops you from the server; the others keep
|
||||
going. If you vanish without saying goodbye, you are marked stale after 15 s
|
||||
(stops blocking) and removed after 30 s. The app reconnects by itself and
|
||||
restores your ready flag.
|
||||
|
||||
There are no roles: anyone can seek, swap, force-ready others and kick
|
||||
anyone except the host.
|
||||
|
||||
## Where video comes from
|
||||
|
||||
- **Link**: you paste an http(s) URL. Every app plays it directly with mpv.
|
||||
Works for a Jellyfin direct-download link or any file on the web.
|
||||
- **Local file (host only)**: you pick a file on your disk. The server
|
||||
streams it to everyone, host included, over HTTP with range requests, each
|
||||
with a per-person token. Nothing is stored or converted.
|
||||
- No transcoding, ever. Quality is whatever the file is.
|
||||
+75
-102
@@ -1,120 +1,93 @@
|
||||
# PROTOCOL — v4 (Phase 3: sync engine + readiness)
|
||||
# Protocol (proto = 4)
|
||||
|
||||
`proto = 4`. Bump history: v1 hello/state/event shapes; v2 `token` in
|
||||
welcome; v3 `kicked`/`kick`; v4 readiness intents, room fan-out, feed,
|
||||
public member ids (tokens never leave their owner — media URLs carry
|
||||
only the requester's own token).
|
||||
Wire format only. Behavior (gate, clock, timeouts) is in
|
||||
[ARCHITECTURE.md](ARCHITECTURE.md). Source of truth:
|
||||
`packages/core/lib/src/protocol.dart` and `packages/server/lib/src/room_hub.dart`.
|
||||
|
||||
## Transport vs readiness (the core split)
|
||||
Any wire change bumps `RelayConstants.protoVersion` and edits this file in
|
||||
the same commit. Changes are additive.
|
||||
|
||||
Room transport is PLAYING or HELD. Seeks move transport position only:
|
||||
while PLAYING anyone seeks freely, everyone follows, catch-up buffering
|
||||
is tolerated with no flag changes. Readiness gates only the
|
||||
HELD→PLAYING transition. Starvation pauses transport without touching
|
||||
flags; recovery auto-resumes on buffers, no Space needed.
|
||||
## Transport
|
||||
|
||||
## Message catalog
|
||||
- WebSocket at `/sync`, JSON text frames, each an object with a `t` field.
|
||||
- The first client frame must be `hello` within 10 s. Server pings every 5 s.
|
||||
- Match incoming frames by `t`, never by order: `welcome` can race `room`.
|
||||
- Failures: server sends `{"t":"err","message":...}` and closes.
|
||||
Messages: `bad hello: ...`, `proto mismatch: got X, want Y`, `wrong secret`,
|
||||
`rejected: <reason>` (e.g. avatar too large), `bad message: ...` (a frame
|
||||
with a non-finite number, e.g. NaN position), `bad swap media`,
|
||||
`swap needs a url or the hosted file` (empty swap).
|
||||
- `GET /health` returns JSON (`ok`, `room`, `proto`, `viewers`, `bytesServed`,
|
||||
`uptimeSecs`; unauthenticated). `GET /m/<sid>/<token>/<name>`
|
||||
serves the host's file (Range table in ARCHITECTURE.md).
|
||||
|
||||
Client→server intents (all post-hello text frames):
|
||||
## Client to server
|
||||
|
||||
```
|
||||
{"t":"ready","ready":true,"source":"manual|auto"} Space flip / idle timer
|
||||
{"t":"force","target":"all|<memberId>","ready":false}
|
||||
{"t":"seek","position":123.4} doSeek event, room jumps
|
||||
{"t":"swap","media":{"name":..,"size":..,"duration":..,"url":..}}
|
||||
{"t":"hello","proto":4,"secret":"","user":{"name":"","avatar":""},
|
||||
"media":{"name":"","size":0,"duration":0,"url":null}}
|
||||
{"t":"ready","ready":true,"source":"manual"} source: manual | auto
|
||||
{"t":"force","target":"all"|"<memberId>","ready":true}
|
||||
{"t":"seek","position":123.4}
|
||||
{"t":"swap","media":{"name":"","size":0,"duration":0,"url":"..."}}
|
||||
{"t":"report","position":123.1,"buffered":6.2,"eventId":41}
|
||||
{"t":"chat","text":"…"}
|
||||
{"t":"kick","target":"<memberId>","reason":"…"} v3, now by member id
|
||||
{"t":"chat","text":"..."}
|
||||
{"t":"kick","target":"<memberId>","reason":""}
|
||||
```
|
||||
|
||||
Server→client:
|
||||
Field notes:
|
||||
- `hello.secret`: the server password; a server with an empty secret is open.
|
||||
The member connecting from a loopback address is the host.
|
||||
`hello.media` is carried but not used by the server today.
|
||||
- `user.avatar`: base64 image, at most 256 KiB of characters. `name` is
|
||||
trimmed, empty becomes `?`, cut to 32 characters.
|
||||
- `chat.text` is cut to 500 characters.
|
||||
- `swap.media.url` is the link; `name`, `size`, `duration` are informational.
|
||||
- Frames before `welcome` (other than `hello`) are ignored; unparsable or
|
||||
unknown frames are ignored silently.
|
||||
|
||||
## Server to client
|
||||
|
||||
```
|
||||
{"t":"room",{eventId,transport:"playing"|"held",position,serverTime,
|
||||
mediaId,media:{…}|null,
|
||||
members:[{id,name,avatar,ready,inSet,buffered,stale}]}}
|
||||
{"t":"feed",{actor,action,target?,reason?,at}} toasts + sheet log
|
||||
{"t":"welcome",{token,memberId,mediaUrl?,state}} v2+4, state now a room
|
||||
{"t":"kicked",{reason}} {"t":"err",{message}}
|
||||
{"t":"welcome","token":"...","memberId":"m1","mediaUrl":"...","state":{<room>}}
|
||||
{"t":"room","eventId":7,"transport":"playing"|"held","position":12.3,
|
||||
"serverTime":"<ISO-8601 UTC>","mediaId":1,
|
||||
"media":{name,size,duration,url?}|absent,
|
||||
"members":[{"id","name","avatar","ready","inSet","buffered","stale"}]}
|
||||
{"t":"feed","actor":"","action":"","target":"","reason":"","at":"<ISO-8601>"}
|
||||
{"t":"chat","text":"...","from":"<name>"}
|
||||
{"t":"kicked","reason":""}
|
||||
{"t":"err","message":""}
|
||||
```
|
||||
|
||||
`room` broadcasts at 1 Hz and on every change. Clients diff by
|
||||
`eventId`: position jump ⇒ seek, transport change ⇒ play/pause.
|
||||
`feed` actions: ready, unready, force-ready, force-unready, force-play,
|
||||
force-hold, kick, join, leave, stale, swap, seek, pause, resume, chat —
|
||||
every one carries actor + server timestamp.
|
||||
Force-play-all is `force` with `target:"all",ready:true`; anyone not
|
||||
ready Spaces back. Sounds key off feed actions.
|
||||
Chat dual delivery: the server emits both `{"t":"chat","text","from"}`
|
||||
(render this one — it carries the text) and a textless `feed` chat
|
||||
entry (housekeeping echo). Clients must render exactly one row per
|
||||
send; the reference client drops the textless feed entry.
|
||||
- `welcome.token` is private to its owner and appears only in that member's
|
||||
media URL. `mediaUrl` is omitted when there is no media; it is a built
|
||||
`/m/...` URL for a host file (host part = what the client dialled), otherwise
|
||||
the shared link.
|
||||
`welcome.state` has the same shape as `room`.
|
||||
- `room` is sent every second and on every change. `position` is the value at
|
||||
`serverTime`; clients extrapolate while `transport` is `playing`.
|
||||
`eventId` increases on seek, swap and transport changes.
|
||||
`mediaId` increases on swap.
|
||||
- `feed.action` values: `ready`, `unready`, `force-ready`, `force-unready`,
|
||||
`force-play`, `force-hold`, `kick`, `join`, `leave`, `stale`, `swap`,
|
||||
`seek`, `pause`, `resume`, `chat`. `target` and `reason` are optional;
|
||||
`reason` carries `auto`/`manual` for ready and `silent too long` or
|
||||
`no signal` for leave/stale.
|
||||
- Chat is delivered twice: a `chat` frame with the text and a textless
|
||||
`feed` with `action:"chat"`. Render the `chat` frame only; the reference
|
||||
client drops the textless feed entry.
|
||||
|
||||
## Membership set (kills join-cascade + ghost-quorum)
|
||||
## Join and reconnect
|
||||
|
||||
- Join: `inSet=false, ready=false`. Catch up in background (open media,
|
||||
seek to room edge, buffer). Flipping ready joins the set.
|
||||
- Unready while in set: stays in set, room re-evaluates → HELD.
|
||||
- Disconnect: removed from set, never counts toward quorum.
|
||||
- Swap: all sets cleared, all flags false.
|
||||
- Gate to PLAYING: set non-empty AND every in-set member
|
||||
ready + buffered ≥ 4 s + fresh (not stale) AND quorum: in-set
|
||||
size ≥ 2, unless no more than one member is fresh. A lone
|
||||
watcher can play alone; the moment a second watcher is present,
|
||||
one Space waits for the other. Empty set ⇒ HELD.
|
||||
`hello` then `welcome`, then the app opens `mediaUrl`, seeks to the room
|
||||
position and buffers. A joining member is `ready=false, inSet=false` until
|
||||
the first `ready`. A socket that dies loses its token; reconnect is a fresh
|
||||
`hello`, after which the client re-sends its prior ready flag.
|
||||
|
||||
## Clock rules (Syncplay port)
|
||||
## Reserved, not implemented
|
||||
|
||||
Server owns the advancing clock:
|
||||
`positionAt(now) = position + (playing ? now-serverTime : 0)`.
|
||||
Plain reports are informational and can never move it. Only seek, swap,
|
||||
and gate transitions write position/serverTime/eventId. Reports tagged
|
||||
with `eventId < current` have their position ignored (no ping-pong);
|
||||
buffered is still accepted. Pause (HELD) freezes position immediately;
|
||||
resume sets serverTime=now — laggards seek forward, never drag back.
|
||||
|
||||
Tiered client correction on room state: |drift| > 5 s ⇒ seek;
|
||||
ahead > 1.5 s ⇒ 0.95× until level; inside deadzone ⇒ nothing.
|
||||
|
||||
## Join state machine
|
||||
|
||||
connect → hello → welcome → open media (server mediaUrl or shared
|
||||
url) → seek to room edge → buffer → not-ready catch-up → Space →
|
||||
in set → HELD→PLAYING when gate passes. Reconnect: re-hello (tokens
|
||||
die with the socket), prior flag restored, buffer gate holds playback
|
||||
until truly caught up.
|
||||
|
||||
## Swap state machine
|
||||
|
||||
anyone sends swap → HELD, mediaId++, clock reset (0/now), sets
|
||||
cleared, flags false → all clients re-run Join → auto-resume when the
|
||||
gate passes. No authority check in v1 (same trust basis as force).
|
||||
|
||||
## Join / auth (v2–v3, unchanged)
|
||||
|
||||
proto must equal server proto; secret is the room password; avatar
|
||||
≤64 KiB; failures get `err` + close. `sid="default"`, map keyed for
|
||||
later multi-session. `mediaUrl` carries the playable URL: a built
|
||||
`/m/…` URL when the server holds bytes, otherwise the shared link.
|
||||
|
||||
## Media endpoint (Phase 1, unchanged)
|
||||
|
||||
`GET /m/<sid>/<token>/<name>` Range table. 410/401/403/503 gates.
|
||||
|
||||
## Heartbeat (Phase 2 baseline, feeds the gate)
|
||||
|
||||
5 s WS pings; per-token lastSeen on every frame; silent >15 s ⇒ out
|
||||
of the gate (`stale` feed); silent >30 s ⇒ socket closed, member
|
||||
removed (`leave`), client reconnects via re-hello. No limbo sockets.
|
||||
Closed sockets are swept every second (`isClosed` poll — transport
|
||||
`onDone` is unreliable, so presence never depends on it).
|
||||
Client-side reconnect loop (fixed polite interval + manual escape)
|
||||
is app work.
|
||||
|
||||
## Playlist (deferred, space reserved)
|
||||
|
||||
Not v1. Shape when it lands: server-held `queue:[media…]` alongside
|
||||
the current media; anyone appends links (`queue-add`), anyone skips
|
||||
(`queue-skip`); reaching EOF auto-advances exactly like a swap
|
||||
(HELD, mediaId++, sets cleared, Join rerun, gate resumes). No code
|
||||
until then — `mediaId` + swap already carry it.
|
||||
Server-held queue (`queue-add`, `queue-skip`) would auto-advance like a swap.
|
||||
`RelayConstants.chunkBytes`, `creditWindowChunks`, `maxConcurrentOpen` and the
|
||||
`open`, `ack`, `cancel`, `eof`, `buffered`, `state`, `event` kinds in
|
||||
`MessageKind` are unused leftovers.
|
||||
+87
-77
@@ -1,87 +1,97 @@
|
||||
# ROADMAP — sync_relay_project (backlog tracker)
|
||||
# Roadmap
|
||||
|
||||
This is the durable source of truth for what is built and what is next.
|
||||
Chat is ephemeral; this file + `HANDOFF.md` (root) + `PROTOCOL.md` /
|
||||
`ARCHITECTURE.md` carry state across sessions. Check an item off when it
|
||||
lands, same commit as the code. Proto bumps still require PROTOCOL.md
|
||||
same-commit.
|
||||
Backlog and epics, in order. Done work lives in git history. Check an item
|
||||
off by deleting it in the same commit as the code. Wire changes also edit
|
||||
[PROTOCOL.md](PROTOCOL.md).
|
||||
|
||||
Model: direct-serve (host opens ports, clients open nothing). The
|
||||
brief's old WSS host-upload tunnel is obsolete unless a VPS tier
|
||||
needs remote-host upload. Server = Relic, client WS =
|
||||
web_socket_channel. Protocol still v4.
|
||||
## Backlog
|
||||
|
||||
## Built (phases 0–4 + hardening — done, details in git history)
|
||||
1. **Isolate server + dedicated server.** Replace host detection by loopback
|
||||
peer (breaks behind a same-machine reverse proxy or tunnel) with an explicit
|
||||
host token. Run the server on a Dart isolate
|
||||
when hosting from the app; build the dedicated server as a separate
|
||||
`dart compile exe` of `packages/server` (a Flutter binary cannot be
|
||||
headless); drop `--headless`; rename "room" to "server" in code
|
||||
(`RoomServer`, `RoomHub`, `RoomBroadcast`, wire `t:'room'`, `room` in
|
||||
`/health`, config defaults) with the proto bump that implies; refresh
|
||||
`deploy/` and the packaging.
|
||||
|
||||
Scaffold + CI, mpv Range contract, sessions/tokens/media endpoint,
|
||||
sync engine + quorum, desktop shell (SyncClient, RoomDiff, media_kit,
|
||||
avatars, toasts+sheet, reconnect), presence hardening, frameless
|
||||
chrome, chat/idle fixes, chat double-row fix, avatar 256 KiB cap,
|
||||
Linux tarball packager + Discord upload card, updater v1
|
||||
(`--upgrade`: manifest check, download, sha256, apply-over).
|
||||
WAN-tested 2-client. Counts at last green: core 42, server 41,
|
||||
desktop 17 + 7 updater.
|
||||
2. **Sync fixes.**
|
||||
- Jumpy seek. Root cause: the media_kit seek bar calls `player.seek`
|
||||
continuously while dragging; the coordinator only notices through its
|
||||
1 s position poll (`_detectUserSeek`), so it fights the server echo and
|
||||
the seek-landed retry (`_checkSeekLanded`).
|
||||
- End-of-file stall. The 4 s buffered gate cannot be met in the last 4 s of
|
||||
a file, the server never learns the duration, and there is no finished
|
||||
state. Add duration reporting and a finished state.
|
||||
- Tiny read-ahead. Our own pins (`cache-secs=10`,
|
||||
`demuxer-readahead-secs=10` in `playback_controller.dart`) cap it; want
|
||||
about two minutes for large files.
|
||||
- Remote clock offset is not corrected: clients assume their clock equals
|
||||
the server's.
|
||||
- Stale-ready. A ready member who went stale (no reports for 15 s) should
|
||||
return to the ready set when reports resume; the client re-sends ready as
|
||||
a workaround (remove it then).
|
||||
- Ghost member after an unclean drop until `dropAfter`.
|
||||
|
||||
## Backlog (priority order, each one sitting-sized)
|
||||
3. **Rendering, then shaders.** media_kit falls back to software rendering
|
||||
on the dev machine. Suspect Impeller on Linux against media_kit's GL
|
||||
texture path; investigate first, since shaders need hardware rendering and
|
||||
it may also cause the video freeze. Then shaders: per-viewer local setting
|
||||
using mpv `glsl-shaders`, shipping Anime4K presets (bloc97, MIT).
|
||||
|
||||
1. First real release: `tool/release.sh` (derives the version,
|
||||
writes the manifest, no typing) → review →
|
||||
`git push --follow-tags` → CI (`release.yml`) builds the
|
||||
tarball and publishes the Forgejo release. Verifies the whole
|
||||
`--upgrade` chain end to end. Manual fallback is
|
||||
`tool/package_linux.sh` + `tool/release_publish.sh <tag>`.
|
||||
2. Jumpy seek: live repro with timeline log first, then fix the
|
||||
seek-land retry vs deadband/debounce fight.
|
||||
3. EOF stall (bug, dogfood 09-12): playback stops just before the
|
||||
end and gates HELD ("waiting for X (3.9s)"). Suspect the
|
||||
buffered≥4s readiness gate is unsatisfiable near end of media.
|
||||
4. Tiny buffer (bug, dogfood 09-12): 1–3 s observed; want ~2 min
|
||||
for a 1.4 GB / 24 min file (intro skip + surviving short
|
||||
drops). Investigate demuxer forward-queue props + 64 MB
|
||||
`bufferSize`, then raise.
|
||||
5. Recovery controls: reload-current-media button + restart-player
|
||||
button for when playback wedges.
|
||||
6. Link conveniences: Ctrl+V pastes clipboard into the media
|
||||
field; session link swaps update the remembered media (today
|
||||
only the connect screen persists); session shows the open link
|
||||
with an open-in-browser button.
|
||||
7. Time UI: remaining time until end + scrubber preview (target
|
||||
time while scrubbing).
|
||||
8. Shortcut parity pass with mpv / mpv.net.
|
||||
9. Sounds swap-in: replace `NoopSfxService` (flutter_soloud preload,
|
||||
Kenney/ObsydianX/akx sources in HANDOFF §7); keep seek/pause/
|
||||
resume cues muted or near-silent.
|
||||
10. `/health` auth + stop broadcasting the full media URL (api_key
|
||||
leaks onto the wire).
|
||||
11. File-join wart: welcome mediaUrl advertises 127.0.0.1, remote file
|
||||
joiners fail-open then self-heal on first room broadcast.
|
||||
12. Cache-exhaustion long outage via `tool/shaper.dart --kill-after`.
|
||||
13. Phase 5 chaos suite (host drop 5 s / 30 s, viewer stall, 20-seek
|
||||
storm) + OOM / stuck-room / drift-recovery assertions.
|
||||
14. Windows zip in packaging + Discord card.
|
||||
15. TLS/Caddy doc.
|
||||
16. Chat history pane (schema exists, Activity log only for now).
|
||||
17. Shaders: anime4k upscaling via mpv glsl-shaders + UI toggle.
|
||||
18. Avatar caching + static serving (worth-it decision open):
|
||||
serve avatar bytes once over HTTP by hash instead of
|
||||
base64-in-every-room-broadcast.
|
||||
19. Backfill LAN soak + 1.4 GB test details (record-only, user ran
|
||||
both 09-12).
|
||||
4. **Windows build + CI.** There is no Windows runner. Try
|
||||
`ubuntu2610/flutter_build` (LLVM-MinGW + Wine `gen_snapshot`) on the Linux
|
||||
runner, fallback a GitHub Actions mirror. The manifest needs per-platform
|
||||
URLs and `release_publish.sh` must handle a second platform. The updater's
|
||||
Windows apply path exists but is untested. Desktop tests are not in CI
|
||||
either.
|
||||
|
||||
## Epics (multi-sitting, scheduled whole — not before the backlog
|
||||
friction items above, except Jellyfin which is next feature)
|
||||
5. **Jellyfin** (epic, below).
|
||||
|
||||
- File relay (original brief, second half): links shipped, relay
|
||||
unbuilt. Split into sub-tasks when scheduled.
|
||||
- Jellyfin native — NEXT feature.
|
||||
- Updater expansion: in-app prompt, Windows helper (locked-exe
|
||||
dance), auto-restart after apply.
|
||||
- Playlist (`queue-add`/`queue-skip`, shape reserved in PROTOCOL.md),
|
||||
sidecar subtitles, local-file fallback (viewer plays matching
|
||||
size+duration file locally, still synced), `HttpSource` (host
|
||||
pastes a direct-stream URL, server fetches + re-serves).
|
||||
6. **Smaller items**
|
||||
- Link conveniences: Ctrl+V pastes into the media field; a link swap in
|
||||
session updates the remembered media; show the open link with an
|
||||
open-in-browser button.
|
||||
- Time UI: remaining time, scrubber preview.
|
||||
- mpv / mpv.net shortcut parity.
|
||||
- Sounds: replace `NoopSfxService` (cues already fire). Candidates:
|
||||
Kenney Interface Sounds (CC0), ObsydianX Pack 1 (CC0), akx/Notifications
|
||||
(CC0); playback via flutter_soloud. Keep seek/pause/resume near silent.
|
||||
- Chaos suite: host drop 5 s / 30 s, viewer stall, 20-seek storm, long
|
||||
outage with cache exhaustion (`tool/shaper.dart --kill-after`); assert no
|
||||
OOM, no stuck session, drift recovers.
|
||||
- Chat history pane (today: activity log only).
|
||||
- Avatar caching: serve avatar bytes once by hash instead of base64 in
|
||||
every `room` broadcast.
|
||||
- The `room` broadcast carries the full media URL, which can include an
|
||||
`api_key`.
|
||||
- TLS: document a reverse proxy in front of the server.
|
||||
- Playlist: `queue-add`, `queue-skip`, auto-advance like a swap.
|
||||
- Sidecar subtitles.
|
||||
|
||||
## Deferred (needs explicit approval; transcode is never)
|
||||
## Epic: Jellyfin
|
||||
|
||||
P2P/WebRTC, multi-session, Android. Transcode: NEVER (user decision
|
||||
09-12 — not the age for it anymore).
|
||||
- Each viewer logs in to Jellyfin themselves (Quick Connect or password).
|
||||
- Several Jellyfin servers can be linked, keyed by `ServerId`.
|
||||
- The sync server shares only `{ServerId, ItemId}`. Pasted Jellyfin links are
|
||||
recognised and reduced to that pair.
|
||||
- Viewers with an account direct-play `/Videos/{id}/stream?static=true`
|
||||
with their own token and report Playing / Progress / Stopped, so watched
|
||||
state and Continue Watching update per account.
|
||||
- Guests without an account play through the sync server, which fetches with a
|
||||
credential that stays server-side (the host's own or a configured one).
|
||||
- Next episode: the successor in `/Shows/{seriesId}/Episodes` (NextUp means
|
||||
first unwatched, not next-after-X); audio and subtitle track matching by language, then title,
|
||||
then codec/channels.
|
||||
- Library browser; anyone can put up items.
|
||||
|
||||
Spike facts (Jellyfin 10.11.11): `/Videos/{id}/stream?static=true&mediaSourceId=`
|
||||
serves the original with byte ranges (206). PlaybackInfo gives `PlaySessionId`
|
||||
and DirectPlay. All audio and subtitle tracks are embedded in the mkv. Quick
|
||||
Connect is enabled but `/QuickConnect/Initiate` returned 400: investigate
|
||||
(likely missing client/device fields in the auth header).
|
||||
|
||||
## Deferred
|
||||
|
||||
P2P, Android. Never: transcoding, web clients.
|
||||
@@ -1,70 +0,0 @@
|
||||
# mpv HTTP contract (Phase 1, verified against Relic endpoint)
|
||||
|
||||
Spike: `tool/range_logger.dart` (Relic, `mountMediaRoute` + `onLog` hook).
|
||||
Player: stock mpv CLI headless (`--vo=null --ao=null`). Server logs
|
||||
`METHOD path range=<h> -> status bytes=<n>`.
|
||||
|
||||
## Real traces
|
||||
|
||||
mp4, 570692 B, full play (`--frames=300`):
|
||||
|
||||
```text
|
||||
get /m/file range=- -> 200 bytes=570692
|
||||
get /m/file range=bytes=0- -> 206 bytes=570692
|
||||
```
|
||||
|
||||
One no-Range open (mpv probes full length), then a single open-ended
|
||||
stream to EOF. Small file fits demuxer cache; one connection does it all.
|
||||
|
||||
mkv (ffmpeg remux of same file), 562197 B, play + `--start=8` seek:
|
||||
|
||||
```text
|
||||
get /m/file range=bytes=0- -> 206 bytes=562197
|
||||
get /m/file range=bytes=0- -> 206 bytes=562197
|
||||
get /m/file range=bytes=562093- -> 206 bytes=104
|
||||
get /m/file range=bytes=243694- -> 206 bytes=318503
|
||||
```
|
||||
|
||||
Two `bytes=0-` opens (demuxer probe + playback), a 104 B tail read at
|
||||
`size-104` (Cues-at-EOF probe — mpv expresses it open-ended, not as
|
||||
`bytes=-n`; the server answers both identically), then a mid-file open
|
||||
at a cluster boundary for the seek/start position. Seeks always arrive
|
||||
as a fresh `bytes=N-` connection, never as a mutation of the old one.
|
||||
|
||||
Outage: server SIGINT-killed 5 s mid-play (looping file, comma-form
|
||||
reconnect flags), restarted. mpv kept playing from cache, no error, no
|
||||
reconnect storm; no new Range request inside the post-restart window.
|
||||
Short drops are ridden out from cache — matches the north-star case.
|
||||
Cache-exhaustion reconnect (long drop) is a Phase 5 chaos assertion;
|
||||
`tool/shaper.dart` (KB/s cap + latency + `--kill-after`) exists for it.
|
||||
|
||||
## Server behavior table (all byte-exact tested, 16 server tests)
|
||||
|
||||
| Request | Response |
|
||||
|---|---|
|
||||
| no Range | 200, exact `Content-Length`, full bytes |
|
||||
| `bytes=a-b` | 206, `Content-Range: a-b/size`, exact length |
|
||||
| `bytes=a-` | 206 open-ended to EOF, exact length |
|
||||
| `bytes=-n` | 206 tail `size-n..EOF` (Cues probe) |
|
||||
| unsatisfiable start | 416 + `Content-Range: bytes */size` |
|
||||
| host offline | 503 + `Retry-After: 2` (mpv retries with reconnect flags) |
|
||||
| bad token / kicked / gone | 401 / 403 / 410 |
|
||||
|
||||
Always: `Accept-Ranges: bytes`; `Content-Type` from extension
|
||||
(mkv `video/x-matroska`, mp4 `video/mp4`, webm, avi, mov, ts, mka/mp3).
|
||||
|
||||
## Relic notes
|
||||
|
||||
- Range arrives as typed `RangeHeader` (`req.headers.range`), not a raw
|
||||
string. Multi-range takes the first entry; unit is preserved so
|
||||
non-`bytes` units still 416. `resolveMediaRequest` in
|
||||
`packages/server` stays framework-agnostic and is the tested truth.
|
||||
- Reconnect flag syntax is build-sensitive: this mpv rejected the
|
||||
colon-joined `--stream-lavf-o=a=1:b=1` form and accepted commas.
|
||||
The exact media_kit property set is re-verified in Phase 4 through
|
||||
mpv `set_property`, not CLI flags.
|
||||
|
||||
## Remaining manual (human + media_kit, Phase 4)
|
||||
|
||||
1.4 GB real file through the endpoint with the pinned buffer props;
|
||||
long-outage reconnect with cache exhaustion via shaper.
|
||||
Reference in new issue
Block a user