# Architecture Plain overview first: [HOW-IT-WORKS.md](HOW-IT-WORKS.md). Wire format: [PROTOCOL.md](PROTOCOL.md). Plans: [ROADMAP.md](ROADMAP.md). ## Packages | Path | What | Rule | |---|---|---| | `packages/core` | protocol messages, sync engine (`SyncEngine`, with what's next: `whatsNext`), constants, config, Range parsing, logging, Jellyfin client, updater, the rules every side reads alike (`whats_next.dart`: who puts an Up next item up, who adds the continuation), watched rule (`WatchSession`), the library's L1 (`library/`: release and folder names, `groupLocal`); `testing.dart` (test support for all three packages, not in `core.dart`: `FlakyProxy`) | pure Dart, no Flutter | | `packages/server` | Relic HTTP + WebSocket server (`SyncServer`, `ServerHub`, `Session`, media endpoint, relay buffer, Jellyfin relay), the `serve()` entrypoint, `bin/server.dart` (dedicated server) | pure Dart, no Flutter | | `apps/desktop` | Flutter app: UI, `AppGraph` (composition root), `SyncClient`, `Connection`, `SessionCoordinator`, `Navigation`, `YourServer` (`ServerIsolate`), `MediaPipeline` (opening media), `QueueController` (what's next on this app), `WatchTracker` (watch tracking), `PlaybackController` (the session player's marks and commands), the player host (`PlayerHost`, `MediaKitHost`: the only media_kit user besides the `Video` widget glue), update UI (`UpdateController`) | ships with `starsync-server` | | `tool` | harnesses: `range_logger`, `shaper`, `fake_viewer`, `e2e_check` | dev only | | `deploy` | systemd unit for the dedicated server | | | `third_party/media_kit_video` | upstream media_kit_video 2.0.1 with Starsync's patches (`starsync/README.md`: Flutter ≥ 3.38 EGL display on the platform thread, the X11 GLX step-aside) | vendored; re-apply patches on update | Guard: `packages/core/test/flutter_free_test.dart` fails if anything under `core/lib` imports Flutter, `dart:ui` or `signals_flutter`; CI runs analyze + tests for core and server on every push. Server has no such test but the same rule; CI checks it with `dart compile exe` of `bin/server.dart`. The client WebSocket (`web_socket_channel`) lives in `apps/desktop`; Relic has no client. ## Server lifecycle One entrypoint: `serve(config, events:)` in `packages/server`. `config` is an `AppConfig.toMap()` (primitives only, so it crosses isolates). It starts a `SyncServer` (bound to all IPv4 interfaces, or 127.0.0.1 with an extra `'loopbackOnly': true` key) and reports on the `events` port: `ready` (with the port and a control SendPort), `error` (start failed), and every log record as `log`. On the control port: `{'t':'stop'}` closes all sockets and returns, and the isolate exits; `{'t':'file', 'path', 'reply'}` makes a local file the hosted one and answers its name and size (`ServerIsolate.setFile`: a File put up here then sends a `swap` with that name and size, which the server matches to the file); `{'t':'share', 'port', 'secret', 'jellyfin', 'reply'}` and `{'t':'unshare', 'reply'}` attach and detach the network listener (below). - **Your server** (`session/your_server.dart`, `YourServer`): every app runs one server for its whole life, started when the app starts (`Navigation.start`) as a `ServerIsolate` running `serve`: port 0 (a free one), loopback only, no password, a random `hostKey` per app run, named "'s server", its session log in the app's logs folder and its log records in the app's log as `server.` (see "Logging"). Nobody else can reach it. The isolate runs with `errorsAreFatal: false`: an uncaught async error is logged and the server keeps going. If the isolate dies anyway, `YourServer.crash` is set (the shell says so in a dialog) and, unless you are in a friend's server, the navigation starts a new one and goes home to it. - **Listeners**: `SyncServer.share(port, secret, jellyfin)` attaches a second listener on every IPv4 interface at the port from Settings → Party & network (default 8080) to the running server (Relic serves the same routes on both), sets the password and the Jellyfin relay logins (one per linked account, `relayCredentialsFor`, so guests without an account watch Jellyfin items through the server with your sign-in on the item's own Jellyfin); `unshare` closes it. Nothing restarts for either: members on the loopback listener keep their sockets, the media and Up next stay. Unsharing sends everyone but the host out: each leaves at once (feed `leave`, reason `closed`) and its socket closes with 4016, so its app does not retry (see "Connection"); the password goes back to none. A port another program holds fails the share with its reason; nothing else changes. - **Invite** (`session/invite.dart`): `YourServer.invite()` shares the server if it is not yet and returns an `Invite`: the host, the port, the name, whether a password is needed (never the password), the version and an `fp` field reserved for a pinned certificate (`wss://` with a self-signed one, later; always null). The host is the best of `inviteAddresses` over `NetworkInterface.list`: IPv4 only, no loopback, link-local (169.254/16) or proxy fake-IP (198.18.0.0/15, what TUN proxies like Clash hand out) address and no container or VM bridge (`docker*`, `br-*`, `veth*`, `virbr*`, `vmnet*`, `vboxnet*`, `cni*`, `podman*`, `lxcbr*`, `lxdbr*`, `vEthernet`, VirtualBox, VMware); 192.168 first, then 10/8 and 172.16/12, then a mesh VPN's 100.64/10, then any other (public) address. No public-address lookup: the members card says that friends elsewhere need the port forwarded or a server they can all reach. `Invite.text` is what Invite copies ("Join me on Starsync: 192.168.1.20:8080 (ask me for the password) · Starsync 1.3.0"); `addressIn` reads the address back out of a pasted invite (the Join card's address field). `stopSharing` unshares. - **Where you are** (`session/navigation.dart`, `Navigation`): `place` is `none` (starting, or between two servers), `yours` or `friend`. The app starts at home: your server, joined over 127.0.0.1 with the host key (the `hostKey` stands in for the password, so a password set later never locks this app out), `hostFile` set to `YourServer.setFile`. A join first reads the friend's `/health` (the version gate, below); an unreachable server stops there, still at home. Then this app leaves your server (`bye`) and connects to theirs; a join that fails or is cancelled goes home again. Leaving theirs (Leave), or theirs closing to friends (`ConnRefused` `closed`: "The host stopped sharing their server"), goes home. Your server stays up all the while: what plays there and Up next wait for you, and whoever you invited stays in it. A joined server's name and version (`serverLabel`, `serverVersion`) come from its `/health`; yours is this app's. The Join card's recents are the friends' servers that got in. - **Version gate**: `/health` gives `protoMin`, `protoMax` and `version` (a proto 15 server only `proto`, used for both). `versionFit` against this build's range: ranges that meet are fine; a server whose newest is older than this app's oldest "runs Starsync 1.2.1 — they need to update", one whose oldest is newer "— you need to update", and then the Join card offers "Check for the update" (the updater, see "Updater"). Nothing is dialled for either. A server that says nothing about its version is left to the hello's refusal (see "Connection"). - **Dedicated server**: `bin/server.dart` compiled with `dart compile exe` and shipped next to the app as `starsync-server`. It parses flags (`--port`, `--secret`, `--name`, `--media`, `--advertise-host`, `--jellyfin `, `--[no-]yt-dlp`, `--yt-dlp-dir`, `--yt-dlp-cookies`, `--config`; default port 8080) into the same map and calls `serve` (`--log-dir` only picks where its log files go, see "Logging") (`--upgrade` / `--check-update` run the [updater](#updater) instead); SIGINT and SIGTERM stop it. A Flutter binary cannot run without a display, which is why the server is its own build. - **App close**: after the players and the `bye` (see "Desktop UI", closing), your server stops (`stop`, then killed after 3 s). `SyncServer` runs a 1 s heartbeat (`ServerHub.heartbeat`): closed-socket sweep, the engine's sweep (silence, joining, gone notices, grace), relay source state, warm relay, rates, state + feed broadcast. There is one session, id `default`. Passwords: the default secret is empty, which means an open server (a warning is logged). Otherwise one shared secret, compared at hello; your server's is the one from Settings, set when it is shared. Host: the app generates a random `hostKey` per app run, puts it in your server's config and sends it in its own `hello`. The member whose key matches is the host (`members[].host`: the crown, art rights) and needs no password. The address a connection comes from plays no part (so a reverse proxy or tunnel on the same machine does not make everyone the host). The dedicated server has no key, so it has no host. Nobody removes anyone: members set each other away ("Presence" below), the host included, since your server and its `/m/` file serving do not need the host's player; only stopping sharing sends guests out. ## Composition root `AppGraph` (`app/app_graph.dart`) is built once in `main.dart` (`AppGraph.build`) and constructs and disposes the app's systems: the `Ports` (the player host among them, handed to the coordinator), the sounds (`SynthSfx`), the `Notifier`, the `Connection`, the `SessionCoordinator` (handed the settings and the watch tracker), the connect form and recents (`ConnectStore`, `state/connect_prefs.dart`), `YourServer`, `Navigation`, the library's matches (`ShowMatcher`, "Library", whose `refForPath` it hands the coordinator as `fileRef`, and a `LocalNextFinder` over it as `nextFile`) and watch tracking (`WatchTracker` with the device's `WatchHistory`, "Watch tracking"); it wires them through their constructors (and gives `Profile` the form and the push) and has no logic or signals of its own. Widgets reach it through one `AppScope` InheritedWidget (`AppScope.of(context)`, readable in `initState`); `shutDownApp` gets it from `main` (`closeAppWith`). Tests build their own (`test/test_graph.dart`). The app's remaining `.instance` singletons (settings, Jellyfin accounts, the updater, the profile, Files' folders, yt-dlp, recent links) may only lose uses: `singletons_test` fails when one gains a use or a new one appears. ## Sync model Loosely modeled on Syncplay, with a different readiness design. - **Transport** is `playing`, `held` or `finished`. The server owns the clock: `positionAt(now) = position + (playing ? now - serverTime : 0)`. Reports from clients never move it. Only seek, swap, gate transitions and the end of media write `position`/`serverTime` and bump `eventId`. - **Clock offset**: `serverTime` is on the server's clock. Each client sends a `ping` with its clock every second and estimates the offset NTP style (`ClockSync`: `offset = s - (t0 + t3) / 2` of the sample with the smallest round trip among the last 16). All extrapolation uses `Connection.serverNow()` (= `SyncClient.serverNow()`: local clock plus offset), so a client whose clock is off still lands on the server position. - **Reports**: clients send `position`, `buffered`, `eventId` every 1 s, plus the media `duration` and its `mediaId` once the player knows it. The server stores `buffered` and liveness, and takes the first reported duration for the current media (swap resets it); a report with `eventId` older than current has its position ignored. `buffered` is the current media's only: the client sends 0 until the player holds it (a swap's media still resolving, being fetched by the server or opening), and the server counts a report naming an earlier `mediaId` as 0, so the old media's buffer never passes the gate for the new one. Besides the 1 s tick, a report goes out right after a new media is open and, while HELD, the moment the buffer reaches `need`, so the gate does not wait for the next tick. - **Ready gate** (HELD to PLAYING): the members that count for the gate (`SyncEngine.countsForGate`: in the set, no player error, and their presence counts, see "Presence") are not empty and every one is ready and has `buffered >= need`. `need` is 4 s (`minBufferSecs`), or near the end what is left minus 2 s (`endSlackSecs`, floor 0), since mpv cannot buffer past the end. So a lone viewer plays alone; with others present everyone must ready. While HELD, `state.hold` says why (`HoldReason`: a joiner within its bound, someone unready, a short buffer, the same with the relay's source gone, or nobody to start). - **In-set** (`EngineMember.inSet`, who the gate may wait for): a member joins it on joining, or on coming back from `gone` or `away`, while not PLAYING (any media, the one the server started with included); joining while PLAYING it stays out (`ready=false`) until it readies. A swap puts every member in the set and keeps everyone's ready flag, so the next video starts once everyone ready has buffered (the client setting "Unready when the video changes", `AppSettings.unreadyOnSwap`, default off, sends `ready=false` itself on a new `mediaId`; not for the first state of a session, and a reconnect does not restore a ready that a swap meanwhile would have dropped). A player error removes from the set; presence decides whether a member in the set counts. - **Identity**: an install sends a random client id (`hello.client`, private: never broadcast, never logged) and a `seq` that rises with every connection. The server maps the client id to one member for its life: the same id and token on every reconnect (`welcome.resumed`), so the member keeps its ready flag, presence, queue items (`by`), the relay it serves and its picture. A hello takes the member over when its `seq` is not older than the member's; the older socket gets `replaced` and closes, an older hello gets `stale` (so two windows of one install cannot fight: the newest wins, the client stops on `replaced`). `welcome.instance` changes per server boot, so a client tells "the server restarted" from "my grace ran out" (both: `resumed` false). - **Presence** (`Presence`, `members[].presence`; D1, D2): `joining` -> `in`, `away`, `gone`. What each counts for is one table, `PresencePolicy.countsFor(presence, Purpose)` in core (`presence.dart`), shared by the server and the app: `in` counts for the gate, the ready counter, watched tracking, preloads and social notifications; `joining` for all but tracking, and for the gate only while HELD and for at most `joinerHold` (15 s) after it started joining, never while PLAYING; `away` and `gone` for none. - `joining`: a new member, one back from `gone`, or one back `in` from `away`, while media is up (with nothing up it is `in` at once). It loads in the background; a report that holds the current media at the server position (within the 5 s seek threshold) with `need` buffered makes it `in`, already in sync, so a joiner during playback never pauses anyone (D2). A report error or `joiningTimeout` (60 s) ends joining too. Coming `in` while PLAYING it stays out of the set until it is ready (its own auto-ready follows its buffer); while HELD it keeps its place in the set. While HELD a joiner holds the start at most 15 s (bounded hold), then the others go on without it. - `away` (sat out): anyone sets anyone away (`presence` `away`, the host included); only the member itself comes back (`in`, through `joining`). Its ready flag and reports are kept but never count; `force` leaves it alone; a swap leaves it away. Away survives a drop (back from `gone` it is `away` again). - `gone`: its socket closed or fell silent. Kept for the grace (`Liveness.grace`, 10 min) with everything it had, its token working (`/m` `/r` `/j` URLs, the relay channel); out of the gate. The feed says `gone` only after `goneNotice` (10 s), so a quick reconnect is silent; back after that line it says `return`. After the grace it leaves (feed `leave`, reason `grace`); `bye` leaves at once (reason `bye`). Only a hello brings it back; frames of a socket that no longer holds the member are ignored. - **Player error** (`EngineMember.error`, `report.error`, `state.members[].error`): a member whose player cannot open the current media says why in its reports; the server takes it out of the set at once (like away, but automatic: `ready`/`force` do not put it back, its ready flag is kept), so the others start without it; it also ends a `joining`. A report without it (the media opened, after a reload too) or a swap clears it: back in the set like a join while not PLAYING; while PLAYING at once if it is ready (the others then hold for its buffer), else from its `ready`. - **Pause**: while PLAYING, any member counting for the gate who becomes not ready or under `need` buffered flips the server to HELD (feed action `pause` by `server`). Recovery resumes by itself when the gate passes. - **End**: while PLAYING with a known duration, reaching it sets FINISHED (feed `finished` by `server`, position = duration). Clients pause where they are, except one within 3 s of the end (`finishSlackSecs`), which plays out to mpv's own stop; at the end the position shows the full duration. FINISHED never auto-resumes. A seek turns it into HELD at the new position (resuming at once if the gate passes); a swap resets to HELD. - **Seek**: allowed from any member in any transport; the server writes the position, bumps `eventId` and records who sent it (`seekBy`, `seekSeq` in `state` for that event); flags are untouched. Clients diff each `state` broadcast by `eventId` (`StateDiff`): new event means seek to the computed server position, transport change means play/pause. The coordinator applies states one at a time, newest first (a state arriving meanwhile replaces any still waiting): interleaved applies could land an older pause after a newer play. Between events: drift over 5 s for more than 2 s seeks, ahead by more than 1.5 s sets rate 0.95 until level, otherwise nothing. - **User seeks**: a hosted player's public `seek` is the user's (its `syncSeek` moves mpv): the session's seek bar (`ChapterSeekBar`, one `seek` per pointer move while dragging and one on release, like media_kit's), the arrow keys, skips, chapter jumps and `seekTo` all become explicit events. The player moves at once (scrub preview); `UserSeekGate` sends one `seek` intent with a fresh `seq` after 300 ms without another request (`seekDebounceMs`). While the user is seeking, and until the echo of the last sent seq arrives (3 s at most), server seeks never touch the player; the echo of our own seek is not applied either (the player is already there). A server seek that has not landed after 1.5 s is re-applied once (`_checkSeekLanded`), to the server position recomputed now, and only while its event is still current, the user is not seeking and the relay it reads has its source ("Viewer recovery" under "Media pipeline"). - **Liveness**: WS ping every 5 s; a socket without any frame for 15 s (`silence`) is closed (4015) and its member goes `gone`; the grace, the `gone` notice, the joiner bound and the joining timeout follow. The durations are a `Liveness` (core, `sync_engine.dart`; defaults `RelayConstants.silenceSecs` / `graceSecs` / `goneNoticeSecs` / `joinerHoldSecs` / `joiningTimeoutSecs` / `pingIntervalSecs`) given to `SyncEngine(liveness:)` and `SyncServer(liveness:)`; all but the ping are measured on the engine's clock (`SyncEngine.clock`, `SyncServer(clock:)`) by the 1 s heartbeat sweep, so a test with a fake clock steps through them; the ping is the socket's own real timer. Any frame counts (`SyncEngine.touch`). Closed sockets are polled with `isClosed` every second: Relic's server-side `onDone` does not fire for dead peers, so presence never relies on it. A half-open link is caught by the silence. - **Client policies**: a drop is not the end of the session. The app keeps the last state (`SessionCoordinator.stateStale` while it reconnects or after the connection ended; the strip says "reconnecting…"), the player and its media, and a relay it serves; it reconnects with backoff and jitter and comes back as the same member (see "Connection" below). No frame for 3 s is "connection unstable", 9 s is lost (well under the server's 15 s). A welcome that resumed needs nothing re-sent; a new member (grace over, server restarted) gets the app's ready and sitting out again, and the media opens again only when it changed (or its URL named the old token). A restarted server that has no media while this app had some gets it back ("Server restart" below). The app's own `ping` every second keeps it under the server's silence limit even with nothing playing. Cancel during setup aborts the join. Leave and closing the window send `bye` (see "App close" under Desktop UI); Update and restart does not. - **Server restart** (the queue stays in memory, the server stores nothing): a reconnect that finds another `welcome.instance` with no media, while this app had media up, holds it: the player plays on and states without media are not applied (the watch goes on with it, see "Watch tracking"). One app offers it back (`swap.restore`: the media, this player's position, whether it played); who goes first is read from the old state alone, so the apps agree: who put it up or passes it on after 0.5 s, the others by their old member id from 3 s on, 1.5 s apart, each only while the server is still empty and only what it can put up again (`_offerBack`: a Jellyfin item with its own account, its file, the link it put up, a link everyone opens). The server keeps the first offer. When it is back as the same media under a new `mediaId`, the player keeps it unless its URL named the old token (a relay viewer's `/r/`, the host's `/m/`: opened again), and what this member passes on is served again (`MediaPipeline.resumeAs`: its file, its link, its Jellyfin item for viewers without an account); the watch, the Jellyfin reports and a cleared continuation carry over. Up next comes back too (`QueueController.restoreAfterRestart`): the app whose offer won adds the links everyone opens and the Jellyfin items, every app its own links passed on and files; a server-fetched link is lost with the old server. Nobody offering within 30 s: the app lets go of it. - **Auto sit-out** (`AutoSitOut`, `session/auto_sit_out.dart`; Settings "Auto sit-out", `AppSettings.autoSitOut`, default on): not playing and no input for 5 min sets this member `away` (reason `idle`); minimised or hidden for 10 min does, playing or not (`minimised`): while playing idle input does not count, and watching is activity, so a pause after a long watch starts a fresh clock. Input (any key or pointer use in the window, `markInput`, from `AppShell` and the controls) or the window's focus brings it back `in` (without touching the ready flag), but only from an away it set itself: a manual one (own or someone else's) stays. Never when alone (`isAlone` in core: no other member connected, `in`, `joining` or `away`). The window changes come from `ShellPort.watchActivity` (focused, unfocused, minimised, hidden); the timings are `AutoSitOutTimes`. - **Nudge**: `nudge` names a member; the server passes it to that member only (no feed), at most 3 per second and 20 per minute per sender (`nudgesPerSec`, `nudgesPerMin`; the rest is dropped silently). The nudged app plays the nudge sound whatever its per-sound switch (the volume still applies), toasts "Alice nudged you" and asks the desktop for attention when its window is not focused: the runners' `starsync/window` channel (`attention`), the urgency hint on Linux (GTK, cleared on focus) and `FlashWindowEx` until foreground on Windows. - **Deviations from Syncplay**: clock never follows the slowest watcher; no pause-on-leave; anyone can force ready/unready (all or one); presence gate; the clock is server time (clients correct for their offset to it). Additive wire changes only; bump `proto` and PROTOCOL.md in the same commit. ## Connection `Connection` (`session/connection.dart`) is the link to the server, made in the composition root (`AppGraph`) and passed to the `SessionCoordinator`. It owns the `SyncClient` socket, the first connect (`begin` + `dial`), reconnects (`delayFor`: 1 s, doubling per failed attempt up to 15 s, each ±20 % jitter), the epoch (`invalidate`, `isCurrent`) that makes a cancelled connect or attempt stale, the 1 s heartbeat (clock `ping`, the diagnostics tick, the silence check), the diagnostics uploader (`diag`, started with the session, stopped by `close` before the socket), "Stop reconnecting" (`cancelReconnect`), "Retry" (`retry`) and setup cancel (`cancelConnect`). Each hello is built by an injectable `HelloBuilder` (`defaultHello`): this build's version range (`RelayConstants.protoMin` .. `protoMax`), the install's `ClientIdentity` (`session/identity.dart`: a random 22 character base64url client id and a `seq` that rises with every connection, both in the settings box, so per data folder and per `STARSYNC_PROFILE`; tests get a fresh in-memory one per `Connection`), `app` (the app version), `platform`, `caps` (source always; resolve when the app's yt-dlp is there; jellyfin when an account is linked, checked by the coordinator at each connect) and, on a reconnect to a server seen before, `feedAfter` (the last `feed.seq`). The replay (`welcome.feed`) reaches the activity log only, marked `replayed`; a welcome of another `instance` (a restarted server) drops it and counts from 1 again. `Connection.state` (a signal of the sealed `ConnState`): `ConnClosed` (before a connect, after leave or a cancel), `ConnConnecting` (the first dial), `ConnOnline` (welcome received, frames arrive), `ConnUnstable` (the same socket, nothing for `unstableAfter`, 3 s), `ConnLost` (the socket closed or errored, or nothing for `silenceLimit`, 9 s: the socket is dropped, the next attempt scheduled), `ConnReconnecting(attempt)` (an attempt dialling, and after it failed until the next one), `ConnRefused(code, message)` (`version`, `secret`, `closed` for close 4016, the host stopped sharing their server, or empty for a first dial that failed otherwise; nothing retries) and `ConnReplaced` (`err` `replaced`, close 4013 or 4014: another window of this install holds the member; nothing retries, "Retry" takes it back). The silence and the lost sound are measured on `Connection.now` (injectable, UTC wall clock by default); the heartbeat and the reconnect delay are real timers. Refusals in plain words (`connectErrorText`, `versionText`): `version` says whether the host runs an older or a newer Starsync (from the server's `min`/`max`; a proto 15 server's codeless `proto mismatch` is always an older host), `secret` a wrong password, `replaced` "Starsync was opened in another window". Close code 4015 (the server found it silent) and any other drop reconnect. The state is the first input of the session's status (see "App events and platform ports": "Connecting…", "Reconnecting…" with Cancel, "Connection unstable…", the refusal or "replaced" with Retry; the strip's "reconnecting…"); the coordinator also reads it as `reconnecting` (lost or reconnecting), `unstable` and `stateStale`. A drop that lasted `lostSoundAfter` (4 s; a silent link counts from its last frame) raises `ConnectionLost`, the reconnect that follows `ConnectionBack` (each once per drop, through the Notifier): a quick flap makes no sound. The session's part of a welcome is the coordinator's `onWelcome` handler (`_onWelcome`), called before the welcome's state goes out: the first welcome and a resumed one need nothing; a new member after a drop gets the app's intent again (`ready`, `presence` `away`) and the media is forgotten (opened again from the state) only when it changed. The incoming streams (`states`, `feeds`, `nudges`, `errors`), the send methods, the `token`, `memberId` and `serverNow()` are read through it. `close(bye:)` sends `bye` and waits briefly for the server's close, so the frame goes out before the socket does. ## Media Three sources, same sync. Any of them can also go through the relay (below). - **Link**: the media field holds an http(s) URL. It travels as `media.url` in `swap` and `state`, and as `mediaUrl` in `welcome`. Each app plays it directly (a web page through its own yt-dlp, see "yt-dlp"); the server is only the clock. A pasted link that `parseJellyfinLink` recognises goes up as a Jellyfin ref instead when the ServerId is known (in the link, else `/System/Info/Public`), so its `api_key` is never shared (`mediaForLink`, host and swap alike). - **Host local file**: `SyncServer.setLocalFile` / `mediaPath` makes the file the session media. `GET /m///` serves it with a Range table, gated by the viewer's token. The server builds the URL per viewer (`http://:/m/default//`), so tokens never reach other viewers. `` is the authority the client dialled; `advertiseHost` (default empty) only overrides it when set explicitly. When a `state` broadcast has no `media.url`, the client builds the URL itself from the host it connected to. Bytes stream file to socket and are counted as streamed (`bytesServed`); nothing is stored. The host plays through loopback like everyone else. - **Jellyfin item**: the media is a ref (`{serverId, itemId, ...}`, wire in [PROTOCOL.md](PROTOCOL.md) "Jellyfin media"); the server never stores or sends a URL or token for it. Each app resolves it with its own account and direct-plays the static stream; guests use the server's own login when it has one ("Jellyfin relay"), else the member who put it up ("Relay"). Range table (`resolveMediaRequest`, tested byte-exact): | Request | Response | |---|---| | no Range | 200, `Content-Length` = size | | `bytes=a-b` | 206, `Content-Range: bytes a-b/size` | | `bytes=a-` | 206 to EOF | | `bytes=-n` | 206 last n bytes | | unsatisfiable | 416, `Content-Range: bytes */size` | | host has no file | 503 + `Retry-After: 2` | | bad / unknown token, unknown session | 401, 410 | Always `Accept-Ranges: bytes`; `Content-Type` from the extension (`contentTypeForFile`). Relic hands the Range over as a typed `RangeHeader`; the endpoint takes the first range and keeps the unit, so non-`bytes` units still get 416. What mpv does against it (recorded with `tool/range_logger.dart`): mp4 opens once with no Range then streams `bytes=0-` to the end; mkv opens `bytes=0-` twice (probe, playback), reads a small tail at `size-n` (cues, expressed open-ended), then every seek is a fresh `bytes=N-` connection. A server outage of a few seconds is ridden out from mpv's cache with no error. Longer outages exhaust the cache and rely on reconnect (not chaos-tested yet). mpv settings (the player host, `platform/desktop/media_kit_host.dart`, set on every player before its first open): `Player` with `bufferSize` 256 MiB (media_kit's `bufferSize` sets both `demuxer-max-bytes` and `demuxer-max-back-bytes`), `libass` on; pinned via runtime property: `demuxer-readahead-secs=150`, `cache-secs=150` (about two minutes of read-ahead: rides out short drops, intro skips are instant), `demuxer-max-bytes=268435456` (150 s up to ~14 Mbit/s; a 1.4 GB / 24 min file is ~8), `demuxer-max-back-bytes=67108864`, `stream-lavf-o=reconnect_streamed=1,reconnect_on_http_error=5xx,reconnect_delay_max=4,reconnect_max_retries=3` (comma form; this mpv build rejects colon-joined): a dropped connection or a 5xx is retried, at most 3 times per read and 4 s apart at most; a 4xx (a CDN refusing a request) fails at once instead of being retried (older ffmpeg ignores `reconnect_max_retries`), `ytdl=no` (pages are resolved by the app, never by mpv's Lua hook). A load that fails (mpv's fatal "No video or audio streams selected", or an EDL track that cannot be opened) before a duration is known is reported once per open (`onLoadError`), with ffmpeg's HTTP status when this player gets ffmpeg's log (ffmpeg logs through one mpv instance per process: the first one, and once that is disposed the next one created; the host passes the HTTP errors any player gets on as `httpErrors`). Since every media gets its own player, the session player is often not that instance: the status is a bonus, load errors are detected either way. That, a page's resolve error (yt-dlp's words and the cookies it used, D11, see "yt-dlp") and a failed open become the coordinator's `playerError` in plain words (`playErrorText`, core `link_probe.dart`: 412/precondition/login/cookies/sign in → "This site needs cookies — Settings → Player → yt-dlp → Cookies", 403 → "Not allowed (403)", 404/410 → "Not found (404)", "Unsupported URL" → "Not a video page yt-dlp knows", network errors → "Couldn't reach the site", anything else as is; `withCookieNote` adds the cookies to the two sign-in texts). It rides in `report.error` (see "Player error" in the sync model) until the media opens (a new duration) or the server's `mediaId` moves on; the pill shows "Can't play this: ..." with Retry (Reload video). The cache is in RAM: `cache-on-disk=no` is pinned (media_kit sets it on without a `cache-dir`, so mpv fell back to RAM anyway, warning "Failed to create file cache." on every open). mpv ignores invalid values silently, so every pin is read back and a mismatch is logged and kept in `pinMismatches`. Measured against the `/m` endpoint (`readahead_e2e_test`): a paused player buffers ~147 s ahead (10 s before). media_kit has no public `setProperty`/`getProperty`; the host calls them through a guarded dynamic call on `player.platform` (`HostedPlayer`). Hardware decoding comes from Settings (auto, off, custom) and is passed to the `VideoController`; env `SYNC_RELAY_HWDEC` (e.g. `no`) overrides it. Auto is mpv `auto` on Linux and `auto-copy` on Windows (`autoHwdec`, also for a blank custom value): media_kit renders through ANGLE there, where `auto` picks the dxva2 interop, which cannot create its EGL surface ("Failed to create EGL surface"); `auto-copy` decodes on the GPU (d3d11va-copy, dxva2-copy, ... whichever works) and copies frames back, needing no interop. It is read per player, so a change applies on the next media, join or Reload video. Shaders (per viewer, local only, no wire change): mpv `glsl-shaders`, set through the same dynamic `setProperty` and read back (`PlaybackController.applyShaders`). The value is the platform path list (`:` joined, `;` on Windows). The read-back is compared as a list (`shaderListMatches`): older mpv (0.37 and before, the bundled Windows libmpv among them) prints it `,` joined. One saved choice in `AppSettings` (`shaderPreset`, `shaderQuality`, `customShaders`): applied on every player creation (join, new media, Reload video) and live when changed from the session (the controls' ⚙ → Upscaling shader menu, Ctrl+0..6, both also save it). The Anime4K v4.0.1 files live in `apps/desktop/assets/shaders/anime4k/` (only the 14 the presets use, plus its MIT LICENSE); mpv needs real paths, so `extractAnime4k` copies them once per version to `/shaders/anime4k-/` (a `.complete` marker is written last). `anime4kFiles` maps each mode to the file list of Anime4K's mpv templates: HQ = the high-end GPU set (VL + M networks), Fast = the low-end set (M + S). A missing file, a path containing the list separator, a read-back mismatch, or an mpv `libmpv_render` error while shaders are on turns shaders off (saved) with a toast. Render mode: media_kit's Linux/Windows video output creates mpv's render context (OpenGL, else its software renderer) when the `VideoController` is created, and nothing in its API says which. The host therefore requests mpv logs at `v` while the first output of an app start is created (`PlayerHost.video(probe: true)`): OpenGL init logs `GL_VERSION` under `libmpv_render` (`RenderMode.gpu`; "suspected software renderer" means llvmpipe, `softwareGl`); none within 1.5 s of the controller being ready means the software path (`software`, where shaders do nothing; the shader toast, the menu and Settings say so). Into the app log (`mpv`, INFO, all starting `render`): once per app start on Linux `render env:` (`XDG_SESSION_TYPE`, `WAYLAND_DISPLAY`, `DISPLAY`, `GDK_BACKEND`, `FLUTTER_LINUX_RENDERER`, `LIBGL_ALWAYS_SOFTWARE`, `__EGL_VENDOR_LIBRARY_FILENAMES`, `-` when unset); from that `v` window only mpv's GL init lines (`GL_VENDOR`, `GL_RENDERER`, `GL_VERSION`, `GL_SHADING_LANGUAGE_VERSION`, `Detected ...`); the mode decided at the end of it; and per player, at its first frame, `hwdec-current` and the `hwdec` asked for (the first one per app start also `mpv-version` and `ffmpeg-version`). Why media_kit fell back to software it only prints on stderr (`media_kit: VideoOutput: ...`), which the app does not capture. Otherwise a session player logs at `warn` (`sessionLogLevel`; the host switches levels with `mpv_request_log_messages` on media_kit's handle, the only raw mpv call in the app): media_kit drains mpv's log on the UI thread, one message at a time, so a chatty level, or ffmpeg logging every retry of a refused request, could starve the UI. Shader errors are `error` lines and still arrive. The host writes every session and preload player's warnings and errors to the app log (`mpv`, a few per kind at a time, `preload ` before the prepared player's). Thumbnail players log nothing. The hardware path is fragile: media_kit borrows whatever EGL context GDK left current on the main thread (upstream media-kit#1404), so an X11 session or a Flutter/GDK change can silently fall back to software. **Player host** (`PlayerHost`, `platform/ports/player_host.dart`; the desktop's `MediaKitHost`, one per app in `Ports.players`, tests make their own): creates players by role (`session`: the full cache, `warn`; `preload`: the prepared one, 48 MiB ahead, muted, `error`; `thumbnail`: no outputs, no cache, no log) with every pin set before the caller's first command (`pinMismatches` lists the read-backs that differ), gives a player its video output (`HostedVideo`, with the hwdec policy: `SYNC_RELAY_HWDEC`, else the setting, `auto-copy` on Windows), runs the render-mode probe and its diagnostics (`renderMode`, also after leaving), routes mpv's log, and owns the order players go in: `retire` (pause, let the view drop the texture: two frames and a route pop, then dispose; a new session player waits for the last one, so at most one old player lingers), `dispose` (one no view showed) and `disposeAll` (app close: every live player, each bounded). A `HostedPlayer` is one mpv: state and streams (position, duration, buffer, volume, completed, width, log), open, play/pause/stop, `seek` (the user's, see "User seeks") and `syncSeek`, rate, volume, properties, commands, `pin` (set and read back), its log level. Nothing else in the app imports media_kit except the `Video` widget glue (`session_player.dart`, `videoControllerOf`). **Media pipeline** (`MediaPipeline`, `session/media_pipeline.dart`, owned by the coordinator): the one way to open media, plan → resolve → open → ready. - Plan (`planMedia`, `session/media_plan.dart`, pure; `media_plan_test` holds the table): from `state.media`, what this member serves (its `MediaOrigin`: a file or a link it passes on, a Jellyfin item it put up; set at its swap, claimed by the state that names it the relay's source), its own capabilities and who is present, where this app reads the media from (`SourcePlan`): your server's file (`/m`), a link everyone opens (`DirectLink`; a page is resolved here), another member's relay or the server's fetch (`/r`, `waiting` until it is fed: `relayWaitsForSource`), its own file or link (it plays its own copy and feeds the relay), a Jellyfin item (`JellyfinSource`: own account, the server's login `/j`, the member passing it on `/r`), or `Unresolvable` (a page on a device that cannot resolve: `LinkResolver.resolvesLinks` false; who can is read from `members[].caps.resolve`; desktop always can, so this is Android's plug-in point). - Resolve: a Jellyfin item through this app's account (or the plan resolved ahead for Up next, `warmJellyfin`), else the server's login, else the member passing it on, else blocked (`openJellyfin`); a web page through the app's own yt-dlp (`LinkResolver.probe`: yt-dlp's stream URLs, an EDL of video and audio, the CDN's headers with its cookies, the subtitles; see "yt-dlp"), failing in plain words with the cookies that were used (`ResolveFailure`, D11); a relay not fed yet is opened once `relay.live`. - Open: a player per media through the host. The prepared player (Up next's, see "Getting ready") is promoted when it holds the same URL and headers; else, when the session player already held media, a fresh one takes over (pins, volume, shaders; with the view up and the render mode known, its video output ready before the view swaps to it, else the view makes it and the probe runs on it); the same media again (a reconnect, joining back) keeps the player. Every hand-over is `PlaybackController.attach`: log, marks and track observers move over, the view swaps to the new output (`generation`), volume and shaders carry over, the old player is retired. The first open after join or Reload video uses the player just created; a dropped connection never touches the player. "Reload video" (button or Ctrl+R) retires the player, starts a fresh one and opens the media again (a page resolved afresh: its stream URLs may have expired). Cost (`SwitchTrace` `player`, headless, without the video output): 45 to 65 ms per swap for one app in the process, about 150 ms with three in one process (`queue_switch_e2e_test` with `WT_SWITCH_PRELOAD=0`), against a cold switch of about 1.6 s. - Ready: the coordinator reports at once after an open (`onOpened`) and when the buffer reaches the gate's need while held, not on the next tick. - Commands: play/pause/seek/rate come only from `StateDiff` actions, the play gate (`localPlayAllowed`: self in-set and ready) or the user's own seeks (see Sync model); a server seek within 0.5 s of the current position is skipped. They go through `PlaybackController.apply`: while a switch is under way (no player, or its media has no duration yet) they wait, the latest seek, transport and rate apply once to the new player when its media is in; mpv never gets a seek it cannot run (media_kit logged "error running command seek" / `event:5` for each). The volume is the controller's (`setVolume`), on every next player. Opens are paused (`play: false`). On media change (`mediaId` bump) the previous media stops at once (no old picture or sound while the new one resolves, is fetched or opens) and `StateDiff` seeks to 0, holds and resets rate. Leaving stops and releases the players. - Being a relay source (`RelayFeeder`, `session/relay_feeder.dart`): the current media's channel and Up next's warm relay (see "Relay"), with the step-1 re-dial (a source rides through this app's own drop). A Jellyfin item this member put up is passed on only once someone present needs it (`jellyfinRelayNeeded`: the server's login does not cover it and a present member does not play that server with its own account; one whose `caps` say Jellyfin but whose `jf` is not reported yet is asked again with the next state), so a session where everyone has an account opens no relay channel. - Viewer recovery (D6): while the relay this app reads has no source (`sourceGone`, or not live with the source away), it plays what is buffered. A seek past that (a server seek, a drift correction) is tried once; more wait, the seek check does not re-apply, and the report says nothing is buffered once this player cannot be where the server is (`starved`), so the server holds everyone with the reason `source` ("Waiting for Bob"; see "App events and platform ports"). When the source is back, one seek to the server position resumes the player, and a player that never loaded meanwhile (it opened while the source was gone and its first bytes never came) opens again in a fresh player. Measured (`relay_e2e`, a source just above the bitrate that drops, a seek past what the server got, a member joining meanwhile): at most one seek per viewer while it is gone, nobody's player broken, all play from the seek once it is back. Chapters, segments, tracks (`PlaybackController` signals, values from `playback/media_marks.dart`, no widgets; any controls UI reads them): - `chapters`: mpv `chapter-list` (observed through the same dynamic `observeProperty` and read as JSON): `MediaChapter` index, title, start; end = next start, the last one's = `duration`. - `segments`: the Jellyfin item's MediaSegments (`GET /MediaSegments/{id}`, 10.10+: Intro, Outro, Recap, Preview, Commercial), fetched by the coordinator with `OwnPlay` only; relay and blocked viewers cannot call Jellyfin and use chapters alone. Empty without a segment provider plugin (the case on the owner's 10.11 server). - `tracks`: audio and subtitle `MpvTrack`s from mpv `track-list` (id, language, title, codec, channels, default / forced / hearing-impaired / external, `selected`), re-read after every selection. - `position` (250 ms steps), `duration`, `buffered` (end of the cached range: mpv `demuxer-cache-time`, media_kit's `buffer`, 250 ms steps); computed `opening`, `ending` (`SkipRange`) and `currentChapter`. Skip rules (pure, `media_marks.dart`): - Opening = first Intro segment, else the first chapter whose title has a whole word OP / Opening / Intro (any case, digits allowed: "OP1"). Ending = first Outro segment, else ED / Ending / Credits / Outro. - `skipOfferAt` (the coordinator's `skipOffer`, null at `finished`): inside the opening (up to its last second) "Skip OP" to its end; inside the ending "Skip ED" to its end only (no guessing about previews: the viewer decides). Without any opening, without chapters (a chaptered file without an OP chapter has none on purpose) and with media over 20 min, "Skip 1:30" during the first 6 min. - +1:30 (`skipForward`: the pill, ⚙ → Skip 1:30, Shift+Right) is a `nudge`, so repeated presses add up. - Chapter steps (PgUp / PgDn, `chapterStepTarget`): forward to the next start; back to the current start, or the previous one within 5 s of it (mpv's `chapter-seek-threshold`). - Every skip and chapter jump is a user seek (debounced, everyone follows). Never unmount GPU-textured widgets (avatar images) on a transport-gated branch: teardown races mpv's frame upload and the window goes black. Keep them mounted and skip painting (`Visibility` with `maintainState`). ## Jellyfin client `packages/core/lib/src/jellyfin/`, pure Dart over `dart:io` `HttpClient` behind a `JellyfinHttp` interface (tests inject a fake). Plans: [ROADMAP.md](ROADMAP.md) "Epic: Jellyfin". - `JellyfinClient`: one server + one user. Auth (Quick Connect, password, `/Users/Me` check, logout), browsing (views, Resume, NextUp, Latest, paged `/Items` with `DateCreated`, search, seasons, episodes, recently played), image URLs (tag + maxWidth, no token needed), PlaybackInfo, stream URLs, MediaSegments (a 404 = none), Playing / Progress / Stopped reports, played / unplayed / resume position. - `normalizeServerUrl`, `parseJellyfinLink`: pasted addresses and item links -> `(baseUrl, itemId)`; accounts are keyed by `ServerId` (`/System/Info/Public`). - `episodeAfter`: next episode is the successor in `/Shows/{id}/Episodes` (crosses seasons, skips other copies of the same episode); NextUp means first unwatched, not next-after-X. - `matchTrack` / `matchTracks`: carry audio/subtitle choice to the next episode (language required, then title words, forced/HI flags, codec, channels). Gotchas (Jellyfin 10.11): - Every request sends `Authorization: MediaBrowser Client, Device, DeviceId, Version[, Token]` (values percent-encoded). Unauthenticated calls missing a field get 400 with no body. - A POST must carry `Content-Length` even when empty: behind nginx a body-less POST without it gets 400 (this was the Quick Connect Initiate failure). - Jellyfin overwrites a token's stored device name and app version with the header's. Only use tokens this app obtained itself; a relay using a viewer's token must send that viewer's client fields (the server relay sends exactly its credential's fields). - Static streams (`/Videos/{id}/stream?static=true`) serve the original file with byte ranges. The share URL has no credentials; each side adds its own `api_key` (`authorizeUrl`). - Ticks: 10,000,000 per second. ## Jellyfin relay For viewers without an account on the item's Jellyfin (`packages/server/lib/src/jellyfin_relay.dart`). The server holds its logins as `JellyfinCredential`s (`ServerHub.jellyfinLogins`, at most one per Jellyfin `serverId`; each: `baseUrl`, `serverId`, `token`, `client`, `device`, `deviceId`, `version`; primitives, so they cross the isolate boundary). The app sends one per linked account (not known expired, the default first) when your server is shared (`share`): its own app-obtained tokens and their device fields; the dedicated server has one, read from `--jellyfin ` (YAML/JSON with those keys, keeps the token off the command line) or a `jellyfin:` map in `--config`. An empty `serverId` is looked up at start (`/System/Info/Public`); if that fails that login stays unused. Tokens are never logged (`toString` and config parse errors leave them out). - Available (`state.relay`) only while the current media is a ref on a server one of the logins is for (`loginFor(serverId)`, which `/j` then uses); `state.relayNext` likewise for Up next's first item, which `/j` also serves so guests can preload it (see "Getting ready" under "Up next"). Only those two items are relayed, only to a valid viewer token: no open proxy. - Each request opens `/Videos/{id}/stream?static=true&mediaSourceId=` with `Authorization: MediaBrowser` built from exactly the configured fields (Jellyfin rewrites a token's device fields from the header), forwards `Range` untouched and `Accept-Encoding: identity`, and returns Jellyfin's status and range headers. Bytes stream through (no buffering) and count toward `bytesServed` as sent. A viewer disconnect cancels the upstream response, which closes that connection (tested: `HttpClientRequest.abort` does nothing once the response has arrived; cancelling the response subscription does). - No progress reporting in the server: watched state is per client. ## Relay A member's media through the server's memory, for viewers who cannot reach it themselves (`packages/core/lib/src/relay.dart` for the wire and the source, `relay_pieces.dart` for segmented streams, `packages/server/lib/src/relay_buffer.dart`, `piece_buffer.dart`, `relay_routes.dart`; wire in [PROTOCOL.md](PROTOCOL.md) "Relay"). What gets relayed (`SessionCoordinator.swapPicked`): | Put up | Others get it | |---|---| | File, on your own server | `/m` from your server (no relay) | | File, by anyone else | relayed from that member's computer | | Link, "Everyone fetches it" | each app fetches it | | Link, "Server fetches it" | the server reads it once into the buffer; the link is never broadcast; refused on an open server (it would fetch any address it can reach for anyone) | | Link, "Through me" (default) | relayed from that member's computer; plays for everyone whenever it plays for them (their cookies, their network) | | Jellyfin item (browser pick or pasted link) | own account first, then the server's login (`/j`), then relayed from the member who put it up (with their sign-in, or the pasted link's key, which stays on their machine) | Relayed links are range-read: a media file as is, a web page as yt-dlp's formats, resolved where the bytes are fetched (see "yt-dlp"); the swap carries only the page URL and the mode. A page's formats are one file, or a video-only and an audio-only file (YouTube has only those): a media of two **tracks**, each relayed as its own byte stream with its own buffer and URL, played together by viewers as an mpv EDL (`edl://!no_clip;%n%