Relay: a member's media through the server's buffer (proto 8)
ci / core (push) Successful in 28s
ci / updater-windows (push) Successful in 49s
ci / server (push) Successful in 49s

Viewers who cannot reach the media get it through the server: a file on
any member's computer, a link passed on by a member ("Through me") or
fetched by the server ("Server fetches it", password-protected servers
only), and Jellyfin items for guests without an account, sourced by
whoever put them up with their own sign-in (or a pasted link's key),
which never leaves their machine. Fixes guests on a dedicated server
without a Jellyfin login being blocked.

The source dials /r/.../source (works behind NAT) and answers pulls with
an adaptive in-flight window (2 x rate x min RTT, 256 KiB..8 MiB). The
server keeps a memory-only buffer of 1 MiB slices sized like a viewer's
mpv buffer (pinned head/tail, 64 MiB back, 256 MiB ahead), fills it with
one sequential pull at a time (blocked viewers first, newest request
only), fetches nothing while nobody watches through it, and drops it on
the next swap or stop. Measured with mpv through shaped links
(tool/relay_e2e.dart): 400 Mbit/s seeks ~0.03 s; 40 Mbit/s no stalls,
3 viewers cost the source ~1x the bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
foxyzandClaude Opus 5.5 committed 2026-10-08 20:41:01 +03:00
1 parent 9c5733719b
commit e7f33b5dc3
27 files changed
+2438 -116

No files matched your search

+96 -7
View File
@@ -8,7 +8,7 @@ Plain overview first: [HOW-IT-WORKS.md](HOW-IT-WORKS.md). Wire format:
| Path | What | Rule |
|---|---|---|
| `packages/core` | protocol messages, sync engine (`SyncEngine`), constants, config, Range parsing, logging, Jellyfin client | pure Dart, no Flutter |
| `packages/server` | Relic HTTP + WebSocket server (`SyncServer`, `ServerHub`, `Session`, media endpoint, Jellyfin relay), the `serve()` entrypoint, `bin/server.dart` (dedicated server) | pure Dart, no Flutter |
| `packages/server` | Relic HTTP + WebSocket server (`SyncServer`, `ServerHub`, `Session`, media endpoint, relay buffer, Jellyfin relay), the `serve()` entrypoint, `bin/server.dart` (dedicated server) | pure Dart, no Flutter |
| `apps/desktop` | Flutter app: UI, `SyncClient`, `SessionCoordinator`, `PlaybackController` (media_kit/mpv), `ServerIsolate`, updater | ships with `watch-together-server` |
| `tool` | harnesses: `range_logger`, `shaper`, `fake_viewer`, `e2e_check` | dev only |
| `deploy` | systemd unit for the dedicated server | |
@@ -144,7 +144,8 @@ Additive wire changes only; bump `proto` and PROTOCOL.md in the same commit.
## Media
Three sources, same sync.
Three sources, same sync. Any of them can also go through the relay
(below).
- **Link**: the media field holds an http(s) URL. It travels as `media.url`
in `swap` and `state`, and as `mediaUrl` in `welcome`. Each app plays it
@@ -166,8 +167,8 @@ Three sources, same sync.
- **Jellyfin item**: the media is a ref (`{serverId, itemId, ...}`, wire in
[PROTOCOL.md](PROTOCOL.md) "Jellyfin media"); the server never stores or
sends a URL or token for it. Each app resolves it with its own account and
direct-plays the static stream; guests use the relay (see "Jellyfin
relay").
direct-plays the static stream; guests use the server's own login when
it has one ("Jellyfin relay"), else the member who put it up ("Relay").
Range table (`resolveMediaRequest`, tested byte-exact):
@@ -362,6 +363,86 @@ errors leave it out).
subscription does).
- No progress reporting in the server: watched state is per client.
## Relay
A member's media through the server's memory, for viewers who cannot
reach it themselves (`packages/core/lib/src/relay.dart` for the wire and
the source, `packages/server/lib/src/relay_buffer.dart`,
`relay_routes.dart`; wire in [PROTOCOL.md](PROTOCOL.md) "Relay").
What gets relayed (`SessionCoordinator.swapPicked`):
| Put up | Others get it |
|---|---|
| File, by the host | `/m` from the hosting app (no relay) |
| File, by anyone else | relayed from that member's computer |
| Link, "Everyone fetches it" (default) | each app fetches it |
| Link, "Server fetches it" | the server reads it once into the buffer; the link is never broadcast; refused on an open server (it would fetch any address it can reach for anyone) |
| Link, "Through me" | relayed from that member's computer; for links only they can open |
| Jellyfin item (browser pick or pasted link) | own account first, then the server's login (`/j`), then relayed from the member who put it up (with their sign-in, or the pasted link's key, which stays on their machine) |
Relayed links must be one file the relay can range-read: a `text/*`
response (YouTube and other pages, which only the player resolves) is
refused before the swap.
How bytes move:
- The source dials `/r/.../source` (outbound, so NAT/CGNAT are fine) and
answers the server's pulls from the file or the URL (`RelaySource`,
`FileRelayBytes`, `HttpRelayBytes`; a Jellyfin source sends its own
`Authorization` header from this machine). Viewers only ever talk HTTP
to the server, so the source never sees mpv's request pattern.
- The server keeps one `RelayBuffer` per relayed media: 1 MiB slices in
memory, filled by one sequential pull at a time. Kept: the first and
last 2 MiB (container indexes; mpv reads both on open and on seeks),
64 MiB behind the slowest viewer and 256 MiB past the furthest, the
same as each viewer's mpv buffer (`PlaybackController`), so a skip
inside the viewers' buffer window is served from memory. Evicted by
distance from the viewers, never LRU. With 2+ viewers the source sends
each byte about once.
- Filling: a viewer blocked on a slice first (oldest wait first, handing
the source over at slice boundaries), then the pinned slices, then
read-ahead from the slowest viewer. Nothing is fetched while nobody
watches through the relay (a Jellyfin source with only account holders
uploads nothing).
- mpv leaves superseded requests open but unread after a seek: only a
viewer's newest request moves its position or asks the source for
anything; older ones still get bytes that are already buffered.
- Flow control on the source (`RelayWindow`): one in-flight budget per
connection, 2 x delivery rate x min RTT (rate max-filtered over 2 s),
clamped 256 KiB to 8 MiB; the server acks bytes as they arrive. A
cancelled pull's unacked bytes are written off, and a cancel that lands
while a frame is being read stops it before it is sent.
- Cleanup: the buffer is memory only. Any swap disposes it (its URLs then
404, the source is closed with 4001 and stops), and so does server
stop. A crash frees it with the process.
- Source away: viewers keep playing from the buffer; a request waits up
to 60 s for missing bytes. The source reconnects with backoff (same
token after a resume); a different file size drops the buffer.
`state.media.relay.live` follows the channel, and viewers show "waiting
for <name> to pass the video on" while it is down.
Measured (mpv 0.41 viewers, real server and source, a shaped link
between source and server; `tool/relay_e2e.dart`, 2-minute 1080p test
file averaging 10 Mbit/s with a 15 Mbit/s busy section):
| Source uplink | Open | Seeks (incl. +90 s skip) | Stalls in 50 s |
|---|---|---|---|
| 400 Mbit/s, 20 ms | 0.8 s | 0.01 to 0.03 s | 0 |
| 40 Mbit/s, 40 ms | 2.1 s | 1.1 s first, then 0.03 s | 0 |
| 20 Mbit/s, 150 ms | 5.4 s | 1.1 to 5.3 s | 3 |
| 12 Mbit/s, 40 ms | 5.4 s | 0.5 to 2.9 s | 3 |
3 viewers skipping together on 40 Mbit/s: no stalls, the source sent
101 MiB for 287 MiB served. A late joiner opened in 0.04 s from the
buffer. A 5 s source outage: no stall; 30 s: one stall once the buffer ran
out, then it recovered. Freezes of 0.3 to 0.8 s in 30% of seconds: no
stall. So the source's upload must beat the media's average bitrate
(peaks are covered by read-ahead once it has some lead), and comfortably
so for quick seeks. Rejected along the way: a fixed per-request window
(seeks of 14 s on slow links, data for abandoned positions clogging the
pipe), one shared window across requests (deadlocked on mpv's unread
requests), Dart `http2` (connection window stuck at 64 KiB).
## Jellyfin in a session
Client side (`lib/src/jellyfin/session_media.dart`, `progress.dart`,
@@ -374,8 +455,9 @@ Client side (`lib/src/jellyfin/session_media.dart`, `progress.dart`,
MediaBrowser` with this install's fields), not in the URL: `OwnPlay`,
`report.jf {serverId: true}`. Else, when `state.relay`, `welcome.relayUrl`
or `/j/default/<token>/<itemId>` on the dialled host: `RelayPlay`,
`jf: false`. Else `BlockedPlay`: the session shows "This needs a
Jellyfin account on <server> (or the host can relay)" with a button that
`jf: false`. Else, when `media.relay` names a source, the `/r/` URL:
`RelayPlay`. Else `BlockedPlay`: the session shows "No Jellyfin account
here for <server>, and nobody is passing it on" with a button that
opens Integrations over the session; linking (or reconnecting) the
account retries by itself. A 401 marks the account expired. The member
list shows "via host" for members whose `jf` is false for the current
@@ -577,6 +659,13 @@ this repo, written by `tool/release.sh`.
- Plain `ws://`, no TLS: friends use LAN, VPN or a forwarded port; TLS
(e.g. a reverse proxy) is a documentation item.
- One session per server, id `default`: the map is keyed for later but unused.
- No server storage: config and logs only; media never touches the server disk.
- No server storage: config and logs only; media never touches the server
disk. The relay buffer is memory only and sized like a viewer's buffer
(about 320 MiB at most), so a small VPS can carry it.
- The relay is the source's own connection through the server, not a
shared token: a member's Jellyfin sign-in or a link's key never leaves
their machine. Prior art (Plex Relay, Tailscale DERP) relays as a
bandwidth-capped fallback on someone else's servers; here the server is
the owner's, so it carries full quality.
- Avatars ride in the 1 Hz `state` broadcast (cap 256 KiB base64 per avatar):
simple, costs host uplink; hash-based serving is on the roadmap.