Read once: a link passed on is fetched from the site once, for the putter's player and the relay

Before, the member passing a page link on downloaded it twice, independently:
their mpv straight from the CDN and the relay's pulls again, with a
different format pick and without yt-dlp's cookies. The two drifted (the
putter played on while viewers stalled, or the reverse) and doubled the
connections the CDN saw. Now one SharedSource per track reads the site
once (resuming after cuts, backing off on 412/429/5xx and silence) into a
bounded buffer (64 MiB ahead, 16 behind) that serves both the putter's
mpv through a loopback endpoint and the server's pulls; the putter plays
the relay's own format pick, and the relay sends yt-dlp's cookies.
Up next's warm relay uses the same reader. Pieces tracks (HLS/DASH) keep
two fetches for now. Measured in the opt-in e2e: half the CDN traffic, no
412s under a lockout CDN, viewer seeks 0.29 s instead of 4 s there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
foxyzandClaude Opus 5.5 committed 2026-10-18 02:05:03 +03:00
1 parent 7715dc479a
commit ec0849eb01
19 files changed
+1894 -101

No files matched your search

+78 -9
View File
@@ -638,8 +638,9 @@ ready.
can, so this is Android's plug-in point).
- Resolve: a Jellyfin item through this app's account (or the plan
resolved ahead for Up next, `warmJellyfin`), else the server's login,
else the member passing it on, else blocked (`openJellyfin`); a web
page through the app's own yt-dlp (`LinkResolver.probe`: yt-dlp's
else the member passing it on, else blocked (`openJellyfin`); a link
this member passes on through its shared reader's loopback URL ("Read
once" under "Relay"); a web page through the app's own yt-dlp (`LinkResolver.probe`: yt-dlp's
stream URLs, an EDL of video and audio, the CDN's headers with its
cookies, the subtitles; see "yt-dlp"), failing in plain words with the
cookies that were used (`ResolveFailure`, D11); a relay not fed yet is
@@ -681,6 +682,10 @@ ready.
- Being a relay source (`RelayFeeder`, `session/relay_feeder.dart`): the
current media's channel and Up next's warm relay (see "Relay"), with
the step-1 re-dial (a source rides through this app's own drop). A
link it passes on is read once for its player and the relay
(`SharedLinks`, `session/shared_links.dart`; "Read once" under
"Relay"): the swap check's reader, else Up next's warm one, else a new
one; it lives while the media does (a sit-out keeps it for the relay). A
Jellyfin item this member put up is passed on only once someone
present needs it (`jellyfinRelayNeeded`: the server's login does not
cover it and a present member does not play that server with its own
@@ -840,8 +845,8 @@ and URL, played together by viewers as an mpv EDL
`relayViewerUrl` / `mpvEdl`). media_kit opens an `edl://` URL unchanged
(it writes it to a playlist and `loadlist`s it) and mpv keeps both
streams in sync across seeks; one URL keeps the app's "one media URL"
path (sync, swap, reopen) as is. The source member plays the page
itself (its own resolve, see "yt-dlp") like any other link.
path (sync, swap, reopen) as is. The source member's own player reads
the same tracks through its shared reader ("Read once" below).
A page with only segmented streams (HLS `m3u8_native`, DASH
`http_dash_segments`, e.g. X's HLS, generic `.m3u8`/`.mpd`) is relayed as
@@ -873,6 +878,64 @@ How bytes move:
this machine). Viewers only ever talk HTTP to the server, so the source
never sees mpv's request pattern. The tracks of a media share the
channel and its flow control (they compete for the same uplink).
- **Read once** (a link passed on: `SharedSource`,
`packages/server/lib/src/shared_source.dart`, kept by the app's
`SharedLinks`): the source reads each track from the site once. Per
track a `RelayBuffer` on the source (64 MiB ahead and 16 MiB behind
for all tracks, split by size like the server's, plus the pins:
`sharedAheadBytes` / `sharedBackBytes`), filled by one sequential
upstream reader, serves two readers: the source's own player, through
a loopback endpoint (`http://127.0.0.1:<port>/<random secret>/<track>`,
an mpv EDL for several tracks, no headers: the site's stay in the
reader) answered with the server's own byte-answer code
(`writeRelayRange`, below: headers within 4 s, `connection: close`),
and the server's pulls (`SharedSource.upload`, served by the usual
`RelaySource`). The player opens at once; the endpoint waits for the
tracks' sizes. The two readers are kept within one window
(`RelayBuffer.together`): while both have a request open, the one more
than the track's `ahead` past the other waits without asking upstream,
so one sequential read serves both instead of handing over at every
slice. The server's read-ahead thus stops about 64 MiB past the
source's player, and a source whose upload lags holds its own player
there (viewers would wait at that point anyway); either reader alone
reads freely, pinned slices always come. A seek (shared, so both
readers go together) is a new upstream read there, as on the server.
Upstream (`ResumingBytes`): a cut resumes from the byte it reached
after 100 ms; an answer without a byte (412, 429, 5xx, a refused or
dead connection, nothing for 20 s) backs off 0.5 s doubling to 8 s;
60 s without a byte fails the read, and the buffer's own backoff asks
again. 403/410/412 first resolve the page again (see "yt-dlp"). The
source's player plays the relay's tracks, so the source and the
viewers watch one format, and the reader sends the headers and cookies
yt-dlp gave for it. The putter's swap check is this reader (its sizes
are the media's: no second read); an Up next link passed on feeds its
warm relay from its own reader, the preload player reads its loopback
URL (one prepared on the site before the reader existed is prepared
again), and the swap carries both on. Reload video opens the loopback
URL again (the reader resolves an expired page itself). Not read once:
a page with a pieces track (HLS/DASH, X's): its player plays yt-dlp's
pick from the site and the relay reads the pieces itself, two fetches;
a file passed on (disk) and a Jellyfin item (see "Media pipeline")
keep their own paths. Measured (server `single_fetch_e2e`, opt-in: a
real server and `RelaySource`, mpv 0.41 with the app's pins on
loopback, a fake CDN with Bilibili's Referer rule and two tracks:
60 s of 720p h264 at 4 Mbit/s and 128 kbit/s aac; 20 s of play, then
a seek to 45 s and back to 5 s):
| CDN / source | First frame: source, viewer | Viewer stalls | Viewer seek to 45 s | CDN requests (player + relay) | CDN MiB | 412s |
|---|---|---|---|---|---|---|
| clean, two fetches | 0.22 s, 0.46 s | 0 | 0.27 s | 8 (2 + 6) | 51.8 | 0 |
| clean, read once | 0.27 s, 0.33 s | 0 | 0.29 s | 6 (0 + 6) | 25.9 | 0 |
| cuts every 0 to 2 MiB, two fetches | 0.22 s, 0.38 s | 1 | 0.27 s | 64 (28 + 36) | 55.9 | 0 |
| cuts, read once (29 resumes) | 0.22 s, 0.27 s | 0 | 0.27 s | 33 (0 + 33) | 25.9 | 0 |
| cuts + 412 lockout (> 40 requests in 10 s), two fetches | 0.21 s, 0.39 s | 1 | 4.03 s | 49 (23 + 26) | 35.3 | 6 |
| cuts + lockout, read once | 0.26 s, 0.33 s | 0 | 0.29 s | 33 (0 + 33) | 25.9 | 0 |
| 768 KiB/s per connection, two fetches | 0.40 s, 0.59 s | 1 | 0.80 s | 12 (5 + 7) | 34.0 | 0 |
| 768 KiB/s, read once | 0.40 s, 0.47 s | 1 | 0.76 s | 7 (0 + 7) | 16.9 | 0 |
The source's own seeks took about what the viewer's did; back to 5 s
took 0.25 to 0.29 s everywhere. The source's player stalled only on
the slow CDN (once, in both modes).
- Upstream GETs (`HttpRelayBytes.read`) are split into back-to-back
ranges of at most yt-dlp's `http_chunk_size` for the format, else
10 MiB (`relayUpstreamChunkBytes`); a buffer pull stays one read.
@@ -918,7 +981,9 @@ How bytes move:
source's upstream dropping, a CDN refusing) is followed by the next
one after 250 ms, doubling up to 8 s (`relayRetryDelay`), back to none
once a slice or a pull completes: a failing CDN is not asked again
every round trip (that drew Bilibili's 412/403). A viewer reads the
every round trip (that drew Bilibili's 412/403). Read-ahead never
fetches the slice that would be evicted next, and a long pull evicts
as its slices come, so a buffer stays within its budget. A viewer reads the
slice being filled as its bytes arrive, not once it is whole (mpv's
demuxer, blocked in a read, takes a seek only once that read returns).
Nothing is fetched while nobody watches through the relay (a Jellyfin
@@ -1126,7 +1191,8 @@ all three link modes. Nothing is bundled or installed by hand.
format's `http_headers` plus its `cookies` as a Cookie header; the
requested subtitles but Bilibili's danmaku, an XML comment feed); and
what a relay reads (`pickTracks`), each format with its URL,
`http_headers` and `http_chunk_size`. For a relay only plain http(s)
`http_headers` plus its `cookies` as a Cookie header (what a player
sends) and `http_chunk_size`. For a relay only plain http(s)
formats without DRM count:
- the best single file (`pickSingleFile`: video and audio not `none`, a
video container, tallest up to 1080p, else the shortest above), unless
@@ -1176,11 +1242,14 @@ all three link modes. Nothing is bundled or installed by hand.
track's playlist must be readable, else the swap is refused ("Live
streams can't be passed on", DRM, or nothing a relay can read) with a
hint to use "Everyone fetches it". The swap names the tracks; its relay
source reads them with yt-dlp's headers (`PageRelayMedia`). On
source reads them with yt-dlp's headers and cookies (`PageRelayMedia`),
and the putter's own player plays those same tracks through the one
reader ("Read once" under "Relay"), not yt-dlp's `requested_formats`:
the putter sees the relay's pick (HEVC first) like the viewers. On
403/410/412 (expired format URLs, 6 h on YouTube; signed segment URLs;
Bilibili's 412) from any track it resolves the page again once for all
tracks (not again within 60 s: a further refusal fails the read and the
server's backoff asks later), finds the same
tracks (not again within 60 s: a further refusal fails the read and is
backed off), finds the same
format ids with the same sizes and piece counts and roles (playlists
re-read and re-mapped by index; else "the video changed on the page")
and goes on from where it stopped. Format URLs are bound to the