Foxyz/sync_replay_project
1
0 Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 13 Wiki Activity

Relay HLS/DASH-only pages as pieces tracks (protocol 11)

Browse Source 
A track can be served as pieces: the server writes a VOD playlist whose
entries are its own piece URLs (segments, init maps, AES-128 keys; byte
ranges folded in; yt-dlp's DASH fragments become the same list). A piece
buffer per track fills like the byte buffer. Picks a single file, then a
split byte pair, then pieces. DRM, live, master playlists are refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
foxyzandClaude Opus 5.5 committed 2026-10-09 23:35:31 +03:00
1 parent 05f9e794a9
commit fe5d6c3b3f
18 files changed
+1593 -207
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/ARCHITECTURE.md
+64 -16
View File
@@ -423,7 +423,8 @@ errors leave it out).
A member's media through the server's memory, for viewers who cannot
reach it themselves (`packages/core/lib/src/relay.dart` for the wire and
the source, `packages/server/lib/src/relay_buffer.dart`,
the source, `relay_pieces.dart` for segmented streams,
`packages/server/lib/src/relay_buffer.dart`, `piece_buffer.dart`,
`relay_routes.dart`; wire in [PROTOCOL.md](PROTOCOL.md) "Relay").
What gets relayed (`SessionCoordinator.swapPicked`):
@@ -450,6 +451,27 @@ streams in sync across seeks; one URL keeps the app's "one media URL"
path (sync, swap, reopen) as is. The source member plays the page
itself (yt-dlp hook) like any other link.
A page with only segmented streams (HLS `m3u8_native`, DASH
`http_dash_segments`, e.g. X's HLS, generic `.m3u8`/`.mpd`) is relayed as
**pieces** tracks (`"as":"pieces"`, one muxed playlist or video + audio):
the fetcher parses the upstream media playlist (`parseHlsPlaylist`), or
yt-dlp's DASH `fragments` (`dashPieces`: a first fragment without a
duration is the init section, so no MPD parser), into a
`RelayPieceList`: each segment, `EXT-X-MAP` init section and AES-128 key
is a piece, one upstream GET, with `EXT-X-BYTERANGE` folded into the
piece. The server serves a VOD media playlist it writes itself from that
list (`RelayPieceList.playlist`: durations, media sequence,
discontinuities, MAP, AES-128 keys with their IVs; every URI its own
`p/<i>`), so a source can never point viewers anywhere else. One URL per
piece also avoids mpv's seek hang on byte-range fMP4 playlists (Apple's
`bipbop` hung on the first seek read directly). Refused: DRM (SAMPLE-AES,
`skd:`, any non-identity key format, `EXT-X-SESSION-KEY`, yt-dlp's
`has_drm`), a live playlist (no `EXT-X-ENDLIST`; yt-dlp's `is_live`
first), a master or I-frame playlist. A remux on the fetcher (ffmpeg
`-c copy` into one pipe) and yt-dlp's own download + merge were rejected:
the size is unknown until the end, a seek past the written part cannot be
served, and it needs ffmpeg or disk.
How bytes move:
- The source dials `/r/.../source` (outbound, so NAT/CGNAT are fine) and
answers the server's pulls from the file or the URL (`RelaySource` over
@@ -467,8 +489,8 @@ How bytes move:
8.6 to 12.5 MB/s); chunked, a 1080p relay seeks in under 0.8 s (4 to
6.6 s unchunked) and even beats mpv reading YouTube directly, since the
buffer reads ahead at full speed.
- The server keeps one `RelayBuffer` per track of a relayed media (one
for a file): 1 MiB slices in
- The server keeps one buffer per track of a relayed media (one for a
file), a `RelayBuffer` for bytes: 1 MiB slices in
memory, filled by one sequential pull at a time. Kept: the first and
last 2 MiB (container indexes; mpv reads both on open and on seeks),
64 MiB behind the slowest viewer and 256 MiB past the furthest, the
@@ -480,9 +502,27 @@ How bytes move:
each, which keeps both windows at the same playback distance (1080p
video + aac: about 246 + 10 MiB ahead); each pins its own head and
tail. A viewer is its token in every track's buffer.
- Filling: a viewer blocked on a slice first (oldest wait first, handing
the source over at slice boundaries), then the pinned slices, then
read-ahead from the slowest viewer. Nothing is fetched while nobody
- A pieces track gets a `PieceBuffer`: whole pieces by index, each one
pull (a piece names its index on the channel), served whole with Range
inside a piece. A viewer's position is its newest requested segment
(opening the playlist puts a new viewer at the first one, so its first
pieces are fetched before it asks). Filled: a blocked viewer's piece
(oldest wait first; a pull nobody waits on gives way at once), the init
and key pieces the viewers' segments use, then segments ahead of each
viewer up to the ahead budget, slowest first. Init and key pieces are
kept; segments are evicted by distance (behind costs 4x), sizes
estimated from the bytes per second of the pieces in so far (before
that 4 Mbit/s video, 160 kbit/s audio, which also splits the budget).
A failed pull is retried after 1 s. Measured (mpv 0.41 through the
dedicated server, loopback, ~30 MB/s upstream): Mux's TS HLS (1080p,
one muxed playlist) opened in 0.8 s, seeks 0.07 to 1.1 s; Akamai's DASH
(1080p + aac, two playlists, EDL) 1.3 s, seeks 0.03 to 1.4 s; an X
video's HLS video + audio 0.3 to 0.75 s, seeks 0.03 to 0.4 s; A/V sync
within 0.2 ms after every seek; a second viewer of the same pieces
opened in 0.02 s from the buffer.
- Filling a byte buffer: a viewer blocked on a slice first (oldest wait
first, handing the source over at slice boundaries), then the pinned
slices, then read-ahead from the slowest viewer. Nothing is fetched while nobody
watches through the relay (a Jellyfin source with only account holders
uploads nothing).
- mpv leaves superseded requests open but unread after a seek: only a
@@ -498,7 +538,8 @@ How bytes move:
stop. A crash frees it with the process.
- Source away: viewers keep playing from the buffer; a request waits up
to 60 s for missing bytes. The source reconnects with backoff (same
token after a resume); a different size drops that track's buffer.
token after a resume); a different size (or piece count and roles)
drops that track's buffer.
`state.media.relay.live` follows the channel, and viewers show "waiting
for <name> to pass the video on" while it is down.
@@ -581,7 +622,12 @@ all three link modes. Nothing is bundled or installed by hand.
and the server buffer, and mpv decodes them all), then bitrate; audio the original language, not a
`-drc` variant unless alone, m4a/aac before opus, then bitrate.
YouTube has only these (299 + 140 for a 1080p60 video).
- Neither (only HLS/DASH playlists) or a live stream: nothing to relay.
- Neither: segmented streams (`pickPieces`: `m3u8_native`/`m3u8` with a
media playlist URL, or `http_dash_segments` with http(s) `fragments`;
not `has_drm`): one muxed playlist (video and audio not known to be
absent) unless a video-only + audio-only pair is taller, with the
same height, codec and audio rules. Relayed as pieces (see "Relay").
- A live stream, or nothing of the above: nothing to relay.
- **Probe** (`probeLink`, `link_probe.dart`): a `Range: bytes=0-0` GET; a
video/audio type, or a generic one on a media file name, is a file
(`looksLikeMediaFile`); anything else goes to yt-dlp, and when yt-dlp
@@ -595,14 +641,16 @@ all three link modes. Nothing is bundled or installed by hand.
(`script-opts=ytdl_hook-ytdl_path=...`, `ytdl-format=bv*[height<=1080]+ba/b`,
once per player), waiting for the download unless the URL names a media
file. A hook error shows as "can't open this link: ...".
- **Through me**: the putter's probe must give tracks, else the swap is
refused ("Live streams can't be passed on", or no video file: only
streaming playlists) with a hint to use "Everyone fetches it". The swap
names the tracks; its relay source reads them with yt-dlp's headers
(`PageRelayMedia`). On 403/410 (expired format URLs, 6 h on YouTube) from
any track it resolves the page again once for all tracks, finds the
same format ids with the same sizes (else "the video changed on the
page") and goes on from where it stopped. Format URLs are bound to the
- **Through me**: the putter's probe must give tracks, and a pieces
track's playlist must be readable, else the swap is refused ("Live
streams can't be passed on", DRM, or nothing a relay can read) with a
hint to use "Everyone fetches it". The swap names the tracks; its relay
source reads them with yt-dlp's headers (`PageRelayMedia`). On 403/410
(expired format URLs, 6 h on YouTube; signed segment URLs) from any
track it resolves the page again once for all tracks, finds the same
format ids with the same sizes and piece counts and roles (playlists
re-read and re-mapped by index; else "the video changed on the page")
and goes on from where it stopped. Format URLs are bound to the
resolving machine's address, so they never leave it.
- **Server fetches it**: the server probes the link itself
(`LinkRelayMedia`) with its own yt-dlp (off with `--no-yt-dlp` /
docs/PROTOCOL.md
+44 -18
View File
@@ -235,17 +235,22 @@ flow control, cases) is in [ARCHITECTURE.md](ARCHITECTURE.md), "Relay".
swap media.relay = {} the swapper's machine serves the bytes
media.relay = {"tracks":[{"kind":"video"},{"kind":"audio"}]}
the same, as two tracks played together
media.relay = {"tracks":[{"kind":"video","as":"pieces"}]}
the same, as one segmented playlist
media.relay = {"fetch":true} the server reads media.url itself
state media.relay = {"source":"m2","live":true} or {"fetch":true,"live":true}
media.relay = {"fetch":true,"live":true,"tracks":[{"kind":"video"},{"kind":"audio"}]}
```
Tracks: a relayed media is one file, or several byte tracks (a page's
separate video and audio files) that viewers play as one. `tracks` lists
them in order, 2 to 4 entries of `kind` `video` or `audio`; absent (or
anything else) means one file. Every track is a Range-readable byte
stream; other kinds of track (e.g. segmented playlists) would add a field
to the entry.
Tracks: a relayed media is one file, or tracks that viewers play as one
(a page's separate video and audio). `tracks` lists them in order, up to
4 entries of `kind` `video` or `audio`, each a Range-readable byte stream,
or with `"as":"pieces"` a segmented stream (an HLS media playlist or DASH
fragments) relayed piece by piece and served as a VOD media playlist. A
piece is one segment, init section (`EXT-X-MAP`) or AES-128 key, fetched
upstream whole with one GET. At least 2 entries, or 1 pieces entry (a
muxed playlist, `kind` `video`); absent, an unknown `kind` or `as`, or
anything else means one file.
- `swap`: `relay` with `jellyfin` keeps the ref (the swapper serves it to
viewers without an account; `fetch` is ignored there). `relay` without
@@ -269,19 +274,27 @@ to the entry.
Clients build it after a swap like the `/m` one:
`http://<host>/r/default/<token>/<mediaId>/<name>` (name with `/` and
`\` made `_`, then URL-encoded; the server ignores it). With `tracks`,
it is an mpv EDL of every track's URL
`http://<host>/r/default/<token>/<mediaId>/<i>/<name>`, each framed by
its byte length:
each track's URL is `http://<host>/r/default/<token>/<mediaId>/<i>/<name>`
(a pieces track: `<name>.m3u8`); one track is opened as is, several as
an mpv EDL of them all, each framed by its byte length:
`edl://!no_clip;%<len>%<track 0 url>;!new_stream;!no_clip;%<len>%<track 1 url>`.
- `GET /r/<sid>/<token>/<mediaId>/<name>`: the media (one file), Range
table as for `/m`. `GET /r/<sid>/<token>/<mediaId>/<i>/<name>`: track
`i` (from 0) of a media with `tracks` (track 0 of a one-file media
too). Errors: 401 bad token, 404 not the current relayed
media (a player still on an older `mediaId` gets 404, never the new
bytes) or no such track (the one-file URL of a media with `tracks`), 410
unknown session, 503 + `retry-after` the source never connected or the
server could not read the link (60 s). A request whose bytes do not
arrive within 60 s ends early. Bytes count toward `bytesServed`.
too); for a pieces track (any name), its media playlist
(`application/vnd.apple.mpegurl`): `#EXT-X-PLAYLIST-TYPE:VOD`,
`EXT-X-MEDIA-SEQUENCE`, `EXTINF` durations, `EXT-X-DISCONTINUITY`,
`EXT-X-MAP` and `EXT-X-KEY:METHOD=AES-128` (with `IV` when given) as
upstream, `EXT-X-ENDLIST`; every URI is `p/<j>`. `GET
/r/<sid>/<token>/<mediaId>/<i>/p/<j>`: piece `j` of pieces track `i`,
whole, once it is in (a Range inside the piece answers 206). Errors: 401
bad token, 404 not the current relayed media (a player still on an
older `mediaId` gets 404, never the new bytes), no such track (the
one-file URL of a media with `tracks`) or piece (or `p/` of a byte
track), 410 unknown session, 503 + `retry-after` the source never
connected or the server could not read the link (60 s), or a piece did
not arrive within 60 s. A byte request whose bytes do not arrive within
60 s ends early. Bytes count toward `bytesServed`.
- `GET /r/<sid>/<token>/source` (WebSocket): the source's relay channel,
dialled by the source (works behind NAT). Only the current source's
token is accepted; anyone else, or the source after the next swap, is
@@ -293,18 +306,31 @@ source -> server {"t":"info","size":<bytes>} first frame, one file
{"t":"info","tracks":[{"kind":"video","size":<bytes>},
{"kind":"audio","size":<bytes>}]}
first frame, tracks
{"t":"info","tracks":[{"kind":"video","as":"pieces","seq":5,
"pieces":[{"init":true},{"key":true},
{"d":6.0,"map":0,"key":1,"iv":"0x..."},
{"d":4.5,"map":0,"disc":true}]}]}
a pieces track
binary: [u32 big-endian pull id][bytes] in order per pull
{"t":"end","id":1} {"t":"fail","id":1,"reason":"..."}
server -> source {"t":"pull","id":1,"track":0,"start":0,"end":1048575}
inclusive; no track: 0
{"t":"pull","id":2,"track":1,"piece":17} piece 17, whole
{"t":"ack","id":1,"n":65536} bytes arrived
{"t":"cancel","id":1} stop that pull
```
Pull ids are unique across the tracks of a connection; `track` is sent
only for a media with `tracks`. An `info` whose track count differs
from the swap's `tracks` closes the channel with 4001. A track `size`
different from the one before (a reconnect) drops that track's buffer.
only for a media with `tracks`. A pieces track's `pieces` are its
pieces in order: `{"init":true}` an init section, `{"key":true}` an
AES-128 key, else a segment of `d` seconds whose `map` / `key` name the
init and key pieces it uses, `iv` an explicit IV (`0x` + hex), `disc` a
discontinuity before it; `seq` (default 0) the media sequence of the
first segment. A piece pull's bytes end with `end` like a range pull.
An `info` whose tracks differ from the swap's `tracks` (count, bytes or
pieces), or a malformed piece list, closes the channel with 4001. A
track `size` or piece count/roles different from the one before (a
reconnect) drops that track's buffer.
## Join and reconnect
docs/ROADMAP.md
+10 -14
View File
@@ -19,22 +19,18 @@ off by deleting it in the same commit as the code. Wire changes also edit
through media_kit; if not, resolve in the app and open the relay
tracks' formats. YouTube: yt-dlp wants a JS runtime (deno) for full
extraction. Formats that need yt-dlp's cookies.
- Relay "pieces" tracks for pages with only HLS/DASH (`m3u8_native`,
`http_dash_segments`): a track served as a synthesized VOD media
playlist whose entries are pieces (one segment, init `EXT-X-MAP` or
AES-128 key = one upstream GET, served whole, Range inside a piece
honoured), with a piece buffer per track (pieces by index, filled
like `RelayBuffer`: a blocked viewer, the init piece, then ahead of
the slowest viewer; evicted by distance); `EXT-X-BYTERANGE` folded
into pieces (it also avoids mpv's seek hang on byte-range fMP4
playlists); yt-dlp's DASH `fragments` turned into the same playlist,
so no MPD parser. Wire: a track entry gains `"as":"pieces"`, `info`
lists piece durations, a pull names a piece. Refuse DRM
(`skd:`/SAMPLE-AES/Widevine). Measured in a spike: piece relay of
Mux/Apple/Akamai HLS/DASH opened in 1.5 to 2.2 s, seeks 0.02 to 2.4 s.
- Live streams through the relay (today refused: the sync model needs a
fixed duration and seeking).
- `tool/relay_e2e.dart`: a two-track case over a shaped source link.
- `tool/relay_e2e.dart`: a two-track case (bytes and pieces) over a
shaped source link. Pieces on a slow source: a cold piece can take
longer than media_kit's 5 s `network-timeout`; if it does, fetch the
first pieces before setting `relay.live`.
- Pieces, unverified so far: the 403/410 re-resolve against a real
expiring HLS (signed segment URLs), HLS discontinuities / ad
splicing, Windows' bundled ffmpeg with extensionless piece URLs
(`p/<i>`; fine with ffmpeg 9 on Linux).
- Thumbnails and scrubbing previews of a relayed media with tracks
should open the video track's URL alone, not the EDL.
4. **AniList account**: sign in (OAuth via a localhost callback), "My list"
filter on the Schedule; today's "Following" (AnimeFlow's monitored