Protocol 15: one shared session log on the server, chat never logged, Mark a problem

Every app streams its redacted diagnostic lines (server time, batched
every 10 s, gzip, buffered while offline) to POST /l/; the server merges
them with its own into session-YYYY-MM-DD.log in time order, tagged by
who. Opt-out in Settings > About; Mark a problem (Ctrl+Shift+B, the
player menus, About) writes a marker with an optional note. Copy
diagnostics can include the server's session log. Chat text never
reaches any log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
foxyzandClaude Opus 5.5 committed 2026-10-12 19:41:06 +03:00
1 parent 4ab37c55cf
commit 7487b4deb1
29 files changed
+1376 -43

No files matched your search

+16 -2
View File
@@ -236,7 +236,7 @@ then everyone (you included) uses Join.
| `--[no-]yt-dlp` | on | read web pages (YouTube, X, …) with yt-dlp when the server fetches a link |
| `--yt-dlp-dir` | next to the binary | writable folder for the server's yt-dlp (systemd: the unit's `StateDirectory`) |
| `--yt-dlp-cookies` | none | cookies.txt or cookie-extension JSON export for that yt-dlp (sites that refuse guests); copied into the yt-dlp folder at start, removed from it when not given. **An account login**: keep it private, use a throwaway account for YouTube |
| `--log-dir` | `logs` next to `--config`, else next to the binary (systemd: the unit's `StateDirectory`) | folder for the log files (`server-YYYY-MM-DD.log`, kept 7 days; passwords, tokens and cookies left out) |
| `--log-dir` | `logs` next to `--config`, else next to the binary (systemd: the unit's `StateDirectory`) | folder for the log files (`server-YYYY-MM-DD.log` and the session log `session-YYYY-MM-DD.log`, kept 7 days; passwords, tokens and cookies left out) |
| `--config` | none | YAML or JSON file with the same settings (`port`, `secret`, `name`, `mediaPath`, `advertiseHost`, `ytDlp`, `ytDlpDir`, `ytDlpCookies`); replaces the flags |
| `--upgrade` | | update the install folder to the latest release ([Install and update](#install-and-update)), then exit |
| `--check-update` | | print whether a newer release exists, then exit |
@@ -251,6 +251,19 @@ Something not playing? Settings → About → **Copy diagnostics** puts the
app's recent log (passwords, tokens and cookies left out) on the
clipboard: paste it to whoever helps you, from both sides if you can.
While you are in a server, the app also sends its log to that server, which
merges everyone's lines and its own into one session log
(`session-YYYY-MM-DD.log` next to the server's logs; the hosting app keeps
it in its own logs folder), so a problem can be read afterwards from every
side. Chat is never logged anywhere, nor passwords, tokens and cookies.
When something goes wrong, press **Mark a problem** (Ctrl+Shift+B, also in
the player's ⚙ and Settings → About) and add a few words if you like: a
marker with your name goes in the session log at that moment. In a server,
Copy diagnostics can add the session log of the last 6 hours (any member
may read it). Don't want to share? Settings → About → **Share diagnostics
with the server** off: your log stays on your machine (the server only
notes that you don't share; a problem you mark still goes in).
## Join
Join in the strip. Servers you got into are listed first (up to 6), with
@@ -281,7 +294,8 @@ overrides it); in the player also the subtitle and audio track. Every
change applies right away.
Anywhere in a server (unless you are typing): Space = ready / unready,
Tab = chat & activity (Tab never moves between buttons). Keys while the
Tab = chat & activity (Tab never moves between buttons). Anywhere:
Ctrl+Shift+B = mark a problem in the session log. Keys while the
player fills the window: Left/Right = seek 10 s, Up/Down = volume, Shift+Right = skip 1:30,
PgUp/PgDn = previous / next chapter (as in mpv), F = fullscreen, Esc =
close the chat, then fullscreen, then shrink the player to mini; from the
@@ -8,6 +8,7 @@ import '../jellyfin/track_prefs.dart' show MpvTrack;
import '../playback/playback_controller.dart' show PlaybackController, RenderMode;
import '../shaders/shader_presets.dart';
import '../state/session_state.dart';
import '../shell/mark_problem.dart';
import '../shell/motion.dart';
import '../state/settings.dart';
import '../widgets/app_menu.dart';
@@ -332,10 +333,17 @@ class PlayerSettingsCard extends StatelessWidget {
const TrackSetting(type: JellyfinStreamType.audio),
],
const SizedBox(height: 4),
Text(
Row(
children: [
Expanded(
child: Text(
'Custom shader files: Settings → Player',
style: TextStyle(color: Theme.of(context).colorScheme.onSurfaceVariant, fontSize: 12),
),
),
const MarkProblemButton(),
],
),
],
),
),
+11 -2
View File
@@ -44,8 +44,10 @@ Future<void> startAppLogging() async {
}
/// The text "Copy diagnostics" gives: versions, then the last [since] of
/// the app log (its hosted server's records included), already redacted.
Future<String> diagnosticsText({Duration since = const Duration(hours: 6)}) async {
/// the app log (its hosted server's records included), already redacted;
/// with [sessionLog], the server's session log of the same hours after it
/// (every member's shared lines and the server's, merged).
Future<String> diagnosticsText({Duration since = const Duration(hours: 6), bool sessionLog = false}) async {
final b = StringBuffer()
..writeln('$appName ${displayVersion()}')
..writeln('OS: ${Platform.operatingSystem} ${Platform.operatingSystemVersion}')
@@ -68,6 +70,13 @@ Future<String> diagnosticsText({Duration since = const Duration(hours: 6)}) asyn
b.writeln('Log: ${files?.currentPath ?? 'none (console only)'}');
b.writeln('--- last ${since.inHours} h of the log ---');
b.write(files?.recent(since) ?? '');
if (sessionLog) {
final server = await SessionState.instance.coordinator.sessionLog(hours: since.inHours);
b
..writeln()
..writeln("--- last ${since.inHours} h of the server's session log ---")
..write(server ?? "(none: the server keeps no session log, or it couldn't be reached)");
}
return redactLog(b.toString());
}
@@ -1,7 +1,12 @@
import 'package:flutter/material.dart';
import 'package:signals/signals_flutter.dart';
import '../../platform/app_log.dart';
import '../../shell/mark_problem.dart';
import '../../state/session_state.dart';
import '../../state/settings.dart';
import '../../update/update_controller.dart';
import '../../widgets/app_menu.dart';
import '../../widgets/copy_open.dart';
import 'settings_page.dart';
import 'ui.dart';
@@ -45,18 +50,24 @@ class AboutPanel extends StatelessWidget {
),
const SizedBox(height: kGap),
Text('Something went wrong? Copy diagnostics and send them to whoever helps you: '
'versions and the last hours of the log, passwords, tokens and cookies left out.',
'versions and the last hours of the log, passwords, tokens and cookies left out. '
'In a session it can add the server\'s session log, everyone\'s lines merged. '
'Mark a problem (Ctrl+Shift+B) when it happens so the spot is easy to find.',
style: t.bodySmall?.copyWith(color: Colors.grey)),
const SizedBox(height: 8),
Wrap(
spacing: 8,
runSpacing: 8,
children: [
FilledButton.tonalIcon(
onPressed: _copyDiagnostics,
Builder(
builder: (context) => FilledButton.tonalIcon(
key: const ValueKey('copy-diagnostics'),
onPressed: () => _copyDiagnostics(context),
icon: const Icon(Icons.bug_report_outlined, size: 18),
label: const Text('Copy diagnostics'),
),
),
const MarkProblemButton(outlined: true),
OutlinedButton.icon(
onPressed: appLogDir == null ? null : () => _openLogs(),
icon: const Icon(Icons.folder_open, size: 18),
@@ -64,13 +75,36 @@ class AboutPanel extends StatelessWidget {
),
],
),
const SizedBox(height: 8),
SignalBuilder(
builder: (context) => SwitchListTile(
key: const ValueKey('share-diagnostics'),
contentPadding: EdgeInsets.zero,
title: const Text('Share diagnostics with the server'),
subtitle: const Text('In a session, this app\'s log goes to the server\'s session log, '
'merged with everyone\'s, so a problem can be read afterwards. Never chat; '
'passwords and tokens left out. Off: only a problem you mark is sent.'),
value: AppSettings.instance.shareDiagnostics.value,
onChanged: AppSettings.instance.setShareDiagnostics,
),
),
],
),
);
}
static Future<void> _copyDiagnostics() async {
final text = await diagnosticsText();
/// In a session it asks whether to add the server's session log.
static Future<void> _copyDiagnostics(BuildContext context) async {
var withServer = false;
if (SessionState.instance.route.peek() == AppRoute.session) {
final pick = await showAppMenu<bool>(context, items: const [
AppMenuItem(value: false, label: "This app's log"),
AppMenuItem(value: true, label: "This app's and the server's session log"),
]);
if (pick == null) return;
withServer = pick;
}
final text = await diagnosticsText(sessionLog: withServer);
final saved = await saveDiagnostics(text);
await copyToClipboard(text,
done: saved == null ? 'Diagnostics copied' : 'Diagnostics copied (also saved in the logs folder)');
@@ -23,6 +23,7 @@ const sessionShortcuts = [
('Ctrl+1…6', 'Shaders: Anime4K mode A, B, C, A+A, B+B, C+A'),
('Ctrl+0', 'Shaders off'),
('Esc', 'Close the panel or fullscreen, shrink the player, then ask to leave'),
('Ctrl+Shift+B', 'Mark a problem in the session log (anywhere)'),
];
/// One titled card of the Settings tab (rounded Material, so list tiles
@@ -17,6 +17,7 @@ import '../playback/playback_controller.dart';
import '../platform/attention.dart';
import '../playback/yt_dlp_app.dart';
import '../schedule/anilist.dart';
import '../screens/home/ui.dart' show appName, displayVersion;
import '../screens/home/picked_media.dart';
import '../sfx/sfx.dart';
import '../state/settings.dart';
@@ -174,6 +175,32 @@ class SessionCoordinator {
/// The app's yt-dlp and link probes (tests pass their own).
final AppYtDlp ytDlp;
final SyncClient sync = SyncClient();
/// This app's log lines to the server's session log (Settings: "Share
/// diagnostics with the server"), and "Mark a problem".
late final DiagUploader diag = DiagUploader(
target: _diagTarget, version: '$appName ${displayVersion()} on ${Platform.operatingSystem}');
DiagTarget? _diagTarget() {
final uri = sync.wsUri;
if (sync.token.isEmpty || uri == null || !sync.clockSync.hasSample) return null;
return (base: DiagUploader.baseOf(uri), token: sync.token, offsetMs: sync.clockSync.offsetMs);
}
/// "Mark a problem": a marker with [note] in the server's session log
/// (and this app's log). False when not in a session: then only this
/// app's log has it.
Future<bool> markProblem(String note) async {
if (!diag.running) {
createLogger(DiagUploader.markLogger).warning('=== PROBLEM marked${note.trim().isEmpty ? '' : ': ${note.trim()}'} ===');
return false;
}
await diag.markProblem(note);
return true;
}
/// The server's session log of the last [hours], null when not in a
/// session or the server has none.
Future<String?> sessionLog({int hours = 6}) => diag.fetchSessionLog(hours: hours);
/// Up next: the server's queue, adding to it, and putting items up.
late final QueueController queue = QueueController(this);
/// The server has a password (it fetches links only then).
@@ -498,6 +525,9 @@ class SessionCoordinator {
}
markInput();
_subscribe();
diag
..share = AppSettings.instance.shareDiagnostics.peek()
..start();
feedLog.value = [];
toasts.value = [];
await initPlayer();
@@ -1201,6 +1231,9 @@ class SessionCoordinator {
toasts.peek().where((t) => now.difference(t.at).inSeconds < 4).toList();
final r = serverState.peek();
if (sync.token.isNotEmpty) sync.sendPing();
diag
..share = AppSettings.instance.shareDiagnostics.peek()
..tick();
if (sync.token.isNotEmpty && r != null && _playerReady) {
_sendReport(r);
if (!_lastOut) {
@@ -2235,6 +2268,8 @@ class SessionCoordinator {
try {
await playback.dispose();
} catch (_) {}
// The last lines go before the token does.
await diag.stop();
try {
await sync.disconnect();
} catch (_) {}
@@ -0,0 +1,113 @@
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:sync_relay_core/core.dart' show RelayConstants;
import '../state/session_state.dart';
import '../widgets/app_menu.dart' show AppGlass;
import 'motion.dart';
// "Mark a problem" (Ctrl+Shift+B, the player's ⚙, Settings → About): a
// marker line in the server's session log at this moment, with an
// optional short note, so whoever reads the log finds the spot.
/// The key that opens it.
const markProblemKey = SingleActivator(LogicalKeyboardKey.keyB, control: true, shift: true);
/// Asks for the note, then marks; the toast says where it went.
Future<void> showMarkProblem(BuildContext context) async {
final note = await showPopover<String>(context, at: Alignment.center, child: const MarkProblemCard());
if (note == null) return;
final coord = SessionState.instance.coordinator;
final inSession = await coord.markProblem(note);
coord.showToast(inSession ? 'Marked in the session log' : "Marked in this app's log");
}
/// The small card: a one-line note and Mark. Pops with the note (maybe
/// empty), or nothing on Cancel / Esc.
class MarkProblemCard extends StatefulWidget {
const MarkProblemCard({super.key});
@override
State<MarkProblemCard> createState() => _MarkProblemCardState();
}
class _MarkProblemCardState extends State<MarkProblemCard> {
final _note = TextEditingController();
@override
void dispose() {
_note.dispose();
super.dispose();
}
void _mark() => Navigator.of(context).pop(_note.text.trim());
@override
Widget build(BuildContext context) {
final muted = Theme.of(context).colorScheme.onSurfaceVariant;
return AppGlass(
key: const ValueKey('mark-problem-card'),
child: SizedBox(
width: 380,
child: Padding(
padding: const EdgeInsets.fromLTRB(16, 12, 16, 10),
child: Column(
mainAxisSize: MainAxisSize.min,
crossAxisAlignment: CrossAxisAlignment.stretch,
children: [
const Text('Mark a problem', style: TextStyle(fontWeight: FontWeight.w600, fontSize: 14)),
const SizedBox(height: 4),
Text("Puts a marker in the server's session log right now, so whoever reads it finds the moment.",
style: TextStyle(color: muted, fontSize: 12)),
const SizedBox(height: 8),
TextField(
key: const ValueKey('mark-problem-note'),
controller: _note,
autofocus: true,
maxLength: RelayConstants.maxProblemNoteLength,
decoration: const InputDecoration(
isDense: true,
hintText: 'What happened? (optional)',
counterText: '',
),
onSubmitted: (_) => _mark(),
),
const SizedBox(height: 8),
Row(
mainAxisAlignment: MainAxisAlignment.end,
children: [
TextButton(onPressed: () => Navigator.of(context).pop(), child: const Text('Cancel')),
const SizedBox(width: 8),
FilledButton.tonal(
key: const ValueKey('mark-problem-go'),
onPressed: _mark,
child: const Text('Mark'),
),
],
),
],
),
),
),
);
}
}
/// "Mark a problem" as a quiet row button (the ⚙ card, Settings → About).
class MarkProblemButton extends StatelessWidget {
final bool outlined;
const MarkProblemButton({super.key, this.outlined = false});
@override
Widget build(BuildContext context) {
const icon = Icon(Icons.flag_outlined, size: 18);
const label = Text('Mark a problem');
void open() => showMarkProblem(context);
return Tooltip(
message: 'Ctrl+Shift+B',
child: outlined
? OutlinedButton.icon(key: const ValueKey('mark-problem'), onPressed: open, icon: icon, label: label)
: TextButton.icon(key: const ValueKey('mark-problem'), onPressed: open, icon: icon, label: label),
);
}
}
@@ -29,6 +29,7 @@ import '../widgets/app_keys.dart';
import '../widgets/text_focus.dart';
import '../widgets/window_buttons.dart';
import 'avatar.dart';
import 'mark_problem.dart';
import 'motion.dart';
import 'now_playing_title.dart';
import 'player_status.dart';
@@ -627,6 +628,7 @@ class _SessionPlayerState extends State<SessionPlayer> {
(title: 'Change video', hint: null, run: widget.onChangeVideo),
(title: 'Chat & activity', hint: 'Tab', run: _toggleChat),
(title: 'Reload video', hint: 'Ctrl+R', run: _reloadVideo),
(title: 'Mark a problem', hint: 'Ctrl+Shift+B', run: () => showMarkProblem(context)),
(title: 'Mark everyone ready', hint: null, run: coord.forcePlayAll),
(title: 'Fullscreen', hint: 'F', run: widget.onToggleFullscreen),
(title: 'Leave server', hint: 'Esc', run: widget.onLeave),
@@ -33,10 +33,12 @@ class ServerIsolate {
/// Spawns the server and waits until it listens. Throws a
/// [StateError] with the server's message if it cannot start.
/// [loopbackOnly]: 127.0.0.1 only (the solo session).
/// [loopbackOnly]: 127.0.0.1 only (the solo session). [logDir]: where
/// the server writes its session log (none when null).
static Future<ServerIsolate> start(
AppConfig config, {
bool loopbackOnly = false,
String? logDir,
required void Function(String line) onLog,
required void Function(String error) onCrash,
Duration timeout = const Duration(seconds: 15),
@@ -85,7 +87,7 @@ class ServerIsolate {
final Isolate isolate;
try {
isolate = await Isolate.spawn(
_serverMain, ({...config.toMap(), if (loopbackOnly) 'loopbackOnly': true}, events.sendPort),
_serverMain, ({...config.toMap(), if (loopbackOnly) 'loopbackOnly': true, 'logDir': ?logDir}, events.sendPort),
errorsAreFatal: false,
onError: errors.sendPort,
onExit: exit.sendPort,
@@ -10,6 +10,7 @@ import '../jellyfin/accounts.dart';
import '../jellyfin/session_media.dart';
import '../screens/home/health_probe.dart';
import '../screens/home/picked_media.dart';
import '../platform/app_log.dart' show appLogDir;
import '../playback/yt_dlp_app.dart';
import '../sfx/sfx.dart';
import '../sfx/sound_bank.dart';
@@ -379,6 +380,9 @@ class SessionState {
ytDlpDir: await _sharedYtDlpDir(),
),
loopbackOnly: soloSession,
// Its session log (every member's shared diagnostics and its own
// records) goes next to the app's; not for a solo session.
logDir: soloSession ? null : appLogDir,
// Its records join the app's log as `server.<logger>`.
onLog: (line) => relogLine('server', line),
onCrash: (cause) => _onServerCrash(server, cause),
+11
View File
@@ -28,6 +28,7 @@ const _keySoundPresetFor = 'soundPresetFor';
const _keyUnreadyOnSwap = 'unreadyOnSwap';
const _keyIdleSitsOut = 'idleSitsOut';
const _keyChatPopups = 'chatPopups';
const _keyShareDiagnostics = 'shareDiagnostics';
enum HwdecMode { auto, off, custom }
@@ -70,6 +71,9 @@ class AppSettings {
/// A 2 px progress line at the bottom of the video while the controls
/// are hidden (both designs). Applies live.
final progressLine = signal(true);
/// Send this app's log lines to the server's session log while in a
/// session (never chat). Off: nothing is sent but a note that it is off.
final shareDiagnostics = signal(true);
/// Chat message toasts (default: only when paused). Applies live.
final chatPopups = signal(ChatPopups.paused);
/// App sounds (`sfx/`): 0..1, 0 = silent.
@@ -129,6 +133,8 @@ class AppSettings {
if (hideRe is bool) hideReReleases.value = hideRe;
final line = box.get(_keyProgressLine);
if (line is bool) progressLine.value = line;
final share = box.get(_keyShareDiagnostics);
if (share is bool) shareDiagnostics.value = share;
final popups = box.get(_keyChatPopups);
chatPopups.value = ChatPopups.values.where((m) => m.name == popups).firstOrNull ?? ChatPopups.paused;
final vol = box.get(_keySoundVolume);
@@ -203,6 +209,11 @@ class AppSettings {
_put(_keyProgressLine, on);
}
void setShareDiagnostics(bool on) {
shareDiagnostics.value = on;
_put(_keyShareDiagnostics, on);
}
void setChatPopups(ChatPopups mode) {
chatPopups.value = mode;
_put(_keyChatPopups, mode.name);
+9 -2
View File
@@ -1,6 +1,7 @@
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import '../shell/mark_problem.dart';
import '../state/session_state.dart';
import 'text_focus.dart';
@@ -15,11 +16,12 @@ Map<ShortcutActivator, Intent> get appShortcuts => {
const SingleActivator(LogicalKeyboardKey.space): const DoNothingIntent(),
};
/// The window's two app-wide keys, ahead of whatever has focus: while in a
/// The window's app-wide keys, ahead of whatever has focus: while in a
/// server, Space is ready / not ready and Tab opens or closes the chat,
/// on every tab and in every player mode. Not while a text field has
/// focus (the chat field handles Tab and Esc itself) nor while a dialog or
/// menu is in front of [child].
/// menu is in front of [child]. Ctrl+Shift+B ([markProblemKey]) marks a
/// problem anywhere, a text field focused or not.
class AppKeys extends StatefulWidget {
/// Who toggles the chat, latest first: the strip while it is shown, the
/// full player in fullscreen (both hold the shell's toggle).
@@ -53,6 +55,11 @@ class _AppKeysState extends State<AppKeys> {
bool _key(KeyEvent event) {
if (event is! KeyDownEvent || !mounted) return false;
final key = event.logicalKey;
if (markProblemKey.accepts(event, HardwareKeyboard.instance)) {
if (ModalRoute.of(context)?.isCurrent == false) return false;
showMarkProblem(context);
return true;
}
if (key != LogicalKeyboardKey.space && key != LogicalKeyboardKey.tab) return false;
if (SessionState.instance.route.peek() != AppRoute.session) return false;
if (ModalRoute.of(context)?.isCurrent == false) return false;
+20
View File
@@ -65,4 +65,24 @@ void main() {
expect(readies, 1);
expect(panel, findsNothing, reason: 'Space pressed a focused chat button');
});
testWidgets('Ctrl+Shift+B opens Mark a problem; Esc closes it unmarked', (tester) async {
await tester.pumpWidget(MaterialApp(
theme: ThemeData.dark(),
shortcuts: appShortcuts,
home: const AppKeys(child: Scaffold(body: SizedBox())),
));
final card = find.byKey(const ValueKey('mark-problem-card'));
await tester.sendKeyDownEvent(LogicalKeyboardKey.controlLeft);
await tester.sendKeyDownEvent(LogicalKeyboardKey.shiftLeft);
await tester.sendKeyEvent(LogicalKeyboardKey.keyB);
await tester.sendKeyUpEvent(LogicalKeyboardKey.shiftLeft);
await tester.sendKeyUpEvent(LogicalKeyboardKey.controlLeft);
await tester.pumpAndSettle();
expect(card, findsOneWidget);
expect(textInputFocused(), isTrue, reason: 'the note field takes the keys');
await tester.sendKeyEvent(LogicalKeyboardKey.escape);
await tester.pumpAndSettle();
expect(card, findsNothing);
});
}
+47 -3
View File
@@ -1517,9 +1517,14 @@ dedicated server (`package:logging` loggers named by area: `relay`,
`--config`, else `logs` next to the binary (systemd:
`$STATE_DIRECTORY/logs`), else `~/.cache/watch-together/logs`; it
prints the folder at start. Both also print every line to stdout.
The session log (below) sits next to them: the hosting app's server
writes it in the app's logs folder (not for a solo session), the
dedicated one in its log folder.
- **Files** (`LogFiles`): one per day, the next `.1.log`, `.2.log` once a
file passes 10 MiB; files older than 7 days, then the oldest beyond
100 MiB in all, are deleted whenever one is opened. A line is
100 MiB in all (the app's or server's files and the session log's
together, the oldest of either first, never the other writer's file of
today), are deleted whenever one is opened. A line is
`2026-10-12 18:44:27.202 LEVEL logger: message`, the error and stack
trace on the lines below. Writes are synchronous appends (nothing lost
in a crash). Lines from before the folder is known (the app finds it
@@ -1527,7 +1532,7 @@ dedicated server (`package:logging` loggers named by area: `relay`,
- **Redaction** (`redactLog`): every line, in one place
(`formatLogLine`), before console or file: URL query strings cut to
their parameter names (signed CDN links, Jellyfin `api_key`),
`user:pass@` in URLs, member tokens in `/r /m /j /a /i` paths, and the
`user:pass@` in URLs, member tokens in `/r /m /j /a /i /l` paths, and the
values of passwords, secrets, tokens, API keys, cookies,
`Authorization` / `X-Emby-*` headers (`name=value`, `"name":"value"`,
`Name="value"`, `Name: value`) and `Bearer` credentials. Code logs
@@ -1554,12 +1559,51 @@ dedicated server (`package:logging` loggers named by area: `relay`,
source (media opened and how long it took, channel up and down,
reconnects, upstream refusals with the header names sent, re-resolves
of expired format URLs).
- **Never logged**: chat text (the server relays it without a log line,
the app does not log it), and no user-typed free text other than a
problem mark's note. Names, media titles and links (query cut) are.
- **The session log** (`packages/server/lib/src/session_log.dart`,
`packages/core/lib/src/diag_share.dart`): one shared log per server so
a problem can be read afterwards from every side. While in a session
the app (`DiagUploader`, owned by the coordinator) keeps its own
records (not its hosted server's `server.*`, which the server logs
itself), redacted, and sends them every 10 s from the session's 1 s
ticker and when it leaves (before its token goes, at most 1.5 s), as
gzip batches to `POST /l/<sid>/<token>` (PROTOCOL
"Diagnostics"), timestamps moved to the server's clock with the ping
offset (`ClockSync`), the app version in every batch. Until it can
send (reconnecting, no clock sample yet) it keeps up to 2 MiB, the
oldest dropped and counted. The server (`SessionLog`) takes every
member's lines and its own records, holds them 15 s and writes them in
server-time order to `session-YYYY-MM-DD.log` (day and time on the
server's clock; a line later than that, a reconnect's backlog, goes in
with its own time, a little out of order). A line is
`HH:MM:SS.mmm [who] logger: message` (`LEVEL logger` from WARNING up),
`who` = `name (id)` or `server`, continuation lines indented; every
line is redacted again on the server. The server notes once per member
`Bob (m2) shares diagnostics (<version>)` or `Bob (m2) doesn't share
diagnostics`, and `N lines dropped` when the app's buffer overflowed.
Limits and rate caps: PROTOCOL "Diagnostics"; a refused batch stays in
the app's buffer for the next try.
- **Opting out**: Settings → About → "Share diagnostics with the server"
(default on). Off: no lines are kept or sent; one batch per connection
says so (`share:false`). The local log is unchanged.
- **Mark a problem**: Ctrl+Shift+B (`AppKeys`), the player's ⚙ card and
Settings → About open a small card with an optional note (200
characters, one line); the mark goes at once with the lines waiting,
opted out or not, and the server writes `[who] === PROBLEM marked by
<name>: <note> ===` at its time. The app's own log gets
`=== PROBLEM marked: <note> ===` (logger `problem`, never sent). Out
of a session only the app's log gets it. The toast says which.
- **Getting it**: Settings → About → "Copy diagnostics" puts the app and
OS versions, mpv/ffmpeg and yt-dlp versions, whether yt-dlp has
cookies, and the last 6 hours of the log (up to 4 MiB) on the
clipboard, already redacted, and saves the same text as
`diagnostics-<time>.txt` in the logs folder (kept 7 days); "Open logs
folder" opens it.
folder" opens it. In a session it first asks whether to add the
server's session log of the same 6 hours (`GET /l/...`). Any member
may read the session log, not only the host: friends trust each other,
and it is redacted and holds no chat.
## Health reporting
+14 -1
View File
@@ -145,7 +145,20 @@ on Linux (on Windows under `%APPDATA%`; Settings → About → "Open logs
folder" finds it anywhere), with the hosted server's lines
in the same file; a dedicated server prints its folder at start
(`--log-dir`). Users send theirs with Settings → About → "Copy
diagnostics". For a "can't play" report, grep both sides around the
diagnostics".
Start with the session log when there is one: `session-YYYY-MM-DD.log`
in the server's folder (the hosting app's logs folder, or the dedicated
server's `--log-dir`), every member's lines and the server's merged in
server-clock order, each tagged `[name (id)]` or `[server]`. Any member
gets it with "Copy diagnostics" in a session. Find
`=== PROBLEM marked by` first (a member pressed Ctrl+Shift+B), then read
the lines around it from every side; `grep '\[Bob (m2)\]'` follows one
member. `[server] diag:` lines say who shares (with their app version)
and who doesn't, so a missing side is explained. Times are the server's
clock; a member's local log (`app-*.log`) is on its own clock.
For a "can't play" report, grep both sides around the
time: `swap by` (what went up and how), `relay channel` (when the
source dialled in), `media:` (each viewer's requests: status and time to
headers), `mpv:` (`cannot open the media` and ffmpeg's lines), `yt-dlp:`
+43 -3
View File
@@ -1,4 +1,4 @@
# Protocol (proto = 14)
# Protocol (proto = 15)
Wire format only. Behavior (gate, clock, timeouts) is in
[ARCHITECTURE.md](ARCHITECTURE.md). Source of truth:
@@ -26,12 +26,14 @@ with different `proto` refuse each other at `hello` (`proto mismatch`).
"Jellyfin media" below). `/r/<sid>/<token>/...` is the relay of a
member's media (see "Relay" below). `GET /a/<sid>/<token>/<hash>` serves
a member's picture (see "Avatars" below), `GET /i/<sid>/<token>/<hash>`
the current media's art (see "Media art" below).
the current media's art (see "Media art" below). `POST` and `GET
/l/<sid>/<token>` send and read the session log (see "Diagnostics"
below).
## Client to server
```
{"t":"hello","proto":14,"secret":"","user":{"name":"","avatar":""},
{"t":"hello","proto":15,"secret":"","user":{"name":"","avatar":""},
"media":{"name":"","size":0,"duration":0,"url":null},"hostKey":"...",
"resume":"<token>"}
{"t":"ready","ready":true,"source":"manual"} source: manual | auto
@@ -467,3 +469,41 @@ state "queue":[<item>,...], "queuePlay":"q3"
- `state.queue` (omitted when empty) and `state.queuePlay` (omitted when
none) ride in every state, `welcome.state` included. The first item's
`media.relay` shows its warm relay (see "Relay").
## Diagnostics (proto 15)
Each app sends its log lines to the server, which merges them with its
own into the session log (behavior, file and format: ARCHITECTURE.md,
"Logging").
```
POST /l/<sid>/<token> content-type application/gzip, body = gzip of:
{"v":"Starsync 1.4.0 on linux","share":false,
"lines":[{"t":1767225600123,"l":"WARNING","n":"mpv","m":"..."}],
"dropped":12,"marks":[{"t":1767225600500,"note":"..."}]}
GET /l/<sid>/<token>?hours=6
```
- `POST`: one batch. Every field is optional. `v`: the app and version,
noted once per member (and when it changes). `share:false` (omitted
when true): the member does not share; the server notes it once and
drops any `lines`. `lines[]`: `t` epoch ms on the server's clock (the
client converts with its `pong` offset), `l` level name, `n` logger
(`[A-Za-z0-9_.-]`, up to 40, else `app`), `m` the message, already
redacted, an error and stack trace on the lines below (cut to 8192
characters). `dropped`: lines the client dropped since its last batch
(its buffer was full). `marks[]`: "Mark a problem" at `t` with an
optional `note` (one line, cut to 200 characters); it goes in even
with `share:false`.
- Limits: a body of at most 256 KiB, unpacking to at most 1 MiB (else
413); not gzip of a JSON object: 400. Per member and minute at most 30
posts and 4 MiB of lines, else 429 with `retry-after`: the client keeps
the batch and sends it later. Unknown token: 401. Success: 204, also
on a server that keeps no session log (the batch is dropped).
- `GET`: the session log of the last `hours` (1 to 168, default 6; at
most 8 MiB, the newest kept), lines the server still holds for
ordering included, as gzip of UTF-8 text (`application/gzip`). Any
member may read it (it is redacted and holds no chat). 401 for an
unknown token, 404 on a server that keeps no session log.
- Clients send every 10 s and when they leave; each batch carries at
most 256 KiB of lines.
+1
View File
@@ -1,6 +1,7 @@
export 'src/constants.dart';
export 'src/errors.dart';
export 'src/logging.dart';
export 'src/diag_share.dart';
export 'src/clock_sync.dart';
export 'src/config.dart';
export 'src/protocol.dart';
+17 -1
View File
@@ -1,5 +1,5 @@
abstract class RelayConstants {
static const int protoVersion = 14;
static const int protoVersion = 15;
// Relay (`/r/...`): a member's bytes through the server's buffer.
static const int relaySliceBytes = 1 << 20;
@@ -70,4 +70,20 @@ abstract class RelayConstants {
static const int maxQueueItems = 100;
static const int maxQueueTextLength = 200;
static const int queuePlayTimeoutSecs = 30;
/// Diagnostics sharing (`/l/...`, ARCHITECTURE "Logging"): a member's
/// batch carries at most [diagBatchBytes] of lines (gzip on the wire, at
/// most [diagBatchBytes] compressed and [diagDecodedBytes] unpacked); it
/// keeps up to [diagBufferBytes] while it cannot send (oldest dropped)
/// and sends every [diagIntervalSecs]. The server takes at most
/// [diagPostsPerMin] posts and [diagBytesPerMin] unpacked bytes per member
/// and minute (429 beyond). A "Mark a problem" note is cut to
/// [maxProblemNoteLength] characters.
static const int diagBatchBytes = 256 * 1024;
static const int diagDecodedBytes = 1 << 20;
static const int diagBufferBytes = 2 << 20;
static const int diagIntervalSecs = 10;
static const int diagPostsPerMin = 30;
static const int diagBytesPerMin = 4 << 20;
static const int maxProblemNoteLength = 200;
}
+389
View File
@@ -0,0 +1,389 @@
import 'dart:async';
import 'dart:collection';
import 'dart:convert';
import 'dart:io';
import 'dart:typed_data';
import 'package:logging/logging.dart';
import 'constants.dart';
import 'logging.dart';
// Diagnostics sharing (ARCHITECTURE "Logging", PROTOCOL "Diagnostics"):
// each app sends its log lines to the server, which merges them with its
// own into one session log. The batch format both sides use, and the
// app's sender.
/// One log record of a member: [t] in epoch ms (the app's clock while it
/// waits, the server's on the wire), the level name, the logger and the
/// redacted message (an error and stack trace on the lines below).
class DiagLine {
final int t;
final String level;
final String logger;
final String message;
const DiagLine({required this.t, required this.level, required this.logger, required this.message});
/// Roughly its bytes in a batch.
int get size => message.length + logger.length + level.length + 32;
Map<String, Object?> toJson() => {'t': t, 'l': level, 'n': logger, 'm': message};
static DiagLine? fromJson(Object? j) {
if (j is! Map) return null;
final t = j['t'], l = j['l'], n = j['n'], m = j['m'];
if (t is! num || !t.isFinite || m is! String) return null;
return DiagLine(t: t.toInt(), level: l is String ? l : 'INFO', logger: n is String ? n : 'app', message: m);
}
}
/// A "Mark a problem" from a member: when (epoch ms, server clock) and
/// its optional note.
class DiagMark {
final int t;
final String note;
const DiagMark({required this.t, this.note = ''});
Map<String, Object?> toJson() => {'t': t, if (note.isNotEmpty) 'note': note};
static DiagMark? fromJson(Object? j) {
if (j is! Map) return null;
final t = j['t'], note = j['note'];
if (t is! num || !t.isFinite) return null;
return DiagMark(t: t.toInt(), note: note is String ? note : '');
}
}
/// The body of `POST /l/<sid>/<token>` (gzip of its JSON): the app's
/// [version], whether it [share]s its lines (false: none, the server notes
/// it once), the [lines], how many it [dropped] since the last batch (its
/// buffer was full) and any problem [marks].
class DiagBatch {
final String version;
final bool share;
final List<DiagLine> lines;
final int dropped;
final List<DiagMark> marks;
const DiagBatch({this.version = '', this.share = true, this.lines = const [], this.dropped = 0, this.marks = const []});
Map<String, Object?> toJson() => {
if (version.isNotEmpty) 'v': version,
if (!share) 'share': false,
if (lines.isNotEmpty) 'lines': [for (final l in lines) l.toJson()],
if (dropped > 0) 'dropped': dropped,
if (marks.isNotEmpty) 'marks': [for (final m in marks) m.toJson()],
};
List<int> encode() => gzip.encode(utf8.encode(jsonEncode(toJson())));
factory DiagBatch.fromJson(Map<String, Object?> j) {
final lines = j['lines'], marks = j['marks'], dropped = j['dropped'], v = j['v'];
return DiagBatch(
version: v is String ? v : '',
share: j['share'] != false,
lines: lines is List ? lines.map(DiagLine.fromJson).whereType<DiagLine>().toList() : const [],
dropped: dropped is num && dropped.isFinite && dropped > 0 ? dropped.toInt() : 0,
marks: marks is List ? marks.map(DiagMark.fromJson).whereType<DiagMark>().toList() : const [],
);
}
/// Unpacks a body; throws [FormatException] when it is not gzip of a
/// batch or unpacks to more than [maxBytes].
static DiagBatch decode(List<int> body, {int maxBytes = RelayConstants.diagDecodedBytes}) {
final text = utf8.decode(gunzipCapped(body, maxBytes), allowMalformed: true);
final j = jsonDecode(text);
if (j is! Map) throw const FormatException('not a batch');
return DiagBatch.fromJson(Map<String, Object?>.from(j));
}
}
/// [body] gunzipped, or [FormatException] once it passes [maxBytes] (a
/// small body must not unpack to gigabytes).
List<int> gunzipCapped(List<int> body, int maxBytes) {
final out = BytesBuilder(copy: false);
final sink = _CappedSink(out, maxBytes);
try {
gzip.decoder.startChunkedConversion(sink)
..add(body)
..close();
} on FormatException {
rethrow;
} catch (e) {
throw FormatException('not gzip: $e');
}
return out.takeBytes();
}
class _CappedSink implements Sink<List<int>> {
final BytesBuilder out;
final int max;
_CappedSink(this.out, this.max);
@override
void add(List<int> data) {
if (out.length + data.length > max) throw const FormatException('too large');
out.add(data);
}
@override
void close() {}
}
/// Where the app's lines go: the server's base URL (`http://host:port`),
/// this member's token and the server's clock minus the local one (ms).
typedef DiagTarget = ({Uri base, String token, double offsetMs});
/// The app's side: keeps its log records (not the hosted server's, which
/// the server logs itself) while in a session, and sends them to the
/// server every [RelayConstants.diagIntervalSecs] ([tick]) and on [stop].
/// [target] is null while there is nowhere to send (reconnecting, no
/// clock sample yet): lines wait, up to [bufferBytes], the oldest dropped
/// and counted. With [share] off nothing is kept or sent but one batch
/// saying so per connection; problem marks ([markProblem]) go either way.
class DiagUploader {
final DiagTarget? Function() target;
/// The app and its version, e.g. `Watch Together 1.4.0 on linux`.
final String version;
final int bufferBytes;
final int batchBytes;
final Duration interval;
final DateTime Function() _now;
DiagUploader({
required this.target,
required this.version,
this.bufferBytes = RelayConstants.diagBufferBytes,
this.batchBytes = RelayConstants.diagBatchBytes,
this.interval = const Duration(seconds: RelayConstants.diagIntervalSecs),
DateTime Function()? now,
}) : _now = now ?? DateTime.now;
/// Loggers never sent: the hosted server's records (`server.*`, the
/// server logs them itself) and the local copy of a mark ([markLogger]).
static bool sends(String logger) => logger != 'server' && !logger.startsWith('server.') && logger != markLogger;
/// The local log's copy of a problem mark.
static const markLogger = 'problem';
static final _markLog = Logger(markLogger);
final _lines = Queue<DiagLine>();
var _size = 0;
var _dropped = 0;
final _marks = <DiagMark>[];
StreamSubscription<LogRecord>? _sub;
HttpClient? _http;
var _busy = false;
var _again = false;
DateTime? _last;
/// The token the server was told "doesn't share" for.
String? _toldOff;
var _share = true;
bool get running => _sub != null;
/// Lines waiting to go.
int get pending => _lines.length;
/// Share the log lines (Settings); off drops what waits.
bool get share => _share;
set share(bool on) {
if (on == _share) return;
_share = on;
if (!on) _clear();
_toldOff = null;
}
/// Starts keeping the log's records (a session starts).
void start() {
_sub ??= Logger.root.onRecord.listen(_onRecord);
_last = _now();
}
void _onRecord(LogRecord r) {
if (!_share || !sends(r.loggerName)) return;
final b = StringBuffer(r.message);
if (r.error != null) b.write('\n error: ${r.error}');
final s = '${r.stackTrace ?? ''}'.trimRight();
if (s.isNotEmpty) b.write('\n$s');
add(DiagLine(t: r.time.millisecondsSinceEpoch, level: r.level.name, logger: r.loggerName, message: redactLog('$b')));
}
/// Keeps [line] (local clock) for the next batch.
void add(DiagLine line) {
_lines.add(line);
_size += line.size;
_trim();
}
void _trim() {
while (_size > bufferBytes && _lines.isNotEmpty) {
_size -= _lines.removeFirst().size;
_dropped++;
}
}
void _clear() {
_lines.clear();
_size = 0;
_dropped = 0;
}
/// "Mark a problem": a marker line in the server's session log with
/// [note] (cut to [RelayConstants.maxProblemNoteLength], one line), sent
/// now with the lines waiting. Also written to the local log. True once
/// the server has it; false when it waits (not connected).
Future<bool> markProblem(String note) {
var clean = note.replaceAll(RegExp(r'[\x00-\x1f\x7f]+'), ' ').trim();
if (clean.length > RelayConstants.maxProblemNoteLength) {
clean = clean.substring(0, RelayConstants.maxProblemNoteLength);
}
_markLog.warning('=== PROBLEM marked${clean.isEmpty ? '' : ': $clean'} ===');
_marks.add(DiagMark(t: _now().millisecondsSinceEpoch, note: redactLog(clean)));
return flush();
}
/// Once a second (the app's ticker): sends when [interval] has passed.
void tick() {
final last = _last;
if (last != null && _now().difference(last) < interval) return;
unawaited(flush());
}
/// Sends what waits: true when the server took all of it (or there was
/// nothing to send).
Future<bool> flush() async {
if (_busy) {
// Once the one under way is done (a mark goes now, not next tick).
_again = true;
return false;
}
final tg = target();
if (tg == null) return false;
_busy = true;
_last = _now();
try {
final offset = tg.offsetMs.round();
List<DiagMark> takeMarks() {
final m = [for (final x in _marks) DiagMark(t: x.t + offset, note: x.note)];
_marks.clear();
return m;
}
if (!_share) {
if (_toldOff == tg.token && _marks.isEmpty) return true;
final marks = takeMarks();
final ok = await _post(tg, DiagBatch(version: version, share: false, marks: marks));
if (ok) {
_toldOff = tg.token;
} else {
_marks.insertAll(0, [for (final m in marks) DiagMark(t: m.t - offset, note: m.note)]);
}
return ok;
}
// A few batches at most per flush; the rest goes next time.
for (var i = 0; i < 8 && (_lines.isNotEmpty || _marks.isNotEmpty || _dropped > 0); i++) {
final taken = <DiagLine>[];
var bytes = 0;
while (_lines.isNotEmpty && (taken.isEmpty || bytes + _lines.first.size <= batchBytes)) {
final l = _lines.removeFirst();
_size -= l.size;
bytes += l.size;
taken.add(l);
}
final dropped = _dropped;
_dropped = 0;
final marks = takeMarks();
final batch = DiagBatch(
version: version,
lines: [for (final l in taken) DiagLine(t: l.t + offset, level: l.level, logger: l.logger, message: l.message)],
dropped: dropped,
marks: marks,
);
if (!await _post(tg, batch)) {
// Back in front, in order; the cap applies again.
for (final l in taken.reversed) {
_lines.addFirst(l);
_size += l.size;
}
_dropped += dropped;
_marks.insertAll(0, [for (final m in marks) DiagMark(t: m.t - offset, note: m.note)]);
_trim();
return false;
}
}
return _lines.isEmpty;
} finally {
_busy = false;
if (_again) {
_again = false;
if (_sub != null) unawaited(flush());
}
}
}
static Uri _httpUri(Uri base, String path, [Map<String, String>? query]) => base.replace(
scheme: base.scheme == 'wss' || base.scheme == 'https' ? 'https' : 'http',
path: path,
queryParameters: query,
fragment: null);
/// The server's `http://host:port` for a `/sync` socket URL.
static Uri baseOf(Uri wsUri) => _httpUri(wsUri, '/');
Future<bool> _post(DiagTarget tg, DiagBatch batch) async {
try {
final http = _http ??= HttpClient()..connectionTimeout = const Duration(seconds: 5);
final req = await http.postUrl(_httpUri(tg.base, '/l/default/${tg.token}')).timeout(const Duration(seconds: 5));
final body = batch.encode();
req.headers.contentType = ContentType('application', 'gzip');
req.contentLength = body.length;
req.add(body);
final res = await req.close().timeout(const Duration(seconds: 10));
await res.drain<void>();
// 413: this batch can never go (should not happen); drop it.
return res.statusCode == 204 || res.statusCode == 200 || res.statusCode == 413;
} catch (_) {
return false;
}
}
/// The server's session log of the last [hours] (every member and the
/// server, merged), or null when it cannot be had (not connected, a
/// server without one, an older server).
Future<String?> fetchSessionLog({int hours = 6}) async {
final tg = target();
if (tg == null) return null;
try {
final http = _http ??= HttpClient()..connectionTimeout = const Duration(seconds: 5);
final req = await http
.getUrl(_httpUri(tg.base, '/l/default/${tg.token}', {'hours': '$hours'}))
.timeout(const Duration(seconds: 5));
final res = await req.close().timeout(const Duration(seconds: 10));
final bytes = <int>[];
await for (final c in res.timeout(const Duration(seconds: 30))) {
bytes.addAll(c);
}
if (res.statusCode != 200) return null;
return utf8.decode(gunzipCapped(bytes, 64 << 20), allowMalformed: true);
} catch (_) {
return null;
}
}
/// The session ends: what waits goes now (best effort, [timeout]), then
/// nothing is kept until the next [start].
Future<void> stop({Duration timeout = const Duration(milliseconds: 1500)}) async {
await _sub?.cancel();
_sub = null;
if ((_lines.isNotEmpty || _marks.isNotEmpty || _dropped > 0) && target() != null) {
try {
await flush().timeout(timeout);
} catch (_) {}
}
_clear();
_marks.clear();
_toldOff = null;
_last = null;
_http?.close(force: true);
_http = null;
}
}
+39 -14
View File
@@ -10,6 +10,10 @@ import 'package:logging/logging.dart';
Logger createLogger(String name) => Logger(name);
/// The server's combined log of every member and itself
/// (`session-YYYY-MM-DD.log`, ARCHITECTURE "Logging").
const sessionLogPrefix = 'session';
var _listening = false;
var _console = true;
LogFiles? _files;
@@ -35,12 +39,13 @@ void initLogging({Level level = Level.INFO, bool console = true}) {
/// Writes the files in [dir] from now on: `<prefix>-YYYY-MM-DD.log`,
/// rotated by day and [LogFiles.maxFileBytes], pruned by age and total
/// size. Lines logged before go in first. A folder that cannot be written
/// leaves logging on the console only (and says so there).
/// size (shared with the session log, [sessionLogPrefix], in the same
/// folder). Lines logged before go in first. A folder that cannot be
/// written leaves logging on the console only (and says so there).
LogFiles? startLogFiles(String dir, {String prefix = 'app'}) {
if (_files != null) return _files;
try {
final f = LogFiles(dir, prefix: prefix)..open();
final f = LogFiles(dir, prefix: prefix, shareWith: const [sessionLogPrefix])..open();
_files = f;
for (final l in _early) {
f.write(l);
@@ -123,13 +128,13 @@ final _headerSecret = RegExp(
final _pairSecret = RegExp('\\b($_secretNames)\\s*=\\s*[^&\\s"\',;)}\\]]+', caseSensitive: false);
final _bearer = RegExp(r'\b(Bearer|Basic)\s+[A-Za-z0-9._~+/=-]{8,}', caseSensitive: false);
final _url = RegExp(r'''\b(?:https?|wss?)://[^\s"'<>]+''', caseSensitive: false);
// A member token in a server path: /r/<sid>/<token>/..., /m, /j, /a, /i.
final _pathToken = RegExp(r'(/[rmjai]/[^/\s?#]+/)[0-9a-fA-F]{32}\b');
// A member token in a server path: /r/<sid>/<token>/..., /m, /j, /a, /i, /l.
final _pathToken = RegExp(r'(/[rmjail]/[^/\s?#]+/)[0-9a-fA-F]{32}\b');
final _userInfo = RegExp(r'^([a-z]+://)[^/@\s]+@', caseSensitive: false);
/// [s] with nothing secret left: every URL's query string down to its
/// parameter names (signed CDN links, `api_key`), user:password@ in URLs,
/// member tokens in `/r/ /m/ /j/ /a/ /i/` paths, and the values of
/// member tokens in `/r/ /m/ /j/ /a/ /i/ /l/` paths, and the values of
/// passwords, tokens, API keys, cookies and auth headers. Applied to
/// every log line ([formatLogLine]).
String redactLog(String s) {
@@ -203,16 +208,21 @@ class LogThrottle {
/// Rotating log files in [dir]: `<prefix>-YYYY-MM-DD.log` (then `.1.log`,
/// `.2.log` ... once one passes [maxFileBytes]); files older than
/// [keepDays] and the oldest beyond [maxTotalBytes] are deleted whenever a
/// file is opened.
/// file is opened. [shareWith]: other prefixes in [dir] that count toward
/// the same [maxTotalBytes] (the app's or server's log and the session
/// log); the oldest of all of them go first, never another prefix's file
/// of today (its writer may have it open).
class LogFiles {
final String dir;
final String prefix;
final List<String> shareWith;
final int maxFileBytes;
final int maxTotalBytes;
final int keepDays;
final DateTime Function() _now;
LogFiles(this.dir,
{this.prefix = 'app',
this.shareWith = const [],
this.maxFileBytes = 10 * 1024 * 1024,
this.maxTotalBytes = 100 * 1024 * 1024,
this.keepDays = 7,
@@ -229,7 +239,10 @@ class LogFiles {
String _dayOf(DateTime t) => '${t.year}-${_two(t.month)}-${_two(t.day)}';
String _dayOfFile(File f) => f.uri.pathSegments.last.substring(prefix.length + 1, prefix.length + 11);
String _dayOfFile(File f) => _fileDay(prefix, f);
static String _fileDay(String prefix, File f) =>
f.uri.pathSegments.last.substring(prefix.length + 1, prefix.length + 11);
/// Opens today's file (creating [dir]); throws when it cannot.
void open() {
@@ -256,9 +269,10 @@ class LogFiles {
}
/// Appends [line]; a new day or a full file moves on to the next file.
void write(String line) {
/// [at]: the line's own time picks the day's file (else now).
void write(String line, {DateTime? at}) {
try {
final day = _dayOf(_now());
final day = _dayOf(at ?? _now());
if (_out == null || day != _day || _size >= maxFileBytes) _roll(day);
final bytes = utf8.encode('$line\n');
_out!.writeFromSync(bytes);
@@ -269,7 +283,10 @@ class LogFiles {
}
/// This prefix's files, oldest first.
List<File> files() {
List<File> files() => filesOf(prefix);
/// The files of [prefix] in [dir], oldest first.
List<File> filesOf(String prefix) {
final d = Directory(dir);
if (!d.existsSync()) return const [];
final name = RegExp('^${RegExp.escape(prefix)}-(\\d{4}-\\d{2}-\\d{2})(?:\\.(\\d+))?\\.log\$');
@@ -295,10 +312,18 @@ class LogFiles {
kept.add(f);
}
}
var total = kept.fold<int>(0, (a, f) => a + _len(f));
for (final f in kept) {
final today = _dayOf(_now());
// Every file under the budget, oldest day first: (day, prefix, file).
final all = [
for (final f in kept) (_dayOfFile(f), true, f),
for (final p in shareWith)
if (p != prefix)
for (final f in filesOf(p)) (_fileDay(p, f), false, f),
]..sort((a, b) => a.$1.compareTo(b.$1));
var total = all.fold<int>(0, (a, f) => a + _len(f.$3));
for (final (day, own, f) in all) {
if (total <= maxTotalBytes) break;
if (f.path == _path) continue;
if (f.path == _path || (!own && day == today)) continue;
total -= _len(f);
_delete(f);
}
+87
View File
@@ -0,0 +1,87 @@
import 'dart:convert';
import 'dart:io';
import 'package:logging/logging.dart';
import 'package:sync_relay_core/core.dart';
import 'package:test/test.dart';
void main() {
test('a batch survives gzip both ways; a bomb or junk is refused', () {
const b = DiagBatch(
version: 'WT 1.0',
lines: [DiagLine(t: 5, level: 'WARNING', logger: 'mpv', message: 'x\n error: y')],
dropped: 2,
marks: [DiagMark(t: 7, note: 'lag')]);
final back = DiagBatch.decode(b.encode());
expect(back.toJson(), b.toJson());
expect(DiagBatch.decode(const DiagBatch(share: false).encode()).share, isFalse);
expect(() => DiagBatch.decode(gzip.encode(List.filled(2 << 20, 32)), maxBytes: 1 << 20), throwsFormatException);
expect(() => DiagBatch.decode(utf8.encode('plain')), throwsFormatException);
});
group('DiagUploader', () {
late HttpServer http;
final got = <DiagBatch>[];
var accept = true;
setUp(() async {
got.clear();
accept = true;
http = await HttpServer.bind(InternetAddress.loopbackIPv4, 0);
http.listen((req) async {
final body = await req.fold<List<int>>([], (a, c) => a..addAll(c));
if (accept) got.add(DiagBatch.decode(body));
req.response.statusCode = accept ? 204 : 503;
await req.response.close();
});
});
tearDown(() => http.close(force: true));
DiagUploader make({bool connected = true, int buffer = 1 << 20}) => DiagUploader(
target: () => connected ? (base: Uri.parse('http://127.0.0.1:${http.port}'), token: 't', offsetMs: 1000) : null,
version: 'WT 1.0',
bufferBytes: buffer);
test('sends its records on the server clock, not the hosted server\'s; keeps them while it cannot', () async {
final u = make()..start();
Logger('relay').info('hello api_key=abc');
Logger('server.relay').info('the server logs this itself');
await Future<void>.delayed(Duration.zero);
accept = false;
expect(await u.flush(), isFalse);
expect(u.pending, 1);
accept = true;
expect(await u.flush(), isTrue);
final line = got.single.lines.single;
expect(line.message, 'hello api_key=***');
expect(line.logger, 'relay');
expect(got.single.version, 'WT 1.0');
await u.stop();
});
test('a full buffer drops the oldest and says how many', () async {
final u = make(buffer: 400);
for (var i = 0; i < 10; i++) {
u.add(DiagLine(t: i, level: 'INFO', logger: 'a', message: 'line $i ${'x' * 40}'));
}
expect(await u.flush(), isTrue);
expect(got.single.dropped, greaterThan(0));
expect(got.single.lines.last.message, startsWith('line 9'));
expect(got.single.lines.first.t, greaterThanOrEqualTo(1000), reason: 'offset to the server clock');
});
test('opted out: no lines, one "does not share" per connection; marks still go', () async {
final u = make()
..share = false
..start();
Logger('relay').info('private');
expect(await u.flush(), isTrue);
expect(await u.flush(), isTrue);
expect(got.length, 1);
expect(got.single.share, isFalse);
expect(got.single.lines, isEmpty);
expect(await u.markProblem('it\nfroze'), isTrue);
expect(got.last.marks.single.note, 'it froze');
await u.stop();
});
});
}
+12
View File
@@ -19,6 +19,7 @@ void main() {
expect(redactLog('open http://1.2.3.4:8080/r/default/$tok/2/0/name'), 'open http://1.2.3.4:8080/r/default/***/2/0/name');
expect(redactLog('ws://h:1/r/default/$tok/source?q=7'), 'ws://h:1/r/default/***/source?q=…');
expect(redactLog('/m/default/$tok/file.mkv range=bytes=0-'), '/m/default/***/file.mkv range=bytes=0-');
expect(redactLog('POST /l/default/$tok'), 'POST /l/default/***');
expect(redactLog('edl://!no_clip;%160%http://h/r/default/$tok/2/0/n;!new_stream;!no_clip;%160%http://h/r/default/$tok/2/1/n'),
isNot(contains(tok)));
});
@@ -85,5 +86,16 @@ void main() {
expect(small.files().map((x) => x.uri.pathSegments.last), ['app-2026-10-12.1.log', 'app-2026-10-13.log']);
small.close();
});
test('the session log shares the total budget, never touching another writer\'s today', () {
final now = DateTime(2026, 10, 12, 10);
File('${dir.path}/session-2026-10-10.log').writeAsStringSync('s' * 100);
File('${dir.path}/session-2026-10-12.log').writeAsStringSync('s' * 100);
File('${dir.path}/app-2026-10-11.log').writeAsStringSync('a' * 100);
final f = LogFiles(dir.path, maxTotalBytes: 150, shareWith: const ['session'], now: () => now)..open();
final left = dir.listSync().map((e) => e.uri.pathSegments.last).toSet();
expect(left, {'session-2026-10-12.log', 'app-2026-10-12.log'});
f.close();
});
});
}
+5 -2
View File
@@ -31,7 +31,8 @@ Future<void> main(List<String> args) async {
'An account login: keep it private. Copied into --yt-dlp-dir at start.')
..addOption('config', help: 'YAML or JSON config file (keys as in AppConfig); overrides the other flags.')
..addOption('log-dir',
help: 'Folder for the rotating log files (server-YYYY-MM-DD.log, kept 7 days). '
help: 'Folder for the rotating log files (server-YYYY-MM-DD.log, and session-YYYY-MM-DD.log with '
'every member\'s shared diagnostics merged in; kept 7 days). '
'Default: logs next to --config, else logs next to this binary (systemd: \$STATE_DIRECTORY/logs).')
..addFlag('upgrade',
negatable: false,
@@ -117,7 +118,9 @@ Future<void> main(List<String> args) async {
}
final events = ReceivePort('server events');
final done = serve(config.toMap(), events: events.sendPort);
// The session log (every member's diagnostics and the server's records
// merged) goes next to the server's own.
final done = serve({...config.toMap(), if (files != null) 'logDir': files.dir}, events: events.sendPort);
final signals = <StreamSubscription<ProcessSignal>>[];
await for (final msg in events) {
if (msg is! Map) continue;
+1
View File
@@ -7,3 +7,4 @@ export 'src/jellyfin_relay.dart';
export 'src/relay_buffer.dart';
export 'src/relay_routes.dart';
export 'src/relay_stats.dart';
export 'src/session_log.dart';
+5 -2
View File
@@ -19,7 +19,8 @@ import 'sync_server.dart';
/// swap; a following `swap` with its name and size puts it up) and answers
/// `{'ok':true,'name':n,'size':s}` or `{'ok':false}`. Without [events] a
/// start failure throws. `'loopbackOnly': true` in [config] (next to the
/// [AppConfig] keys) binds 127.0.0.1 only: the app's solo session.
/// [AppConfig] keys) binds 127.0.0.1 only: the app's solo session;
/// `'logDir'` is the folder of the session log ([SyncServer.logDir]).
Future<void> serve(Map<String, Object?> config, {SendPort? events}) async {
StreamSubscription<pkg_logging.LogRecord>? logs;
if (events != null) {
@@ -34,7 +35,9 @@ Future<void> serve(Map<String, Object?> config, {SendPort? events}) async {
}
final SyncServer server;
try {
server = SyncServer(AppConfig.fromMap(config), loopbackOnly: config['loopbackOnly'] == true);
final logDir = config['logDir'];
server = SyncServer(AppConfig.fromMap(config),
loopbackOnly: config['loopbackOnly'] == true, logDir: logDir is String && logDir.isNotEmpty ? logDir : null);
await server.start();
} catch (e) {
await logs?.cancel();
+288
View File
@@ -0,0 +1,288 @@
import 'dart:async';
import 'dart:convert';
import 'dart:io';
import 'dart:typed_data';
import 'package:logging/logging.dart' as pkg_logging;
import 'package:relic/relic.dart';
import 'package:sync_relay_core/core.dart';
import 'server_hub.dart';
import 'session.dart';
// The session log (ARCHITECTURE "Logging"): every member's diagnostics
// (`POST /l/...`) and the server's own records, merged in server-clock
// order into one file per day, `session-YYYY-MM-DD.log`, in the server's
// log folder.
class _Entry {
final int t;
final int seq;
final String text;
const _Entry(this.t, this.seq, this.text);
}
/// The merged log. Lines wait [window] (members send every 10 s) and are
/// written in time order; a line later than that (a reconnect's backlog)
/// still goes in with its own time, a little out of order.
class SessionLog {
final LogFiles files;
final Duration window;
final DateTime Function() _now;
final _pending = <_Entry>[];
var _seq = 0;
StreamSubscription<pkg_logging.LogRecord>? _own;
/// Per member token: the version and share state already noted.
final _noted = <String, ({String version, bool share})>{};
SessionLog(String dir,
{this.window = const Duration(seconds: 15),
List<String> shareWith = const ['app', 'server'],
DateTime Function()? now,
int maxFileBytes = 10 * 1024 * 1024,
int maxTotalBytes = 100 * 1024 * 1024})
: _now = now ?? DateTime.now,
files = LogFiles(dir,
prefix: sessionLogPrefix,
shareWith: shareWith,
maxFileBytes: maxFileBytes,
maxTotalBytes: maxTotalBytes,
now: now) {
files.open();
}
/// Takes this isolate's own log records as the server's lines.
void captureOwn() {
_own ??= pkg_logging.Logger.root.onRecord.listen((r) {
final b = StringBuffer(r.message);
if (r.error != null) b.write('\n error: ${r.error}');
final s = '${r.stackTrace ?? ''}'.trimRight();
if (s.isNotEmpty) b.write('\n$s');
add(r.time.millisecondsSinceEpoch, 'server', r.level.name, r.loggerName, '$b');
});
}
static final _control = RegExp(r'[\x00-\x08\x0b-\x1f\x7f]');
static final _loggerName = RegExp(r'^[A-Za-z0-9_.\-]{1,40}$');
static const _levels = {'FINEST', 'FINER', 'FINE', 'CONFIG', 'INFO', 'WARNING', 'SEVERE', 'SHOUT'};
static const _quiet = {'FINEST', 'FINER', 'FINE', 'CONFIG', 'INFO'};
static String _clean(String s, int max) {
var out = s.replaceAll('\r', '').replaceAll(_control, ' ');
if (out.length > max) out = '${out.substring(0, max)}…';
return out;
}
static String _two(int n) => n.toString().padLeft(2, '0');
static String _clock(DateTime t) =>
'${_two(t.hour)}:${_two(t.minute)}:${_two(t.second)}.${t.millisecond.toString().padLeft(3, '0')}';
/// One record at [t] (epoch ms, server clock) from [who] (`name (id)`
/// or `server`): `HH:MM:SS.mmm [who] logger: message`, the level before
/// the logger from WARNING up, continuation lines indented. Redacted
/// again here.
void add(int t, String who, String level, String logger, String message) {
final lvl = _levels.contains(level) ? level : 'INFO';
final name = _loggerName.hasMatch(logger) ? logger : 'app';
final body = redactLog(_clean(message, 8192)).split('\n');
final head = '[${_clean(who, 80).replaceAll('\n', ' ')}] ${_quiet.contains(lvl) ? '' : '$lvl '}$name: ${body.first}';
_push(t, [head, for (final l in body.skip(1)) ' $l'].join('\n'));
}
/// A problem mark by [name] ([who]) at [t]:
/// `=== PROBLEM marked by <name>: <note> ===`.
void mark(int t, String who, String name, String note) {
final n = redactLog(_clean(note, RelayConstants.maxProblemNoteLength).replaceAll('\n', ' ')).trim();
_push(t, '[${_clean(who, 80).replaceAll('\n', ' ')}] === PROBLEM marked by $name${n.isEmpty ? '' : ': $n'} ===');
}
/// A line of the server's about a member, now.
void note(String message) => add(_now().millisecondsSinceEpoch, 'server', 'INFO', 'diag', message);
void _push(int t, String text) {
final now = _now().millisecondsSinceEpoch;
// A clock far ahead lands now; older than the files keep is dropped.
if (t > now + 60000) t = now;
if (t < now - files.keepDays * 86400000) return;
_pending.add(_Entry(t, _seq++, text));
// A flood must not grow the wait without bound: write early.
if (_pending.length > 20000) flush(all: true);
}
/// Writes the lines older than [window] ([all]: every one), in order.
/// Called every second.
void flush({bool all = false}) {
if (_pending.isEmpty) return;
final until = all ? null : _now().millisecondsSinceEpoch - window.inMilliseconds;
_pending.sort(_order);
var n = 0;
for (final e in _pending) {
if (until != null && e.t > until) break;
final at = DateTime.fromMillisecondsSinceEpoch(e.t);
files.write('${_clock(at)} ${e.text}', at: at);
n++;
}
_pending.removeRange(0, n);
}
static int _order(_Entry a, _Entry b) => a.t != b.t ? a.t.compareTo(b.t) : a.seq.compareTo(b.seq);
/// A batch from the member [token] ([who] for the log, [name] for a
/// mark): its lines, a note the first time (or when its version or
/// sharing changes), how many it dropped, its marks.
void takeBatch(String token, String who, String name, DiagBatch batch) {
final version = _clean(batch.version, 100).replaceAll('\n', ' ');
final before = _noted[token];
if (before == null || before.version != version || before.share != batch.share) {
_noted[token] = (version: version, share: batch.share);
final what = version.isEmpty ? '' : ' ($version)';
if (!batch.share) {
note("$who doesn't share diagnostics$what");
} else if (before == null || !before.share) {
note('$who shares diagnostics$what');
} else {
note('$who now runs $version');
}
}
if (batch.share) {
for (final l in batch.lines) {
add(l.t, who, l.level, l.logger, l.message);
}
if (batch.dropped > 0) {
final t = batch.lines.isEmpty ? _now().millisecondsSinceEpoch : batch.lines.first.t;
add(t, who, 'WARNING', 'diag', '${batch.dropped} lines dropped (the app could not send them in time)');
}
}
for (final m in batch.marks) {
mark(m.t, who, name, m.note);
}
}
/// The lines of the last [since], written and waiting, oldest first; at
/// most about [maxBytes], the newest kept.
String recent(Duration since, {int maxBytes = 8 * 1024 * 1024}) {
final from = _now().subtract(since);
final fromDay = _day(from);
final out = <String>[];
final head = RegExp(r'^(\d{2}):(\d{2}):(\d{2})\.(\d{3}) ');
for (final f in files.files()) {
final name = f.uri.pathSegments.last;
final day = name.substring(sessionLogPrefix.length + 1, sessionLogPrefix.length + 11);
if (day.compareTo(fromDay) < 0) continue;
final date = DateTime.tryParse(day);
if (date == null) continue;
List<String> lines;
try {
lines = const LineSplitter().convert(utf8.decode(f.readAsBytesSync(), allowMalformed: true));
} catch (_) {
continue;
}
var keep = false;
for (final l in lines) {
final m = head.firstMatch(l);
if (m != null) {
final t = DateTime(date.year, date.month, date.day, int.parse(m[1]!), int.parse(m[2]!), int.parse(m[3]!),
int.parse(m[4]!));
keep = !t.isBefore(from);
}
if (keep) out.add(l);
}
}
final fromMs = from.millisecondsSinceEpoch;
final waiting = [..._pending]..sort(_order);
for (final e in waiting) {
if (e.t >= fromMs) out.add('${_clock(DateTime.fromMillisecondsSinceEpoch(e.t))} ${e.text}');
}
var size = 0, start = out.length;
while (start > 0 && size + out[start - 1].length + 1 <= maxBytes) {
size += out[--start].length + 1;
}
return out.sublist(start).join('\n');
}
static String _day(DateTime t) => '${t.year}-${_two(t.month)}-${_two(t.day)}';
Future<void> close() async {
await _own?.cancel();
_own = null;
flush(all: true);
files.close();
}
}
class _Rate {
var start = DateTime.now();
var posts = 0;
var bytes = 0;
}
/// `POST /l/<sid>/<token>`: a member's diagnostics batch ([DiagBatch],
/// gzip) into the [sessionLog]; `GET /l/<sid>/<token>?hours=N`: the session log of
/// the last N hours (gzip text), for any member. Without one (a
/// server with no log folder, the solo session) batches are taken and
/// dropped, and the GET is 404.
void mountSessionLogRoutes(RelicApp app, ServerHub hub, SessionLog? Function() sessionLog) {
Response status(int code, [String? why]) => Response(code, body: Body.fromString(why ?? 'status $code'));
final rates = <String, _Rate>{};
ViewerToken? member(Request req) {
final session = hub.sessions.get(req.pathParameters.raw[#sid] ?? '');
final token = req.pathParameters.raw[#token] ?? '';
if (session == null || session.gate(token) != MediaGate.ok) return null;
return session.tokens[token];
}
app.post('/l/:sid/:token', (final req) async {
final viewer = member(req);
if (viewer == null) return status(401);
final body = BytesBuilder(copy: false);
try {
await for (final c in req.read(maxLength: RelayConstants.diagBatchBytes)) {
body.add(c);
}
} catch (_) {
return status(413);
}
final DiagBatch batch;
try {
batch = DiagBatch.decode(body.takeBytes());
} on FormatException catch (e) {
return status(e.message == 'too large' ? 413 : 400, 'bad batch');
}
final now = DateTime.now();
if (rates.length > 500) rates.removeWhere((k, _) => !hub.session.tokens.containsKey(k));
final rate = rates.putIfAbsent(viewer.token, _Rate.new);
if (now.difference(rate.start) >= const Duration(minutes: 1)) {
rate
..start = now
..posts = 0
..bytes = 0;
}
final size = batch.lines.fold<int>(0, (a, l) => a + l.size);
if (rate.posts >= RelayConstants.diagPostsPerMin || rate.bytes + size > RelayConstants.diagBytesPerMin) {
return Response(429,
body: Body.fromString('too many'), headers: Headers.build((h) => h['retry-after'] = ['30']));
}
rate
..posts += 1
..bytes += size;
final log = sessionLog();
if (log == null) return Response.noContent();
log.takeBatch(viewer.token, hub.who(viewer.token), viewer.name, batch);
return Response.noContent();
});
app.get('/l/:sid/:token', (final req) {
if (member(req) == null) return status(401);
final log = sessionLog();
if (log == null) return status(404, 'this server keeps no session log');
final hours = (int.tryParse(req.url.queryParameters['hours'] ?? '') ?? 6).clamp(1, 24 * 7);
final text = log.recent(Duration(hours: hours));
return Response.ok(
body: Body.fromData(Uint8List.fromList(gzip.encode(utf8.encode(text))),
mimeType: MimeType('application', 'gzip')),
headers: Headers.build((h) => h['cache-control'] = ['no-store']),
);
});
}
+22 -1
View File
@@ -11,6 +11,7 @@ import 'media_endpoint.dart';
import 'relay_routes.dart';
import 'server_hub.dart';
import 'session.dart';
import 'session_log.dart';
/// [configured], else systemd's `$STATE_DIRECTORY/yt-dlp`, else `yt-dlp`
/// next to the server binary when that is writable (not under `dart run`),
@@ -56,8 +57,16 @@ class SyncServer {
/// IPv4 interface.
final bool loopbackOnly;
/// The folder of the session log (`session-YYYY-MM-DD.log`, every
/// member's diagnostics and the server's records merged); null: none.
final String? logDir;
SessionLog? _sessionLog;
/// The session log while running with a [logDir].
SessionLog? get sessionLog => _sessionLog;
/// [ytDlp] replaces the server's own (tests).
SyncServer(this.config, {YtDlp? ytDlp, this.loopbackOnly = false}) {
SyncServer(this.config, {YtDlp? ytDlp, this.loopbackOnly = false, this.logDir}) {
hub = ServerHub(sessions, config.secret,
hostKey: config.hostKey,
jellyfin: config.jellyfin,
@@ -103,6 +112,7 @@ class SyncServer {
mountRelayRoutes(_app, sessions, relayAt: hub.relayAt, sourceOf: hub.sourceOf, onLog: logRequest);
mountAvatarRoute(_app, hub);
mountArtRoute(_app, hub);
mountSessionLogRoutes(_app, hub, () => _sessionLog);
mountJellyfinRelayRoute(
_app, sessions, () => hub.jellyfin, () => _relayClient ??= HttpClient(),
next: () => hub.nextJellyfinRef, onLog: logRequest);
@@ -139,6 +149,14 @@ class SyncServer {
if (config.secret.isEmpty) {
_log.warning('no password set: open server, anyone who can reach it can join');
}
final dir = logDir;
if (dir != null && _sessionLog == null) {
try {
_sessionLog = SessionLog(dir)..captureOwn();
} catch (e) {
_log.warning('no session log: cannot write $dir ($e)');
}
}
await _resolveJellyfinServerId();
_mountRoutes();
_server = await _app.serve(
@@ -158,6 +176,7 @@ class SyncServer {
hub.sampleRates();
hub.broadcastState();
hub.broadcastFeed();
_sessionLog?.flush();
});
_log.info('server "${config.name}" listening on $boundPort');
}
@@ -193,5 +212,7 @@ class SyncServer {
await _app.close();
_server = null;
_log.info('server "${config.name}" stopped');
await _sessionLog?.close();
_sessionLog = null;
}
}
+130
View File
@@ -0,0 +1,130 @@
import 'dart:async';
import 'dart:convert';
import 'dart:io';
import 'package:logging/logging.dart';
import 'package:sync_relay_core/core.dart';
import 'package:sync_relay_server/server.dart';
import 'package:test/test.dart';
import 'package:web_socket_channel/web_socket_channel.dart';
void main() {
late Directory dir;
setUp(() async => dir = await Directory.systemTemp.createTemp('session_log'));
tearDown(() async => dir.delete(recursive: true));
String read() => File('${dir.path}/session-2026-10-12.log').readAsStringSync();
test('merges lines of every side in server-clock order after the window', () async {
var now = DateTime(2026, 10, 12, 18, 0, 30);
final log = SessionLog(dir.path, window: const Duration(seconds: 15), now: () => now);
int at(int s, [int ms = 0]) => DateTime(2026, 10, 12, 18, 0, s, ms).millisecondsSinceEpoch;
log.add(at(12), 'Bob (m2)', 'INFO', 'relay', 'second');
log.add(at(10, 5), 'server', 'WARNING', 'server_hub', 'first\n error: boom');
log.add(at(20), 'Alice (m1)', 'INFO', 'mpv', 'too new');
log.add(at(14), 'Alice (m1)', 'INFO', 'media', 'open http://h:1/r/default/50989dda73c45a4710751f8aff4a1982/2/0/x?sig=abc');
log.mark(at(13), 'Alice (m1)', 'Alice', 'stutter\nhere');
log.flush();
expect(read().trimRight().split('\n'), [
'18:00:10.005 [server] WARNING server_hub: first',
' error: boom',
'18:00:12.000 [Bob (m2)] relay: second',
'18:00:13.000 [Alice (m1)] === PROBLEM marked by Alice: stutter here ===',
'18:00:14.000 [Alice (m1)] media: open http://h:1/r/default/***/2/0/x?sig=…',
]);
expect(log.recent(const Duration(hours: 1)), endsWith('[Alice (m1)] mpv: too new'), reason: 'waiting lines too');
now = DateTime(2026, 10, 12, 18, 1);
await log.close();
expect(read(), contains('18:00:20.000 [Alice (m1)] mpv: too new'));
});
test('takeBatch: notes the version once, opt-out once, dropped lines', () async {
final now = DateTime(2026, 10, 12, 18);
final log = SessionLog(dir.path, now: () => now);
final t = now.millisecondsSinceEpoch;
log.takeBatch('tok', 'Bob (m2)', 'Bob',
DiagBatch(version: 'WT 1.0', lines: [DiagLine(t: t, level: 'INFO', logger: 'app', message: 'hi')], dropped: 3));
log.takeBatch('tok', 'Bob (m2)', 'Bob', DiagBatch(version: 'WT 1.0', lines: [DiagLine(t: t, level: 'INFO', logger: 'app', message: 'again')]));
log.takeBatch('tok2', 'Cleo (m3)', 'Cleo', const DiagBatch(version: 'WT 1.0', share: false, lines: [DiagLine(t: 0, level: 'INFO', logger: 'app', message: 'never')]));
log.takeBatch('tok2', 'Cleo (m3)', 'Cleo', const DiagBatch(version: 'WT 1.0', share: false));
await log.close();
final text = read();
expect('shares diagnostics'.allMatches(text).length, 1);
expect(text, contains('[server] diag: Bob (m2) shares diagnostics (WT 1.0)'));
expect(text, contains("[server] diag: Cleo (m3) doesn't share diagnostics (WT 1.0)"));
expect("doesn't share".allMatches(text).length, 1);
expect(text, contains('[Bob (m2)] WARNING diag: 3 lines dropped'));
expect(text, isNot(contains('never')));
});
group('live', () {
late SyncServer server;
setUp(() async {
server = SyncServer(const AppConfig(port: 0, secret: 's', name: 't'), logDir: dir.path);
await server.start();
});
tearDown(() async => server.stop());
Future<(WebSocketChannel, String)> join(String name) async {
final ch = WebSocketChannel.connect(Uri.parse('ws://127.0.0.1:${server.boundPort}/sync'));
final msgs = StreamIterator(ch.stream);
ch.sink.add(jsonEncode(HelloMessage(
proto: RelayConstants.protoVersion,
secret: 's',
user: HelloUser(name: name),
media: const HelloMedia(name: '', size: 0, duration: 0))
.toJson()));
while (await msgs.moveNext()) {
final m = jsonDecode('${msgs.current}') as Map;
if (m['t'] == 'welcome') {
unawaited(() async {
while (await msgs.moveNext()) {}
}());
return (ch, m['token'] as String);
}
}
throw StateError('no welcome');
}
DiagUploader uploader(String token, {String version = 'WT 1.0'}) => DiagUploader(
target: () => (base: Uri.parse('http://127.0.0.1:${server.boundPort}'), token: token, offsetMs: 0),
version: version);
test('members send lines and marks; chat never lands; any member reads it back', () async {
final (alice, aliceTok) = await join('Alice');
final (bob, bobTok) = await join('Bob');
final a = uploader(aliceTok)..start();
final b = uploader(bobTok)
..share = false
..start();
Logger('relay').info('alice line password=hunter2');
alice.sink.add(jsonEncode(const ChatMessage(text: 'secret chat words').toJson()));
await Future<void>.delayed(const Duration(milliseconds: 200));
expect(await a.flush(), isTrue);
expect(await b.flush(), isTrue);
expect(await a.markProblem('audio drifts'), isTrue);
final text = await a.fetchSessionLog(hours: 1);
expect(text, isNotNull);
expect(text, contains('[Alice (m1)] relay: alice line password=***'));
expect(text, contains('=== PROBLEM marked by Alice: audio drifts ==='));
expect(text, contains("Bob (m2) doesn't share diagnostics"));
expect(text, contains('[server] server_hub: join Alice'));
expect(text, isNot(contains('secret chat words')));
// Bob reads it too (friends); a stranger does not.
expect(await b.fetchSessionLog(), contains('PROBLEM'));
expect(await uploader('0' * 32).fetchSessionLog(), isNull);
final bad = await HttpClient().postUrl(Uri.parse('http://127.0.0.1:${server.boundPort}/l/default/$aliceTok'));
bad.add(utf8.encode('not gzip'));
expect((await bad.close()).statusCode, 400);
await a.stop();
await b.stop();
await alice.sink.close();
await bob.sink.close();
await server.stop();
final files = dir.listSync().map((e) => e.uri.pathSegments.last).where((n) => n.startsWith('session-'));
final all = [for (final n in files) File('${dir.path}/$n').readAsStringSync()].join();
expect(all, contains('PROBLEM'));
expect(all, isNot(contains('secret chat words')));
});
});
}
+2 -2
View File
@@ -64,8 +64,8 @@ void main() {
});
group('core welcome + proto', () {
test('proto 14: avatars by hash, profile intent', () {
expect(RelayConstants.protoVersion, 14);
test('proto 15: avatars by hash, profile intent', () {
expect(RelayConstants.protoVersion, 15);
const v = MemberView(id: 'm1', name: 'a', ready: false, inSet: false, buffered: 0, stale: false);
expect(v.toJson().containsKey('avatarHash'), isFalse);
expect(v.toJson().containsKey('avatar'), isFalse);