VR-Mess/HANDOFF.md

VR Messs — Agent Handoff

Read AGENTS.md in the repo root first — it is the authoritative project brain. This file orients you; AGENTS.md governs. Verify every claim here against the code when you touch it (code wins over both docs if they disagree).

1. What this project is

VR anime-character scene viewer — Koikatsu-VR-class, but with real physics hands. Not a game: no loops, quests, progression. Explore, interact, touch, pose — that's the product. This repo is the experimentation lab: designs are hypotheses validated with cheap throwaway spikes.

TERMINOLOGY (owner-corrected — use precisely):

• "getting a body" = the physical rig (torso/arms/hands, the interaction body).
• "embodying" = viewing from a character's perspective in an animation (view-follow + avatar). Two separate systems. Never call the rig milestone "embodying".

2. Toolchain & environment

• Godot 4.7.2 mono (binary godot-mono), C# / .NET 8 (dotnet SDK 10 present). PCVR, desktop-only.
• Physics: Box3D via addons/godot-box3d GDExtension (bearlikelion server-replacement, v0.2.6); setting physics/3d/physics_engine="Box3D Physics". Exactly ONE .gdextension file may exist — a stray duplicate double-loads the library and breaks the server ("required virtual method" errors). The addon is LOCALLY PATCHED (grab v5.3 depends on it): the wrapper now maps Godot's ConeTwistJoint3D → Box3D's b3WeldJoint (rigid grab weld). Originals in addons/godot-box3d/bin-stock/; source + rebuild recipe + patch in .vendor/ (.vendor/box3d-weld.patch, .vendor/godot-box3d/, see research/godot-box3d-weld.md) — only the Linux binary is patched; Windows/macOS binaries are stock (ConeTwist still errors there until rebuilt). Jolt fallback = flip that one project-setting line; all gameplay code stays standard RigidBody3D/StaticBody3D/CharacterBody3D so the flip is lossless.
• Runtime stack: WiVRN + WayVR on Linux. Headset + runtime support hand tracking (treat as available).
• Renderer Forward+ (Mobile is the docs' VR recommendation; deferred until profiling justifies).
• Build: dotnet build. Headless validation — physics: godot-mono --headless --path . scenes/physics_spike.tscn must print "SPIKE: overall PASS"; grab: godot-mono --headless --path . scenes/grab_spike.tscn must print "GRAB: overall PASS". Both PASS as of this handoff. Headless asset import: godot-mono --headless --import.
• Engine logs: ~/.local/share/godot/app_userdata/VR Messs/logs/godot.log — read after user playtests instead of guessing.
• Docs rules (project's #1 failure mode): only the local pinned 4.7 docs at /home/foxyz/Projects/Games/.shared/godot-docs-4.7 (full manual + class ref in .rst; rg it). Web fallback only https://docs.godotengine.org/en/4.7/... — version goes IN the URL, never /latest/. NEVER trust training-memory API names. Known traps: AudioStreamWAV is AudioStreamWav in C#; collision layer/mask live on CollisionObject3D, NOT CollisionShape3D; SkeletonIK3D is deprecated in 4.7. If docs and compiler disagree, grep the shipped /usr/lib/godot-mono/GodotSharp/Api/Debug/GodotSharp.dll.

3. Current systems (scripts/)

• Main.cs — OpenXR startup: finds OpenXRInterface, UseXR, VSync off, VRS XR mode; wires session signals (begun/visible/focussed/stopping/PoseRecentered → re-emitted as C# signals FocusLost/FocusGained/PoseRecentered). On session begin: picks highest refresh rate ≤ MaximumRefreshRate (export, 90) and sets Engine.PhysicsTicksPerSecond to it. Pauses tree on focus lost. No OpenXR → flat preview fallback (PushWarning, scene still runs).
• Locomotion.cs — locomotion manager. Spawn anchoring ONCE on first FocusGained (HMD xz to SpawnPoint + spawn yaw; real head height and pitch kept — PlacePlayer pins arrival height). PoseRecentered is deliberately IGNORED for placement (WiVRN spams it at ~90 Hz; logged once). Teleport: aim + trigger edge, fade to black 0.12 s in / 0.25 s out, near markers within MarkerNearDistance (1.2 m) hide fully (disc/beam/label) and are not targetable. Lasers via UpdatePointer: shown only when aiming at something interactive (marker or panel) AND beyond LaserMinDistance (0.35 m). Also builds the stylus pointer tip per controller. ReturnToSpawn() is public (panel Spawn knob calls it).
• AnchorMoveMode.cs — grab-world movement, ANCHOR semantics. One-hand grip moves the player OPPOSITE to hand motion (full 3D); both grips = anchor translation + in-place yaw turn around the head. Grip is suppressed while that hand prop-grabs (PropGrabber.IsGrabbing) or the panel is near (WorldPanel.IsBlocking). All deltas and turn pivots in WORLD space (tracking-space math made turns orbit the map center — do not regress). Drives XROrigin3D directly.
• PhysicsHands.cs — physical palm bodies chasing the controller poses: 0.4 kg boxes (flat slab, thumb-side face), gravity off, CCD, layer 1/mask 1. Chase = desired velocity (target-pos)/dt force-clamped to HandMaxForce 90 N; rotation hard-locked by setting angular velocity to the controller's rotation error (clamped to 120 rad/s). Safety: stuck (> 0.2 m for 1 s) → snap + log; > SnapDistance 2 m → silent teleport snap — BOTH SUSPENDED for a hand while PropGrabber.IsGrabbing(controller) (a joint legitimately holds the hand back; teleporting a jointed hand yanks the prop — the grabber's grip break runs first instead, rip at 0.8 m). Hand ghosts: translucent palm boxes shown at the controller pose when the physical hand lags

  0.1 m — ghosts belong to the HANDS, not props. Static GetBody(controller) lookup used by grabber and rig.

• PropGrabber.cs — grab v5.3: SOLVER-OWNED RIGID WELD via a LOCALLY PATCHED addon (needs research/godot-box3d-weld.md + the v5.2 history in research/vr-two-handed-grab.md — the pin joint's hidden spring targets IDENTITY and made every earlier iteration a "winch": point constraint + rotator, which the owner rejected as not-how-grab-works: ball orbiting the palm, anvil surface drifting under the palm). Grip edge grabs nearest prop in group "prop" from palm point (GrabRadius 0.16 m, fallback hand center CenterGrabRadius 0.12 m) — palm-proximity only. On grab: creates a ConeTwistJoint3D (mapped by the patch to Box3D's b3WeldJoint) parented under PropGrabber, NodeA = physical hand body, NodeB = prop, SwingSpan = 0, TwistSpan = 0, Position = palm point, ExcludeNodesFromCollision = true. The weld LOCKS THE RELATIVE POSE AT CREATION (the patch pre-corrects frame B with both bodies' rotations — no identity-target snap). Configure Position + NodeA + NodeB + spans + bias BEFORE AddChild — mandatory ordering (configuring after raced Box3D and produced dead joints). NO script forces/torques/damp anywhere — v5.2's whole servo (PD, gravity joints) — v5.2's whole servo (PD, gravity feed-forward, MaxHeldSpin, AngularDamp raise) is DELETED; the solver owns everything and weight flows through the weld into the hand's 90 N budget honestly. NO per-tick joint re-positioning — the weld is not a hand-following pivot (moving the node after creation does nothing). BOTH hands weld RIGID (bias 0) — v5.3.1 (the owner rejected the soft secondary: it gave ~10-15 cm, "feels like a joint again"; hand₁+prop+hand₂ rigidly welded is one composite body, and incompatible controller demands resolve through the HANDS lagging, not the object stretching — spike-measured anchors 2–3 mm under full opposite tug). If a future jitter appears under wrist-conflict, reintroducing a soft secondary bias is the fallback. NO promotion machinery — with both welds rigid from grab, takeover is automatic (the survivor's weld simply persists). Grip break: prop-local anchor vs live palm point > GripRipDistance (0.8 m) instant rip;

  BreakDistance (0.25 m) sustained > BreakTime (0.25 s); both hands > BreakDistanceBoth (0.4 m) breaks two-handed (no promotion on simultaneous break); with a rigid weld these rarely fire — they are the stretch/anomaly safety net; when a two-handed hold releases one hand, the composite rings briefly (~3° peak, decays < 1 s — stored weld stress discharging). NEW escape hatch: hand body lagging its controller beyond PhysicsHands.SnapDistance (2 m) → "grip dragged off" instant break (heavy props can drag the hand since PhysicsHands' snap is suspended while gripping).

• PropShop.cs — builds table (StaticBody3D) + spawns prop table from Specs[] (ping pong 2.7 g → anvil 50 kg), Label3D captions, CCD for < 0.1 kg, group "prop". Respawn() = Reset knob.
• WorldPanel.cs — prop-UI panel, PARENTED UNDER XROrigin3D (fixed in player's room; revert = reparent to Main + fix node paths). Summoned/dismissed via ax_button (A/X; menu_button belongs to the runtime). Knobs: Spawn (return to spawn + recall panel in front of head), Ping, Reset (PropShop.Respawn), Close. Panel/knob bodies: CollisionLayer 2, CollisionMask 0 (props pass through); all CollisionShapes disabled while hidden (SetShapesEnabled, deferred). Pointer tip is a 12 cm stylus cone; knobs hover-highlight and PRESS ON CONTACT (tip within 4.5 cm, 400 ms cooldown) — physical buttons; trigger still presses laser-hovered knobs. Click = code-generated AudioStreamWav PCM (no asset). Draggable direct or via laser at fixed ray distance, re-orients toward head while gripped. IsBlocking(controller) (within 0.45 m) makes Locomotion defer grab/teleport; IsPanelCollider(node) identifies interactive colliders for lasers.
• TeleportMarker.cs — scenes/teleport_marker.tscn: disc + beam + Label3D, group "teleport_marker", StaticBody pick box. SetNear hides visuals, SetHighlighted swaps colors.
• BodyRig.cs — placeholder cube "getting a body" v1.1 (all primitives boxes per owner). Head 0.15³ pushed 6 cm behind the eyes (HeadBackOffset); torso 0.22×0.5×0.13, dropped 0.42 and retracted 6 cm; yaw basis from HMD (flattened forward). Proportions calibrated at runtime from head height (once, after height > 0.5 m): armTotal = 0.46×headHeight clamped [0.5, 0.72], 50/50 upper/fore split, shoulder half-width 0.16 + 0.02×height. Arms: wrist-anchored IK target (palm base = hand body +Z·WristOffset 0.06, not palm center) with SOFT IK — shoulder→hand distance clamped to reach×SoftIKFactor (0.98) BEFORE the law-of-cosines solve, so the arm is never perfectly straight (if near-extension shake persists, lower SoftIKFactor toward 0.95). Elbow pole = down + back + slight outward, projected OUT of the shoulder→hand axis, minus hand forward × PoleHandWeight (0.25), exp-smoothed at PoleSmoothRate (10/s). Torso yaw SPRING, decoupled from the head: HMD yaw base blended toward hands-derived yaw by forwardness × TorsoYawHandsWeight (forwardness = horizontal dot / TorsoYawForwardness 0.3 m, clamped 0–1), exp smoothing TorsoYawSmoothRate (12/s ≈ 200 ms response). Torso reach budget WITH HYSTERESIS: push only past full reach (0.8 m/s), decay only below 92% of reach (0.4 m/s, MoveToward zero); offset clamped horizontal ≤ 0.25 m, vertical [−0.05, +0.12]. Hidden until calibrated.
• GrabSpike.cs + scenes/grab_spike.tscn — headless grab regression (v5.3): a hand stand-in (0.4 kg body, HandMaxForce 90 N chase) grabs a 1 kg prop via ConeTwist weld (server type verified, configure-before-AddChild); drag 2 m + 360° fast-spin sub-phase — asserts RIGID relative pose (grab-rotation error < 2° drag / < 3.5° spin, anchor separation < 1 cm — the winch/orbit regression; 2° is the gravity-on floor for an off-center grab, don't tighten below that); two-handed phase (second weld soft, bias 0.3): primary anchor < 1 cm rigid under full tug, tug give bounded < 0.25 m (recovers), follow-regime anchor < 0.05 m; then primary releases, survivor weld recreated rigid (takeover) and rigidity re-asserted. Reach-timeout guards; must pass 3× in a row (never accept flaky).
• WeldSpike.cs + scenes/weld_spike.tscn — regression for the PATCHED ADDON itself: two bodies welded via ConeTwistJoint3D (bias 0 rigid, then bias 0.5 soft); asserts relative-pose rigidity under torque/force (rot < 2°, sep < 1 cm), bounded excursion + full recovery for the soft weld. Run this FIRST whenever the addon binaries change.

Research (research/): seven full sourced reports — vr-arm-ik.md, vr-arm-ik-deep.md, vr-grabbing.md, vr-two-handed-grab.md (grab v5.0→5.2 history + the verified Box3D pin joint internals + dated CORRECTION section), godot-box3d-weld.md (the addon patch: weld mapping, param table, build recipe, risks), vr-body-physics-hugs.md (player-body physics + hug prior art: Boneworks' one-powered-body lesson, active-ragdoll recipes, hug = wrap + mutual force + reactivity; phased design), vr-body-physics-godot.md (verified Box3D capabilities: no Generic6DOF, hinge VELOCITY motors work, PD-follow proven by the hands). READ these before re-researching those topics or changing the grab/IK/body implementations — they carry the reasoning behind the current designs. Stink-O's box3d-godot (custom components, 8 joint types) remains the documented plan-B if the patched-wrapper path is ever abandoned (owner pointed at it; adopting it means leaving the standard RigidBody3D stack).

Scene wiring (scenes/main.tscn): Main → WorldEnvironment, Sun, Floor, XROrigin3D (XRCamera3D, LeftHand, RightHand, WorldPanel), SpawnPoint, 3 TeleportMarkers, Locomotion, AnchorMoveMode, PropShop, PropGrabber, PhysicsHands, BodyRig, FadeLayer/Fade. Action map: openxr_action_map.tres.

4. Architecture decisions that must not regress

• World-space grab/turn math: all grab deltas and two-hand turn pivots computed in WORLD space. Tracking-space math made two-hand turns orbit the map center. Do not regress.
• Box3D PinJoint3D quirks (grab history v5.0–v5.2): Box3D's PinJoint3D is a TRUE ball joint with ANCHORS CAPTURED AT CREATION (not vanilla Godot origin-pinning). The joint node's position is READ EVERY TICK (re-setting GlobalPosition each tick = hand-following pivot). Its hidden rotational spring (hertz = bias × 30) targets IDENTITY — aligns the two bodies' world orientations, uncapped, hits both bodies; that was the root of the v5.0–v5.2 grab misbehavior (see the CORRECTION section of research/vr-two-handed-grab.md). The positional (point-to-point) tuning (engine default 60 Hz/ζ2.0 clamped to 0.25×tick) is NOT reachable from the wrapper. IMPULSE_CLAMP is a warned no-op.
• Box3D WeldJoint via the local patch (grab v5.3 lives on it): the patch maps Godot's ConeTwistJoint3D → b3WeldJoint, PRE-CORRECTING frame B with both bodies' rotations so the weld locks the relative pose at creation. Bias semantics are INVERTED vs the pin joint: bias 0 = MAX stiffness (rigid), bias 0.3 = 9 Hz give. Swing/twist spans > 0 and softness/relaxation are warned no-ops (the weld has no limits). The weld is NOT a hand-following pivot — moving the joint node after creation does nothing, and none is needed. Only the Linux binary is patched (bin-stock/ holds originals; .vendor/ the source, patch, and rebuild recipe). Pin/hinge/slider paths untouched — grab v5.2.1 remains runnable as fallback (git history). Stink-O's box3d-godot (custom components, more joint types) is the plan-B.
• Collision layers: 1 = world, 2 = UI. Panel/knob bodies are layer 2 mask 0 (props pass through the menu); hands layer 1. Layers/masks live on CollisionObject3D, never CollisionShape3D.
• Ghosts belong to hands, not props — the whole arm gets a ghost later.
• Standard-RigidBody3D-only for scene objects and tracked bodies, so the Jolt fallback stays lossless.
• pose_recentered is never used for placement — WiVRN fires it continuously (~90 Hz); reacting to it was the original view-locked-forward bug.
• Panel parented under XROrigin3D (deliberate feel); grip thresholds polled as Floats (0.6), bool actions unreliable across runtimes.

5. Known issues / open questions

• Grab v5.3.1 — VALIDATED in-headset (owner: "it just works") — the grab saga is closed. Solver-owned rigid weld via the locally patched addon (ConeTwistJoint3D → b3WeldJoint), rigid two-handed, takeover automatic. Headless: grab/weld/physics spikes all PASS 3×. Known honest physics, not bugs: ~1.2° elastic sag on off-center grabs under gravity (lever-dependent, mass-independent), brief ~3° ring when one hand of a two-handed hold releases, and a heavy prop genuinely strains/drags the 0.4 kg hand (that IS the weight feel). Angular hold strength on heavy props one-handed = Box3D's solver stiffness floor (0.25×tick clamp ≈ 22.5 Hz at 90 Hz) — the documented next lever is raising Engine.PhysicsTicksPerSecond above the refresh rate (global experiment, only with owner go-ahead).
• Hand reaction / weight-through-hand — restored for real in v5.3: the weld transmits prop weight/forces through the solver into the 0.4 kg hand's 90 N chase budget. Validate the heft in-headset.
• Palm offset signs (left +, right −) are taste choices — flip if grabs feel mirrored.
• Near-extension arm shake should be fixed by soft IK (SoftIKFactor 0.98) — unvalidated in-headset; lower toward 0.95 if shake persists.
• One-frame arm jumps on hand-body snap/teleport (stuck snap, SnapDistance, teleport).
• IK research applied except anything marked open in research/vr-arm-ik*.md.
• Hand tracking spike not started (XRHandTracker through WiVRN, see openxr_hand_tracking.rst).
• Head cube front face can peek at extreme down-angles (bump HeadBackOffset if it bothers).
• Real character model: import pipeline owned elsewhere (consumes Skeleton3D + AnimationPlayer). Migration path for IK: SkeletonIK3D deprecated in 4.7 → custom SkeletonModifier3D (_process_modification_with_delta) with our own analytic 2-bone solve (no built-in supports an elbow pole) + LookAtModifier3D for head→neck look distribution (head authoritative, neck cosmetic, stacked above head's). HL:Alyx shipped hands-only deliberately (arms mislead without proprioception) — keep arms cheap and honest.

6. Tuning knobs

Constant	File	Value	Controls
HandMaxForce	PhysicsHands.cs	90 N	hand chase force; also used by GrabSpike stand-in
HandMass	PhysicsHands.cs	0.4 kg	hand body mass
HandMaxSpin	PhysicsHands.cs	120 rad/s	hand rotation-lock clamp
StuckDistance / StuckTime	PhysicsHands.cs	0.2 m / 1.0 s	stuck snap (logged) — SUSPENDED while that hand grips
SnapDistance	PhysicsHands.cs	2 m	silent teleport snap of hand body — SUSPENDED while that hand grips
GhostLagDistance	PhysicsHands.cs	0.1 m	when the hand ghost shows
RigidWeldBias	PropGrabber.cs	0	weld bias for BOTH hands (v5.3.1): 0 = MAX stiffness (rigid) — INVERTED vs pin-joint bias semantics; raise only as a fallback if two-handed wrist-conflict jitters
GripRipDistance	PropGrabber.cs	0.8 m	instant release, no timer (checked before the break timer)
BreakDistance / BreakTime / BreakDistanceBoth	PropGrabber.cs	0.25 m / 0.25 s / 0.4 m	grip break: sustained anchor separation (single hand) / both-hands threshold
GrabRadius / CenterGrabRadius	PropGrabber.cs	0.16 m / 0.12 m	palm-point / hand-center grab range
PalmGrabOffset	PropGrabber.cs	±0.045 m along hand +X (left +, right −)	grab point on the palm (taste choice)
Grip/trigger thresholds	several files	0.6	polled Float edges
SoftIKFactor	BodyRig.cs	0.98	shoulder→hand distance clamp before IK solve (lower toward 0.95 if shake persists)
PoleHandWeight / PoleSmoothRate	BodyRig.cs	0.25 / 10	elbow pole: hand-forward push / exp smoothing
TorsoYawHandsWeight / TorsoYawForwardness / TorsoYawSmoothRate	BodyRig.cs	0.6 / 0.3 m / 12	torso yaw spring: hands blend / forwardness normalizer / exp smoothing (≈200 ms)
armTotal	BodyRig.cs	clamp(0.46×headHeight, 0.5, 0.72)	total arm length, 50/50 split
ReachPushSpeed / ReachDecaySpeed	BodyRig.cs	0.8 / 0.4 m/s	torso lean toward / back from overreach
ReachDecayFactor	BodyRig.cs	0.92	hysteresis band (decay only below 92% of reach)
ReachBudgetMax	BodyRig.cs	0.25 m	horizontal torso offset clamp
ReachVerticalMin/Max	BodyRig.cs	−0.05 / +0.12 m	vertical torso offset clamp (stops upward oscillation)
HeadBackOffset / TorsoBackOffset / TorsoDrop	BodyRig.cs	0.06 / 0.06 / 0.42 m	keeps boxes out of the view
LaserMinDistance	Locomotion.cs	0.35 m	laser hidden nearer than this (tip touch instead)
MarkerNearDistance	Locomotion.cs	1.2 m	near markers hide + become untargetable
Fade times	Locomotion.cs	0.12 s in / 0.25 s out	teleport fade
PressContactDistance / PressCooldownMs	WorldPanel.cs	0.045 m / 400 ms	knob press on contact
DirectGrabRadius / TipHoverRadius / TipLength	WorldPanel.cs	0.45 / 0.07 / 0.12 m	panel grab / tip hover / stylus reach
MaximumRefreshRate	Main.cs	90	physics tick = chosen refresh rate

7. Next steps (in order)

1. Full physics body track — DECISION PENDING (owner holds implementation) — the owner wants to pursue a full physics VR body (also as reusable tech for a second project idea); the hug should be mostly physics with detection for future interactivity. Research done: research/vr-body-physics-hugs.md (prior art + phased design; §6 = Nimso's "Hybrid Presence", the closest known implementation, closed-source, only videos) and research/vr-body-physics-godot.md (verified stack facts: no Generic6DOF, hinge velocity motors work, PD-follow proven by the hands). Owner offered video-extraction of Nimso's deep-dive videos — pending. Recommendation shape: staged torso → 2-segment arm chains (chase-driven, Boneworks/Nimso tier, NOT per-bone skeleton sim), test hugging props. Character model: keep boxes for the physics work (owner asked; see research §5-6).
2. Grab points on special objects — grab is palm-proximity to the prop origin today; special objects may want authored grab anchors (see research/vr-grabbing.md taxonomy).
3. Grab points on special objects — grab is palm-proximity to the prop origin today; special objects may want authored grab anchors (see research/vr-grabbing.md taxonomy).
4. Physical torso/head capsules — folded into the body-physics track (item 1): the torso chase body IS the capsule replacement; head/body colliders join it in the same stage.
5. Hand tracking spike — XRHandTracker through WiVRN; controller tip and fingertip must feed the same cursor concept (interaction must not assume controllers).
6. Neck/head look system — LookAtModifier3D head→neck distribution (needs a real skeleton).
7. EMBODYING — view-follow + avatar remapping (chest calibration transform); experiments only after head/neck look override exists. Theory until proven.
8. UI milestone — deliberate full action-map pass + SubViewport/composition-layer decision for dense screens (at most ONE such screen; everything else stays props).

8. Working agreements

• Read AGENTS.md fully before working; it overrides this file where they differ.
• Agent-orchestration workflow: the orchestrator delegates to subagents and reviews their output.
• The USER playtests in VR and reports feel — nothing "works" until the owner confirms in-headset.
• Engine logs (~/.local/share/godot/app_userdata/VR Messs/logs/godot.log) are the diagnosis source after playtests — read them instead of guessing.
• Spikes are cheap and throwaway; a failed experiment is progress, not rework.
• No GDScript-only addon dependencies (Godot XR Tools is reference-only); C# everywhere.