Initial commit: VR Messs workspace (grab v5.3.1, body-physics research)
This commit is contained in:
Foxyz1302 committed
commit
5d2c5bf265
57 files changed
+4990
No files matched your search
@@ -0,0 +1,236 @@
|
||||
# VR Messs
|
||||
|
||||
VR anime-character "scene viewer" (Koikatsu-VR-class, but with real physics hands).
|
||||
Not a game: no gameplay loops, quests, or progression. Gameplay features are the
|
||||
owner's domain; explore, interact, touch, pose — that's the product.
|
||||
|
||||
TERMINOLOGY (owner-corrected, use precisely): **"getting a body"** = the physical rig
|
||||
(torso/arms/hands — the interaction body). **"embodying"** = literally viewing from the
|
||||
perspective of a character in an animation (the view-follow + avatar system). They are
|
||||
separate systems; never call the rig milestone "embodying".
|
||||
|
||||
This repo is the experimentation lab (hence the name). The designs below are working hypotheses
|
||||
to be validated with cheap, throwaway spikes — a failed experiment is progress, not rework. The
|
||||
embodiment/view system in particular is theory until proven.
|
||||
|
||||
## Toolchain pins (do not drift)
|
||||
|
||||
- **Godot 4.7.2** (current stable, Forward Plus renderer), **C# / .NET 8**. Desktop-only (PCVR).
|
||||
- **Physics: Box3D** via the `godot-box3d` GDExtension (server-replacement package by
|
||||
bearlikelion) — **installed, v0.2.6**, setting value `physics/3d/physics_engine="Box3D
|
||||
Physics"`. Validation spike (`scenes/physics_spike.tscn`, run headless) PASSES: drop+contact
|
||||
and velocity-chase-into-wall both work from C#. WARNING: exactly ONE `.gdextension` file may
|
||||
exist — a stray duplicate (e.g. at `addons/` root) double-loads the library and breaks the
|
||||
server with "required virtual method" errors. Jolt (built-in) is the fallback: reverting is
|
||||
the one project-setting line, and all gameplay code must stay standard-Godot-physics-node
|
||||
based so that flip is lossless.
|
||||
- Character import (Koikatsu etc.) is a separate pipeline owned by someone else. We consume a
|
||||
character scene that provides: `Skeleton3D` with humanoid bone mapping + `AnimationPlayer`.
|
||||
Don't touch import code; if the interface is insufficient, say so instead of hacking around it.
|
||||
- Dev runtime stack: **WiVRN + WayVR** on Linux. Headset and runtime both support hand tracking —
|
||||
treat hand tracking as available. View-follow milestone verified working (OpenXR + WiVRN).
|
||||
- Graphics are deliberately **stylized** — fits the anime look and buys performance headroom.
|
||||
Renderer choice (currently Forward+; docs recommend Mobile for desktop VR) is deferred until
|
||||
profiling justifies a change.
|
||||
|
||||
## Documentation rules (hard rules — this project's #1 failure mode)
|
||||
|
||||
- Local pinned docs: `/home/foxyz/Projects/Games/.shared/godot-docs-4.7` — full manual +
|
||||
class reference in `.rst` (source of docs.godotengine.org, branch `4.7`). Grep it first.
|
||||
- For ANY engine API detail, class name, property, or signal: **trust only these 4.7 docs or a
|
||||
live spike.** Training-memory APIs and older-version knowledge are banned — verify or don't write it.
|
||||
- Web fallback: `https://docs.godotengine.org/en/4.7/...` only — the version goes IN the URL.
|
||||
Never `/latest/`, never another version path.
|
||||
- Code examples: rst sources contain `code-tab::` blocks with BOTH gdscript and csharp — grep for
|
||||
`code-tab:: csharp`. Class reference lives in `classes/class_*.rst`.
|
||||
- XR tutorials: `tutorials/xr/` (setting_up_xr, xr_action_map, xr_room_scale,
|
||||
openxr_body_tracking, basic_xr_locomotion). Physics: `tutorials/physics/` (rigid_body,
|
||||
ragdoll_system, using_jolt_physics, physics_introduction).
|
||||
|
||||
## Godot/C# traps
|
||||
|
||||
- Addons that are GDScript-only can't be referenced from C# — **Godot XR Tools is GDScript-only**:
|
||||
treat it as a reference implementation to port ideas from, never a dependency.
|
||||
- Default OpenXR action map names (verified from engine source, 4.7): `trigger`/`grip` (Float),
|
||||
`ax_button` (A/X), `by_button` (B/Y), `menu_button` (menu/system — often eaten by the runtime,
|
||||
e.g. WiVRN screenshot+anchor-hold), `primary` (thumbstick). `select_button` is Pico-only.
|
||||
Poses: `default_pose`, `aim_pose`, `grip_pose`, `palm_pose`.
|
||||
- C# in Godot covers desktop + mobile only (fine — PCVR target).
|
||||
- Signals: C# `+=`/`-=` events; check the class ref for exact C# signatures.
|
||||
- The C# binding can diverge from engine class names: `AudioStreamWAV` is `AudioStreamWav`
|
||||
in C#. When docs and compiler disagree, grep the shipped `GodotSharp.dll` (strings) —
|
||||
it's at `/usr/lib/godot-mono/GodotSharp/Api/Debug/GodotSharp.dll` on this machine.
|
||||
|
||||
## Architecture (working hypotheses — test cheap before committing)
|
||||
|
||||
Four layers, bottom-up, applied per frame in this order:
|
||||
|
||||
1. **Animation** — `AnimationPlayer` drives the skeleton (any pose, any animation).
|
||||
2. **Tracked override / view embodiment** — several input modes, all blended over whatever
|
||||
animation is playing:
|
||||
- **View-follow (core feature, Koikatsu-VR-style)**: the view follows the character's head
|
||||
frame including tilt, so weird animations stay correct. The player's HMD look is remapped
|
||||
through a one-time calibration transform captured relative to the character's chest
|
||||
("chest calibration", e.g. once while lying flat) and drives a look override on the
|
||||
head+neck (possibly slight chest) bones. Skeleton-space only — the physics layer stays
|
||||
separate from this. Pivot behavior (rotation-only snap vs positional head offset) and the
|
||||
calibration UX are open experiments.
|
||||
- **Free-view**: ordinary playspace (`XROrigin3D`); "lean back mode" = origin offset +
|
||||
recline calibration, zero physics involved.
|
||||
- **Single-controller mode**: one controller as master tracker — hand override, or strapped
|
||||
to the hip driving the animation root/hip (hip-tracker mode).
|
||||
- Full-body trackers are a far-future extension of this same layer (more override targets),
|
||||
not a separate system.
|
||||
3. **Physics embodiment** — a small set of rigid bodies (hand L/R, head, torso/hip capsule) that
|
||||
CHASE layer-1/2 targets: velocity-clamped steering (`v = clamp((target - pos) / dt)`), CCD on,
|
||||
no joints between them. Unobstructed → effectively 1:1 tracking; obstructed → pushes with
|
||||
tunable strength. The chase deviation is the feedback signal: it clamps the visual hand offset,
|
||||
pushes the character pose via spring-damped offsets ("pushable from animation"), and fires
|
||||
logical touch-reaction events. Boneworks calls the general technique "heuristic force limit".
|
||||
4. **Secondary motion (hair/skirt/cloth) — custom spring-bone solver.** Koikatsu DynamicBones and
|
||||
VRM springBone are poorly interactive and/or expensive; VRChat PhysBones are the gold standard
|
||||
but proprietary. So: original PhysBones-*style* math (spring-damper chains on the skeleton,
|
||||
animation-space) with analytic sphere/capsule colliders (body, hands, floor). The physics
|
||||
engine contributes only its QUERY API (throttled ray/shape casts) so bones respect scene
|
||||
geometry — **never rigid-body chains for jiggle**. Performance is a first-class goal of this
|
||||
project: chain budgets, screen-size/distance LOD, simplified colliders. If the per-bone C#
|
||||
loop profiles hot, the hot loop moves to a GDExtension — profile first, don't pre-optimize.
|
||||
|
||||
Do NOT build anything that needs 6DOF/cone-twist joints between tracked bodies (the wrapper lacks
|
||||
them; velocity steering replaces them). Scene objects stay standard nodes (`RigidBody3D`,
|
||||
`StaticBody3D`, `CharacterBody3D`) so arbitrary imported scene content works and the Jolt fallback
|
||||
stays real.
|
||||
|
||||
## XR
|
||||
|
||||
OpenXR is built into Godot 4.7 — no plugin. Node set: `XROrigin3D` / `XRCamera3D` /
|
||||
`XRController3D` (+ `XRBodyTracker`/`XRHandTracker` for body/hand tracking data). OpenXR runs on
|
||||
device only; the editor has an XR simulator for dev. Trackers/action map: see `xr_action_map.rst`.
|
||||
|
||||
## Interaction & UI (design directions — experiments welcome)
|
||||
|
||||
- **Touch-first hybrid cursor**: interactive surfaces prefer *touch*. A pointer tip (controller
|
||||
tip or index fingertip) acts as a cursor when in proximity of a surface — push, hold, slide,
|
||||
physical press feel. Out of proximity, fall back to a laser ray for distant precision. Lasers
|
||||
ruin immersion and are hard to aim at distance; they are the fallback, not the default.
|
||||
- **Hand tracking**: cheap spike early, full support later. Interaction logic must not assume
|
||||
controllers — controller tip and fingertip feed the same cursor concept.
|
||||
- **Locomotion** (implemented, play-feel tuning ongoing): spawn re-anchoring
|
||||
happens ONCE on first focus gained (HMD xz at `SpawnPoint` + spawn yaw; real head height and
|
||||
pitch kept) — physical room position never matters. `pose_recentered` is deliberately NOT a
|
||||
placement trigger AND its log is throttled to one line: measured on WiVRN it fires
|
||||
continuously at refresh rate (~90 Hz, 22k lines/session) — reacting to it was the original
|
||||
view-locked-forward bug. `Locomotion.cs` is the manager (spawn placement,
|
||||
teleport markers, lasers, fade); **grab movement lives in `AnchorMoveMode.cs` as its own
|
||||
movement mode** — scene-viewer-first, other modes (walk, tp-only variants) are future nodes
|
||||
beside it. Teleport markers (`scenes/teleport_marker.tscn`):
|
||||
aim + trigger, with fade; near markers (within `MarkerNearDistance` 1.2 m) fully hide
|
||||
(disc, beam, label) AND are not teleport-targetable. Grab semantics are
|
||||
ANCHOR-based: hands are anchors to push/pull from — one-hand grip
|
||||
moves the player OPPOSITE to hand motion in full 3D, both grips = anchor translation +
|
||||
IN-PLACE yaw turn around the head. All grab deltas and turn pivots are computed in
|
||||
WORLD space (tracking-space math made turns orbit the map center — do not regress).
|
||||
Spawn/teleport placement pins arrival height (head height preserved). Locomotion drives
|
||||
`XROrigin3D` directly; physics reconciliation comes with the embodiment layer. Grip/trigger
|
||||
are polled Floats with thresholds per docs
|
||||
(bool actions unreliable across runtimes). Engine logs land in
|
||||
`~/.local/share/godot/app_userdata/VR Messs/logs/godot.log` — agents: read this after user
|
||||
playtests instead of guessing.
|
||||
- **UI direction — physical prop controls first**: buttons, knobs, sliders as real 3D objects
|
||||
(touch interaction, later Box3D for feel). Godot has NO native world-space widget system —
|
||||
`Control`s are screen-space; world UI is always Control-rendered-into-3D via `SubViewport`
|
||||
(optionally `SubViewportCompositionLayer` for XR-native sharpness/perf). Use at most one
|
||||
SubViewport/composition-layer screen for dense content (file browser, settings) if needed;
|
||||
everything else is props. World-space text is native and cheap: `Label3D` (SDF, crisp at
|
||||
distance) and `TextMesh` (extruded) — no SubViewport required.
|
||||
- **`WorldPanel.cs` is the first prop-UI prototype**: summoned/dismissed via `ax_button` (A/X —
|
||||
menu_button belongs to the runtime). It is PARENTED UNDER `XROrigin3D` — Koikatsu-VR-style
|
||||
"fixed in the player's room": IRL movement doesn't move it relative to you, game locomotion
|
||||
carries it. If that feel is wrong, reverting = reparenting to `Main` and fixing the `_Ready`
|
||||
node paths. Floats in front of the summoning hand facing the player (full 3D facing — works
|
||||
from lying poses), draggable directly or via laser at fixed ray distance while gripped
|
||||
(continuously re-orients toward the head). The Spawn
|
||||
knob also recalls the panel in front of the head. The pointer
|
||||
tip is a protruding stylus cone (apex ~12 cm ahead of the controller) and is THE cursor:
|
||||
knobs hover-highlight and PRESS ON CONTACT (tip within 4.5 cm, with cooldown) — physical
|
||||
buttons, no trigger needed; trigger still presses laser-hovered knobs (knobs are
|
||||
laser-targetable). Presses play a synthesized click (code-generated `AudioStreamWav` PCM
|
||||
— no asset). Spawn/Ping/Close knobs. Locomotion defers
|
||||
grab/teleport to the panel when a hand is near it (`WorldPanel.IsBlocking`). Laser lines are
|
||||
owned by `Locomotion.UpdatePointer`: shown only when aiming at something interactive (marker
|
||||
or panel) AND beyond `LaserMinDistance` (0.35 m) — near targets are touched via the pointer
|
||||
tip instead, which is the touch-first hybrid cursor in miniature. Grab deltas and turn
|
||||
pivots are computed in WORLD space (tracking-space math made two-hand turns orbit the map
|
||||
center — do not regress this).
|
||||
- **Control scheme is deliberate**: decide what lives on controllers vs. what lives in menus
|
||||
before stacking bindings — controller button budget is scarce, menus carry the rest.
|
||||
|
||||
## Order of work
|
||||
|
||||
1. ~~Get VR working at all~~ — **done**: OpenXR + WiVRN verified in-headset, flat preview
|
||||
fallback works without a headset.
|
||||
2. ~~Input foundation (partial)~~ — grip/trigger Float polling + `ax_button` summon verified
|
||||
from engine source; the deliberate full action-map pass happens with the UI milestone.
|
||||
**Hand tracking spike still TODO** (`XRHandTracker` through WiVRN, see
|
||||
`openxr_hand_tracking.rst`).
|
||||
3. ~~Locomotion: anchor-grab mode~~ — **done** (`AnchorMoveMode.cs` + teleport markers +
|
||||
spawn anchoring); other movement modes are future nodes beside it.
|
||||
4. ~~UI: first prop panel~~ — **done** (`WorldPanel.cs`); SubViewport/composition-layer screen
|
||||
deferred until dense content (file browser) is actually needed.
|
||||
5. ~~Box3D validation spike~~ — **done, PASS** (drop+contact, velocity-chase vs wall, headless
|
||||
via `scenes/physics_spike.tscn`).
|
||||
6. **Physics hands & props sandbox — grab v5.3 = SOLVER-OWNED RIGID WELD (patched addon).**
|
||||
Research first (see `research/vr-grabbing.md`, `vr-two-handed-grab.md` + its dated
|
||||
CORRECTION, `godot-box3d-weld.md` — sourced taxonomy, Box3D internals, the whole v5.0→5.2
|
||||
history). The owner rejected every point-joint variant as "hanging on a winch" (ball
|
||||
orbits the palm, anvil surface drifts, palms clip) — a grab must lock the object's POSE
|
||||
relative to the hand, not one point + a rotator. **The addon is now locally patched**
|
||||
(source + recipe in `.vendor/`, originals in `addons/godot-box3d/bin-stock/`, Linux binary
|
||||
only): Godot's ConeTwistJoint3D maps to Box3D's `b3WeldJoint`, pre-corrected so the weld
|
||||
locks the relative pose AT CREATION (no identity-target snap; bias semantics INVERTED vs
|
||||
pin — 0 = max stiffness). `PropGrabber.cs`: grab creates a ConeTwistJoint3D weld at the
|
||||
palm point (`SwingSpan/TwistSpan 0`, `ExcludeNodesFromCollision`, configured BEFORE
|
||||
AddChild — mandatory ordering). NO script forces/torques/damp anywhere and NO per-tick
|
||||
joint re-positioning (the weld is not a hand-following pivot) — the solver owns everything
|
||||
and weight flows into the hand's 90 N budget honestly. BOTH hands weld RIGID (bias 0 —
|
||||
v5.3.1: the owner rejected the soft secondary; hand₁+prop+hand₂ welded = one composite
|
||||
body, conflicting controller demands resolve through hand lag, not object stretch);
|
||||
takeover is automatic — the survivor's weld persists, and the brief ~3° ring when one
|
||||
hand releases is stored weld stress discharging (honest physics). Break: stretch-based (rip 0.8 m / 0.25 m for 0.25 s / both-hands
|
||||
0.4 m) + "grip dragged off" when the hand body lags its controller > `PhysicsHands.SnapDistance`.
|
||||
Headless regressions: `scenes/weld_spike.tscn` (addon patch), `scenes/physics_spike.tscn`,
|
||||
`scenes/grab_spike.tscn` (rigid pose under drag + fast spin — the winch regression; don't
|
||||
tighten rotation limits below ~2°, that's the gravity floor for off-center grabs).
|
||||
`PhysicsHands.cs`: physical palm bodies (0.4 kg, 90 N chase, rotation lock, stuck-safety
|
||||
snap — SUSPENDED for a hand while it grips, HAND ghosts > 10 cm lag). Collision layers:
|
||||
1 = world, 2 = UI (panel mask 0 — props pass through even when shown); layers live on
|
||||
CollisionObject3D, NOT CollisionShape3D. Panel = prop-UI prototype (ax_button summon,
|
||||
contact-press knobs, Reset). Research reports live in `research/` (arm IK, IK
|
||||
stability/deep, grabbing, two-handed grab/Box3D internals, addon weld patch) — READ before
|
||||
re-researching or changing grab/IK. Stink-O's box3d-godot (custom components) = documented
|
||||
plan-B. Laser/distant grab REMOVED (user never knew it existed); grab = palm-proximity only,
|
||||
per-hand palm-face offset (left +X, right −X — taste). Box3D PinJoint quirks: anchors are
|
||||
CAPTURED AT CREATION; the joint node's position is READ EVERY TICK (keep it on the palm =
|
||||
hand-following pivot); break must measure the prop-local anchor vs the palm point. Headless
|
||||
regression tests: `scenes/physics_spike.tscn` AND `scenes/grab_spike.tscn` — both must print
|
||||
overall PASS. `PhysicsHands.cs`: physical palm bodies (0.4 kg, 90 N chase, rotation lock,
|
||||
stuck-safety snap, HAND ghosts > 10 cm lag). Collision layers: 1 = world, 2 = UI (panel mask
|
||||
0 — props pass through even when shown); layers live on CollisionObject3D, NOT
|
||||
CollisionShape3D. Panel = prop-UI prototype (ax_button summon, contact-press knobs, Reset).
|
||||
Research reports live in `research/` (arm IK, IK stability/deep, grabbing,
|
||||
two-handed grab/Box3D joint internals) — READ before re-researching or changing grab/IK.
|
||||
7. **Getting a body (rig) — placeholder v1.2 (cube character, agent-built).** `BodyRig.cs`:
|
||||
all-BOX primitives (owner preference) — head 0.15³ behind the eyes, slim torso retracted,
|
||||
wrist-anchored 2-bone arms. **Soft IK applied**: shoulder→hand distance clamped to
|
||||
reach×0.98 before the solve (the straight-arm singularity was the overextension shake; if it
|
||||
persists, lower SoftIKFactor toward 0.95). Elbow pole: torso frame, shoulder→hand axis
|
||||
PROJECTED OUT, hand weight 0.25, exp-smoothed 10/s (twitchy wrists fixed). **Torso yaw
|
||||
spring**: HMD yaw base blended toward hands-derived yaw (weight 0.6 × forwardness over
|
||||
0.3 m), exp-smoothed 12/s, decoupled from the head — the "chest = headset + hands" ask.
|
||||
Reach budget hysteresis unchanged. Full sourced findings in `research/vr-arm-ik.md` and
|
||||
`research/vr-arm-ik-deep.md` (HL:Alyx went hands-only on purpose; SkeletonIK3D deprecated →
|
||||
custom SkeletonModifier3D when a real skeleton arrives; LookAtModifier3D for head→neck).
|
||||
Physical torso/head capsules later. EMBODYING (view-from-character-perspective) comes after
|
||||
the head/neck look override exists. **HANDOFF.md at project root is the takeover document**
|
||||
(constant table verified against code — code wins on drift); keep it updated when systems
|
||||
change.
|
||||
Reference in new issue
Block a user