Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat-fix(server_openvr): Add timewarped compositing to Windows OpenVR compositor #2472

Merged
merged 6 commits into from
Oct 31, 2024

Conversation

shinyquagsire23
Copy link
Contributor

Fixes juddery menus on Riven, and possibly judder on other OpenXR apps (HUDs, 2D menus, text, etc rendered at higher resolutions are the most likely candidates).

Before vs After

tl;dr on compositing

  • To composite with timewarp, we need three things: The exact view poses the layer frame was rendered at, the headset's per-eye view frustum tangents, and a renderer which can render a quad with positions/sizes in meters.
    • FrameRender is now passed the pose of each layer, the view tangents, and the eye transforms to make this happen.
  • The layer's quad is centered at 0,0 and extended 1 unit out in each direction, and then each direction is multiplied by the view tangents.
    • Each layer's quad is projected forward by 700m (to "infinity") and scaled proportionally, to remove small positional differences from IPD and the user's head rotation.
    • If the layer is at the exact current view position (which is always the case for layer 0), it will be projected pixel-perfect onto the framebuffer.
    • Layers indices >=1 only ever seem to appear in OpenXR-based titles, I think OpenVR started compositing its app layers into layer 0 on our behalf within the last 6 months (the dashboard used to be juddery as well).
  • For simplicity, and to make debugging a bit easier, I did all the vertex math on the CPU with the DirectX math functions (it's only 8 vertices and DirectX has SIMD support anyway).
    • I also split the draw command into two calls with different viewports/scissor rects, since the quads could cross between eyes after the transform
  • Also I removed the depth buffer for now, since we were not actually using it. If we ever decide to try and get fancy with depth buffers we can add it back I guess.

Copy link
Member

@zmerp zmerp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry it took a bit, left some comments.

@shinyquagsire23
Copy link
Contributor Author

@zmerp Addressed the nits, I guess I kinda see the logic on m_pD3DRender since it would allow render passes to do things with per-eye transforms and such if they needed to w/o a bunch of arg passing, but it still seems a bit odd to me.

Copy link
Member

@zmerp zmerp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice now. I didn't test but either this code is all correct it or it will crash, edge cases are unlikely. Your testing should suffice.

Copy link
Member

@zmerp zmerp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, my bad. I said the class d3dRender but I meant FrameRender. I thought they were the same class.

@shinyquagsire23 shinyquagsire23 force-pushed the fix-timewarp-compositing branch from 76ae446 to 85c959a Compare October 31, 2024 22:53
@shinyquagsire23
Copy link
Contributor Author

should be set now

@zmerp zmerp merged commit 8aa0f7a into alvr-org:master Oct 31, 2024
8 checks passed
zmerp pushed a commit that referenced this pull request Dec 7, 2024
… compositor (#2472)

* WIP

* cleanup

* done

* Fix linux build

* nits

* take 2
zmerp pushed a commit that referenced this pull request Jan 15, 2025
… compositor (#2472)

* WIP

* cleanup

* done

* Fix linux build

* nits

* take 2
zmerp added a commit that referenced this pull request Jan 15, 2025
* wiki updates for flatpak stuff (#2401)

* wiki updates for flatpak stuff

* explain difference between native and flatpak steam more clearly

* give examples of using hybrid graphics with other fixes

* notes about wlx-overlay-s testing

---------

Co-authored-by: failboat78 <[email protected]>

* refactor: 🚨 Use "crate-type"

* ci: ⬆️ Update platform tools to v32

* refactor(streamer): ♻️ Clarify distinction between bitrate and throughput (#2423)

* chore: ⬆️ Update some dependencies

* fix(client_core): 🐛 Fix potential blank render bug

This bug was already mitigated on #2397 but this is the proper fix. Discovered by @nowak-pl

* Add Quest 3S support

* refactor(client): ♻️ Use VIEW reference space; tweak client API

* feat(client): ✨ Add platform vendor shortcuts

* refactor(client): ♻️ Move decoder out of ClientCoreContext; fix IDR race condition (#2421)

* feat(client_core): ✨ Add `alvr_initialize_logging()` (#2439)

* fix(client_openxr): 🐛 Fix crash on Focus 3 with 0 timestamp (#2440)

* feat(server_openvr) NVENC AV1 support (#2438)

* Initial attempt at AV1, ffplay is happy with the AV1 stream, but the Quest 3 is not.

* Trim out IVF bits for NVENC AV1

* Working OSLog for iOS/macOS

* Not AMD only any more

* One more bounds check

* Fix writing AV1 to files

* Remove logging changes

* No hex

* tracking rewrite(1): client API

* fix(server): 🐛 Fix IDR resend logic (#2403)

Co-authored-by: zarik5 <[email protected]>

* build: ✨ Support aarch64 Windows host

* fix(client_core): 🐛 Fix wrong controller timestamp

Fixes #2445

* Revert "chore: ⬆️ Update some dependencies"

This reverts commit c08e67e.

* fix(server_core): 🐛 Fix tilt mode origin; refactoring

* tracking rewrite (2): server API

* Negotiated encoding settings for OLED-based headsets (#2442)

* fix(server_core): 🐛 Do not crash with Custom controllers profile

* fix(client_openxr): 🐛 Fix blank flashes

* feat(client): ✨ Add Pico 4 Ultra support (#2466)

* fix(client_openxr): 🐛 Fix controllers profile for Pico 4 Ultra (#2470)

* feat: ✨ Use generic loader for Pico >= 4 (#2471)

* Create iptables folder during firewall set up if it doesn't exist. (#2457)

* Publish tracking data through VMC Protocol (#2461)

* VMC Protocol implementation

* VMC Protocol: add configuration parameters

* VMC Protocol: work with id's instead of strings

* fix formating

* Merge VMCSinkConfig into VMCConfig

* Apply bone rotation comply with VMC standard.

* VMC: Add toggle for orientation correction

* VMC: set the correct default port

* VMC: Add hand tracking

* VMC: Update rotations

* VMC: Weird constant collision error

* refactor(server_openvr): ♻️ Refactor around TrackedDevice

* refactor(server_openvr): ♻️ Use TrackedDevice for FakeViveTracker

* fix(client_core): 🐛 Fix Pico 4 detection; add Focus Vision (#2479)

* Fix SteamVR restart loops on AVP (#2484)

* feat-fix(server_openvr): Add timewarped compositing to Windows OpenVR compositor (#2472)

* WIP

* cleanup

* done

* Fix linux build

* nits

* take 2

* refactor(client): ♻️ Move platform to alvr_system_info; rename video decoder (#2488)

* feat: ✨ Reduce session file length (#2480)

* feat(client): ✨ Make lobby use passthrough (#2485)

* feat(server_openvr): Add Quest Pro controller and headset emulation; Use VRLink hand tracking icons for the hand tracker controllers; Misc refactoring to reduce prop.rs linecount (#2489)

* feat: integrate adb port forwarding (#2451)

* fix(server_openvr): Fix CPU/GPU deadlocking at 100% usage before the headset connects (#2495)

* Fix CPU/GPU deadlocking at 100% usage before the headset connects

* Temporary stats manager and nits

* Remove optimize_game_render_latency

* fix(server_openvr): 🐛 Fix prop parse fail; use key name for log

* refactor: 💄 Fix dashboard spacing; refactor alvr_adb

* refactor(client_openxr): ♻️ Remove redundant state and fix potential refactoring pitfall

* docs: 📝 Add Steam store link

* feat(xtask): ✨ Add Pico store build; fix Pico 4e and G3

* fix(adb): fix device serial parsing and use TCP (#2507)

* Fix device serial number parsing

* Force stream protocol to TCP when wired

* feat(server_openvr): ✨ Add option to disable hand skeleton prediction (#2517)

* feat(server_openvr): ✨ Add option to disable hand skeleton prediction

* Add `predict` help string

* feat(adb): add client flavors and autolaunch (#2515)

* Add client flavours

* Add client autolaunch

* Apply requested changes

* Fix wired connection status message

* Fix wired client type help string

* fix(sockets): 🐛 Ignore error 997 (overlapped IO) (#2522)

* refactor(client_core): ♻️ Port FFE to Rust and remove all C++ code (#2523)

* build(xtask): 💚 Pin cargo-about (#2524)

* build(xtask): 💚 Really fix cargo about (#2525)

* feat(client): ✨ Add passthrough alpha support (#2527)

* refactor(launcher): refactor client APK installation from launcher (#2531)

* Refactor client APK installation from launcher

* Apply requested changes

* refactor(client_openxr): ♻️ Do not recreate OpenXR session (#2519)

* refactor(client_openxr): ♻️ Do not recreate OpenXR session

* Address comments

* docs(linux): update wiki (#2512)

* docs(linux): Updates for linux wiki, make sure user knows where wiki for linux is

* docs(linux): finalize wiki update

---------

Co-authored-by: Leonhard Saam <[email protected]>

* docs: fix wiki breaking yet again (#2540)

* docs: temp fix for quest framerate throttling (#2538)

* fix(xtask): 🐛 Fix package commands; update release ci (#2545)

* build(xtask): 🔧 Fix Flatpak script (#2547)

* chore: recreate logos as SVG (#2548)

* chore: recreate logos as SVG

* fix: update image link in readme to new logos

* fix: convert text to paths in SVG logo

* fix: 🐛 Fix package-launcher requiring openvr submodule (#2549)

* fix: 🐛 Fix package-launcher requiring openvr submodule

* Do not delete launch build dir with package-launcher; fix launcher installations dir

* fix(dashboard): 🐛 Reject unsupported servers (#2536)

* feat(client): ✨ Add reprojection support for C API (#2543)

* feat(client): ✨ Add reprojection support for C API

* Address comment

* chore(ci): update deprecated actions (#2554)

* chore(ci): update deprecated actions

* chore(ci): Fancy annotations uwu

* chore(ci): update ci some more

* chore(xtask): Dont run submodule update (#2556)

* chore(xtask): Dont run submodule update

* fix(linux): Fix compilitation error without jack installed

* chore: Add composited background resource (#2561)

* docs(linux): fix for no steamvr dashboard on hybrid graphics (#2511)

* docs(linux): fix for no steamvr dashboard on hybrid graphics

* docs(linux) specify dashboard fix is for nvidia

* docs(linux): note to first close the steam client before opening it with render offload

* chore(cpp): Fix warnings from openvr headers

* chore(xtask): Fully remove windows installer

* chore: Bump rust msrv to 1.82

* chore: Fix clippy lints

* chore(ci): fix cargo msrv and print clippy output

* feat(common): ✨ Add `Pose * DeviceMotion` operator (#2574)

* refactor(client_openxr): ♻️ Refactor around reference spaces (#2576)

* refactor(client_openxr): ♻️ Refactor around reference spaces

* Fix clippy

* fix(client_openxr): 🐛 Fix head angular velocity (#2577)

* chore: 💚 Fix CI build (#2578)

* fix launcher on deck (#2585)

* Made the dashboard and launcher report respective appids to the system (#2595)

* fix(server_openvr): 🐛 Fix tracked toggle for controllers (#2604)

* fix(server_core): 🐛 Don't trigger ALVR gestures if holding controllers (#2606)

* Full range wgpu partial fix (#2597)

* Full range HDR works?

* Fix full range for non-HDR

* Cleanup

* Adjust this note

* Fix CI

* Fix server

* Return fractions

* fix(server_core): 🐛 Fix controllers disabling with lag (#2609)

* fix(server_core): 🐛 Fix controllers disabling with lag

* clippy

* fix(client): 🐛 Fix controller angular velocity for some devices; show velocities in lobby (#2608)

* fix(client): 🐛 Fix controller angular velocity for some devices; show velocities in lobby

* Draw velocities only in debug builds

* Add push constants consts and bound checks

* Add support for Virtual Audio Cable (#2483)

* docs: 📝 Mark Vive Focus Vision as supported (#2613)

* fix(client): 🐛 Fix Pico wrong prediction; refactoring (#2612)

* fix(server_core): 🐛 Fix server locking the session unnecessarily (#2619)

* fix(client_openxr): 🐛 Fix crash on unsupported eye tracking permission on Pico (#2618)

* feat: Make on connect script toggle only + disable remote dash by default (#2621)

* Bump to v20.12.0

* chore: fix prepare-release ci (#2624)

---------

Co-authored-by: failboat78 <[email protected]>
Co-authored-by: failboat78 <[email protected]>
Co-authored-by: Max Thomas <[email protected]>
Co-authored-by: Vixea <[email protected]>
Co-authored-by: Simon Storfors <[email protected]>
Co-authored-by: Grillo <[email protected]>
Co-authored-by: E1int <[email protected]>
Co-authored-by: Meister1593 <[email protected]>
Co-authored-by: Leonhard Saam <[email protected]>
Co-authored-by: Leonhard Saam <[email protected]>
Co-authored-by: Ozzy Helix <[email protected]>
Co-authored-by: Issac Dowling <[email protected]>
Co-authored-by: yobson1 <[email protected]>
Co-authored-by: Kirottu <[email protected]>
Co-authored-by: Sobble Entertainment Inc. <[email protected]>
@zmerp zmerp mentioned this pull request Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants