Fix spatial mode on macOS Tahoe + remove gyro yaw drift#2
Open
alexiscrocilla wants to merge 3 commits into
Open
Fix spatial mode on macOS Tahoe + remove gyro yaw drift#2alexiscrocilla wants to merge 3 commits into
alexiscrocilla wants to merge 3 commits into
Conversation
The borderless NSWindow used for spatial mode output was failing to display
on the XReal Air glasses on macOS 26 Tahoe in two ways:
1. canBecomeKeyWindow defaults to false for borderless windows. On Tahoe
this stops MTKView's display link from delivering active draw
callbacks — the first frame renders then the renderer idles.
2. The `.screenSaver` window level was demoted by the Tahoe window server
so a window at negative Cocoa coordinates (which is where the XReal
Air sits in a typical desk arrangement) gets clamped onto the
built-in display as a thin strip.
Fixes:
* Subclass NSWindow as KeyableBorderlessWindow overriding canBecomeKey
and canBecomeMain → true.
* Raise window level to CGShieldingWindowLevel + 1 so it stays on top of
Mission Control, spaces transitions, and other system overlays.
* Call setFrame(_:display:) after orderFront(_:) to force the window
back onto the XReal screen if Tahoe clamped it during initial mount.
Tested on M4 MacBook Pro 14" 2024 / macOS Tahoe 26.2 (25C56) / XReal Air gen 1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… drift
device_imu_open already streams the per-unit calibration JSON blob from
on-device flash via the GET_CAL_DATA_LENGTH (0x14) / CAL_DATA_GET_NEXT_SEGMENT
(0x15) HID message pair, but the streamed bytes were being discarded into
a 56-byte scratch buffer with the comment "json-c calibration helpers
removed — using device defaults + Madgwick convergence". As a result every
unit fell back to identity calibration, leaving roughly 1°/s of residual
gyro yaw bias — the visible head-locked drift users report.
This commit accumulates the streamed segments into a malloc'd buffer
(capped at 1 MiB for safety), parses the resulting JSON with a small
hand-rolled string-scanner targeting only the ~10 keys we consume, and
populates device->calibration. apply_calibration() already converts
units correctly (rad/s → deg/s for gyro, m/s² → g for accelerometer), so
no math changes downstream.
Files:
* UltraXReal/Vendor/xreal-imu/cal_json.{h,c} — tolerant parser, ~140 LOC,
no new dependencies. Matches sections (acc/gyro/mag) and key names
(bias/offset, scale/sensitivity, misalignment/alignment) heuristically
so it handles XReal's exact JSON layout variations without a schema.
* UltraXReal/Vendor/xreal-imu/device_imu.c — replace the discard loop
with a collect-then-parse-then-apply block. Falls back silently to
identity calibration if parsing fails (never worse than today).
* UltraXReal.xcodeproj — register cal_json.c in the build.
For empirical verification of the on-wire JSON format, set
ULTRAXREAL_DUMP_CAL=/path/to/cal.json before launch — the raw blob is
dumped on success.
Tested on XReal Air gen 1: stationary head produces no visible drift
after the fix (was ~10° in 30 seconds before).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nvas Two UX fixes for spatial mode discoverability: 1. Cursor warp. Spatial mode unmirrors the XReal Air and takes the panel directly with a borderless renderer window, so the OS cursor stays on the built-in display where the user can't see it. Auto-warp it onto the centre of the virtual display when spatial activates (after the recenter delay), and expose ⌘⇧M / "Move Cursor to Virtual Display" so it can be re-invoked if the user accidentally clicks back to the built-in display. 2. Grey clearColor when no real frame yet. SCK delivers status-only sample buffers (`.idle`, `.blank`, `.suspended`) when the captured display content isn't changing — an empty virtual display produces *only* those, so the renderer used to early-return on every callback and the MTKView never painted, leaving solid black on the glasses. Indistinguishable from a broken renderer. Now: read SCStreamFrameInfo status, skip non-`.complete` buffers, and when the renderer hasn't received a real frame yet, render a dark-grey (#1a1a1a) clear pass instead of returning early. User sees grey → knows the renderer is alive and the canvas is empty (drag an app onto it). Also adds an os_log signpost for the first complete frame and for SCK status transitions to aid future bug reports. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three independent fixes for issues reported by users (spatial mode invisible on macOS Tahoe + chronic gyro drift). Split across three commits so each can be reviewed and reverted independently if needed.
1.
fix(spatial): route output window to glasses on macOS TahoeThe borderless
NSWindowused for spatial output was failing to display on the XReal Air on macOS 26 Tahoe in two ways:canBecomeKeyWindowdefaults tofalsefor borderless windows. On Tahoe this stops MTKView's display link from delivering active draw callbacks — the first frame renders then the renderer idles, panel goes black..screenSaverwindow level was demoted by the Tahoe window server, so a window at negative Cocoa coordinates (where the XReal Air sits in a typical desk arrangement) gets clamped onto the built-in display as a thin strip.Fix: subclass
NSWindowasKeyableBorderlessWindow(overridecanBecomeKey/canBecomeMain→true), raise window level toCGShieldingWindowLevel + 1, and callsetFrame(_:display:)afterorderFront(_:)to defeat Tahoe's post-creation clamp.2.
fix(imu): apply factory calibration from glasses flash to remove gyro driftdevice_imu_openalready streams the per-unit calibration JSON blob from on-device flash via theGET_CAL_DATA_LENGTH(0x14) /CAL_DATA_GET_NEXT_SEGMENT(0x15) HID message pair, but the bytes were being discarded with the comment "json-c calibration helpers removed". As a result, every unit fell back to identity calibration, leaving ~1°/s of residual gyro yaw bias — the visible head-locked drift users report.Fix: accumulate the streamed segments into a malloc'd buffer (1 MiB sanity cap), parse with a small hand-rolled string-scanner (~140 LOC, no new dependencies) targeting only the ~10 keys we consume, and populate
device->calibration.apply_calibration()already handles unit conversion (rad/s → deg/s for gyro, m/s² → g for accel), so no math changes downstream. Falls back silently to identity if parsing fails — never worse than before.For empirical verification of the on-wire JSON format, set
ULTRAXREAL_DUMP_CAL=/path/to/cal.jsonbefore launch and the raw blob is dumped on success.3.
feat(spatial): cursor warp on activation + grey fallback for empty canvasTwo UX improvements:
.idle/.blank/.suspended) when the captured content isn't changing. An empty virtual display produces only those, so the renderer used to early-return on every callback and MTKView never painted — solid black on the glasses, indistinguishable from a broken renderer. Now: skip non-.completebuffers, and when no real frame has arrived yet, render a dark-grey (#1a1a1a) clear pass so the user knows the renderer is alive and the canvas is just empty.Testing
Caveats
acc/accelerometer,bias/offset,scale/sensitivity,misalignment/alignment). If XReal ever changes their JSON schema we may need to add variants — theULTRAXREAL_DUMP_CALenv var helps diagnose this.SCStream.startCapture()silently fails with -3801 if the new cdhash doesn't match the cached permission. Happy to add this in a follow-up PR if you prefer.