From 746c95a4cfbae2de71529b63e46205e6aa6b1ebf Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 11:39:00 +0200 Subject: [PATCH 01/21] =?UTF-8?q?Documented=20finding=20=E2=80=94=20docs/O?= =?UTF-8?q?URA=5FPROTOCOL.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 0x43 debug spill wasn't documented, so I added it in three linked spots: - §2.4 (reassembly) — a dated "byte-alignment desync → phantom tags" note: how a run of 0x43 debug records surfaced as a phantom 0x4C "sleep_summary" because 0x4C=L/0x49=I/0x53=S are legal ASCII chars that alias real tags, plus the defensive resync rule (reject len<4, resynchronise on implausible type/len, never let a mid-stream byte mint a Tier-B summary). - §6.15 (0x43) — expanded with the actual decoded strings ("Sw to App", "in_bed=0", "…in_info=6", "bc 0x43"), the TLV layout, and the aliasing caution. - §6.12 (sleep summaries) — a "beware phantom sightings" warning so nobody chases an ASCII 0x4C as a real fixture. --- docs/OURA_PROTOCOL.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/OURA_PROTOCOL.md b/docs/OURA_PROTOCOL.md index 164b86f43..05d5b0736 100644 --- a/docs/OURA_PROTOCOL.md +++ b/docs/OURA_PROTOCOL.md @@ -93,6 +93,8 @@ Returned during history fetch (`0x10`/`0x11`) and live streaming. Each record: [ ### 2.4 Multi-packet payloads There is no application-level fragmentation header beyond the TLV `len`. A record never spans two notifications in the verified corpus; each notification contains whole frames/records. NOOP's parser must still be defensive: buffer partial trailing bytes across notifications and only emit complete `2+len` records. +> **Byte-alignment desync → phantom tags (live Gen 3, 2026-07-08).** Observed: a run of `0x43` debug_event records (ASCII, §6.15) surfaced in the log as a single **`0x4C` "sleep_summary"** whose "payload" spilled across the following real `0x43` records (the later `43 0c ..`/`43 0b ..` headers were visible inline; the first record's `type` byte was missing). Root cause: the reassembler lost byte-alignment and read an ASCII byte inside a debug string as a `type` byte — `0x4C` is the letter **`L`**, `0x49`=`I`, `0x53`=`S`, all legal debug-text characters that alias real event tags. A desynced parser can therefore *fabricate* a `0x4C`/`0x49` "sleep summary" (or corrupt a genuine record). **Defensive rule:** validate each record before emitting it — reject `len < 4` (a record must cover its 4 timestamp bytes) and, on an implausible `type`/`len`, **resynchronise** (advance one byte and re-scan) rather than emitting the record; never let an unverified `type` byte from a mid-stream position mint a Tier-B summary event. `0x43` debug text should be decoded to ASCII and surfaced, not left to alias a sleep tag. + --- ## 3. Authentication Handshake @@ -354,7 +356,7 @@ bits 14–15 : qual_b - **`0x72` `sleep_acm_period`** (16 B): values0–2 = `whole(8)+frac(8)/255`; values3–5 = `whole(4)+frac(12)/4095`. [ringverse] - **`0x49` `sleep_summary_1`**: start/end as uint16 LE minutes-before-event. [ringverse] - **`0x76` `bedtime_period`**: start/end as uint32 LE ringTimestamps → map to UTC (§5.5). [ringverse] -- Tags `0x48,0x4A–0x4D,0x4F,0x57,0x58` are additional sleep summary/feature variants in the dictionary; layouts **(UNVERIFIED)** - decode only after fixtures. [ringverse] +- Tags `0x48,0x4A–0x4D,0x4F,0x57,0x58` are additional sleep summary/feature variants in the dictionary; layouts **(UNVERIFIED)** - decode only after fixtures. [ringverse] **Beware phantom sightings:** a `0x4C`/`0x49` "sleep_summary" whose payload is ASCII (e.g. contains `in_bed=…`, readable words) is almost certainly misframed `0x43` debug text, not a real summary — see §2.4 desync rule and §6.15. Confirm a candidate summary is binary and length-consistent before treating it as a fixture. ### 6.13 Motion / activity - **`0x47` `motion_events`** (variable): byte6 bits`[7:5]`=field_a, `[4:0]`=field_b; bytes7–9 = three **int8 × 8** axis magnitudes; optional bytes10–11. [ringverse] @@ -371,7 +373,7 @@ bits 14–15 : qual_b ### 6.15 Lifecycle / state - **`0x41` ring_start_ind** (18 B): bytes6–10 = 40-bit device id; bytes15–19 config; triggers anchor invalidation on rt regress. [ringverse][open_ring] -- **`0x43` debug_event**: ASCII text (state strings). [open_ring][open_oura-r3] +- **`0x43` debug_event**: ASCII text (state strings), one string per TLV record (`43 `). Live Gen 3 captures (2026-07-08) show short firmware diagnostics with sequential ring counters, e.g. `"Sw to App"`, `"in_bed=0"`, `"…in_info=6"`, `"bc 0x43"` — useful sleep/history signal (an explicit `in_bed` flag). NOOP decodes these to `OuraEvent.debugText`; surface them in the strap log. **Caution:** because the payload is arbitrary ASCII, a byte-misaligned parser can read a letter (`L`=0x4C, `I`=0x49, `S`=0x53) as a record `type` and mis-emit a phantom sleep-summary — see the desync rule in §2.4. [open_ring][open_oura-r3] - **`0x45` state_change_ind / `0x53` wear_event**: byte6 = STATE_* enum; optional trailing UTF-8 string if payload>5. STATE enum: `0 unspecified,1 not_in_finger,2 finger_detection,3 user_active,4 user_in_rest,5 hr_user_active,6 hr_user_in_rest,7 out_of_power,8 charging,9 hibernate_low_power,20–22 production,30 hw_test`. [open_ring] - **`0x85` rtc_beacon_ind** (10 B): `unix_s:u32 LE`, reserved 4 B, trailer u16 LE ∈ {`0x01F6`,`0x01F8`}. [open_ring] From 63d62dff8efe770268422d0d78b46443f3b20a1c Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 11:39:51 +0200 Subject: [PATCH 02/21] =?UTF-8?q?Added=20live=200x43=20debug-text=20loggin?= =?UTF-8?q?g=20Gave=20.debugText=20its=20own=20ingest=20case=20(it=20previ?= =?UTF-8?q?ously=20fell=20into=20default=20and=20was=20dropped).=20It=20no?= =?UTF-8?q?w=20logs=20each=20non-empty,=20trimmed=20diagnostic=20string=20?= =?UTF-8?q?with=20its=20ring=20counter:=20=20=20Oura:=20debug=20(0x43)=20r?= =?UTF-8?q?t=3D60470=20"in=5Fbed=3D0"=20=20=20Diagnostics=20only=20?= =?UTF-8?q?=E2=80=94=20never=20persisted=20or=20scored.=20As=20a=20bonus,?= =?UTF-8?q?=20decoding=20these=20here=20also=20means=20a=20debug=20string?= =?UTF-8?q?=20can=20no=20longer=20silently=20alias=20a=20real=20event=20ta?= =?UTF-8?q?g=20on=20that=20code=20path.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- Strand/BLE/OuraLiveSource.swift | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index 68ed5d943..1d3ceb8c1 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -687,8 +687,21 @@ public final class OuraLiveSource: NSObject, ObservableObject { // steps (MET is not a step count; OuraStreamMapping drops .activityInfo unconditionally). log("Oura: activity (Tier-B) state=\(info.state) met=\(info.met)") + case .debugText(let ringTimestamp, let text): + // 0x43 debug_event: the ring's OWN ASCII firmware diagnostics (state strings), one per TLV + // record (OURA_PROTOCOL.md §6.15). Surfaced live for investigation - these carry useful + // sleep/history signal (captures show an explicit `in_bed=…` flag, `…in_info=…`, etc). Decoding + // them here also stops a debug string from *aliasing* a real event tag: a byte-misaligned + // parser reads a letter inside the text ('L'=0x4C / 'I'=0x49) as a `type` byte and mints a + // phantom sleep-summary (§2.4 desync rule). Diagnostics only - never persisted or scored. The + // trim+non-empty gate keeps a truncated tail byte from logging a blank line. + let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) + if !trimmed.isEmpty { + log("Oura: debug (0x43) rt=\(ringTimestamp) \"\(trimmed)\"") + } + default: - break // motion / state / rtcBeacon / debugText: not a durable Streams row (see OuraStreamMapping) + break // motion / state / rtcBeacon: not a durable Streams row (see OuraStreamMapping) } } } From a042268cee151c4c04b4f6d0669a01800bfb2232 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 13:14:12 +0200 Subject: [PATCH 03/21] =?UTF-8?q?A=20logDebug(=5F:rt:)=20gate=20on=20the?= =?UTF-8?q?=200x43=20case=20(OuraLiveSource.swift)=20with=20two=20mechanis?= =?UTF-8?q?ms:=201.=20Drop-prefix=20list=20for=20the=20known=20high-rate?= =?UTF-8?q?=20firmware=20chatter:=20=20=20DHR=5Fmode,=20DHR=20data=20sub,?= =?UTF-8?q?=20DHR=5Fstate,=20DHR=5Finfo,=20=20=20=E2=86=90=20~150ms=20HR?= =?UTF-8?q?=20state-machine=20spam=20=20=20PPG=5Fcont,=20S:,=20E:,=20AFs,?= =?UTF-8?q?=20blestda,=20nr,=20BQ=20=20=20=20=20=20=20=20=20=E2=86=90=20pe?= =?UTF-8?q?r-beat=20PPG=20windows=20/=20AGC=20/=20BLE=20state=20=20=201.?= =?UTF-8?q?=20Case-sensitive=20on=20purpose:=20the=20uppercase=20S:/E:=20P?= =?UTF-8?q?PG=20markers=20are=20dropped,=20while=20the=20lowercase=20s:/e:?= =?UTF-8?q?=20sleep=20boundaries=20survive.=20=20=202.=20Consecutive-dupli?= =?UTF-8?q?cate=20suppression=20=E2=80=94=20collapses=20runs=20of=20identi?= =?UTF-8?q?cal=20lines=20(e.g.=20the=20DHR=5Finfo:neg=20t=20=C3=9720=20bur?= =?UTF-8?q?st)=20via=20a=20lastDebugText=20guard.?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- Strand/BLE/OuraLiveSource.swift | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index 1d3ceb8c1..32cf06087 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -695,10 +695,15 @@ public final class OuraLiveSource: NSObject, ObservableObject { // parser reads a letter inside the text ('L'=0x4C / 'I'=0x49) as a `type` byte and mints a // phantom sleep-summary (§2.4 desync rule). Diagnostics only - never persisted or scored. The // trim+non-empty gate keeps a truncated tail byte from logging a blank line. + // The channel FLOODS during streaming (hundreds/connect) with repetitive firmware + // state-machine chatter - `DHR_mode:3` / `DHR data sub` every ~150 ms, per-beat PPG + // windows (`S:…`/`E:…`, uppercase), `DHR_state`/`PPG_cont`/`AFs`/`blestda`, and identical + // lines repeated dozens of times (`DHR_info:neg t`). `logDebug` drops those known + // high-rate internals and collapses consecutive duplicates, so the useful LOW-rate lines + // survive: sleep/lifecycle (`check_sleep`, `no_bedtime`, `wakeup`, `in_bed=…`, the + // lowercase `s:`/`e:`/`nb:` sleep boundaries), `auth_key_set`, `batt:…`, `orientation`. let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) - if !trimmed.isEmpty { - log("Oura: debug (0x43) rt=\(ringTimestamp) \"\(trimmed)\"") - } + logDebug(trimmed, rt: ringTimestamp) default: break // motion / state / rtcBeacon: not a durable Streams row (see OuraStreamMapping) @@ -706,6 +711,28 @@ public final class OuraLiveSource: NSObject, ObservableObject { } } + // MARK: - Debug-text (0x43) filter — Phase-1 investigation logging + + /// Prefixes of the ring's high-rate firmware chatter to DROP from the strap log (case-sensitive, so + /// the UPPERCASE PPG markers `S:`/`E:` are dropped while the lowercase sleep boundaries `s:`/`e:` + /// survive). Widen this list to quieten the log further, or trim it to see more while investigating. + private static let debugDropPrefixes = [ + "DHR_mode", "DHR data sub", "DHR_state", "DHR_info", // ~150 ms HR state-machine chatter + "PPG_cont", "S:", "E:", "AFs", "blestda", "nr", "BQ", // per-beat PPG windows / AGC / BLE state + ] + /// Last debug string we logged, to collapse consecutive identical lines (e.g. `DHR_info:neg t` ×20). + private var lastDebugText: String? + + /// Log a decoded 0x43 debug string unless it is empty, a known high-rate flood (drop-prefix), or an + /// immediate duplicate of the previous logged line. Investigation only — never persisted or scored. + private func logDebug(_ text: String, rt: UInt32) { + guard !text.isEmpty, + !Self.debugDropPrefixes.contains(where: { text.hasPrefix($0) }), + text != lastDebugText else { return } + lastDebugText = text + log("Oura: debug (0x43) rt=\(rt) \"\(text)\"") + } + // MARK: - Re-engagement timer (daytime-HR auto-reverts ~20s) private func startReengageTimer() { From 95a884c97a5209443fd75d56dbde08b4e81a1075 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 15:41:23 +0200 Subject: [PATCH 04/21] Oura: send SyncTime on connect to unlock the UTC anchor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The .streaming block now hands the ring the current UTC (OuraCommands.syncTime) BEFORE draining history, so it emits a usable 0x42 time-sync event (§5.5). Without this every fetched record stayed "[no anchor yet]" and last-night sleep / skin-temp could not be placed on a calendar day (Sleep screen read matched=0). SyncTime writes the ring clock, exactly as the official Oura app does on every connect. Sent once per session, before the first fetch. Co-Authored-By: Claude Opus 4.8 --- Strand/BLE/OuraLiveSource.swift | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index 32cf06087..1eba36b13 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -480,6 +480,13 @@ public final class OuraLiveSource: NSObject, ObservableObject { log("Oura: live-HR enabled - streaming HR / IBI") startReengageTimer() startHistoryFetchTimer() + // §5.3 step 1 / open_oura sync recipe step 4: hand the ring the current UTC BEFORE draining + // history so it can emit a usable 0x42 time-sync anchor (§5.5). Without this every fetched + // record stays "[no anchor yet]" and last-night sleep / skin-temp can't be placed on a + // calendar day (the Sleep screen reads matched=0). SyncTime WRITES the ring's clock - the + // phone owns ring time, exactly as the official Oura app does on every connect. Sent ONCE + // per session here, before the first fetch; the ack-fetch loop never re-sends it. + write([OuraCommands.syncTime(unixSeconds: Int(Date().timeIntervalSince1970))]) fetchHistoryIfIdle() // pull last night's banked temp/SpO2/HRV/sleep-phase right away write([OuraCommands.getBattery()]) // ask once HR streams; the 0x0D reply routes to onBattery } From 09ef1381480e1566bfee6ab38599000f7e5164ff Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 15:41:23 +0200 Subject: [PATCH 05/21] Oura: resync the TLV reassembler on an implausible record type OuraReassembler.feed only validated len >= 4, never the type byte. Once byte-misaligned inside a 0x43 debug-text payload, ASCII letters that alias real tags (p=0x70, I=0x49, L=0x4C, Q=0x51) minted a PHANTOM "summary" whose bogus len swallowed the following real records (e.g. the sleep-phase timeline). Now a record only starts on a plausible type - a known OuraEventTag or the 0x11/0x0D outer responses that round-trip - else drop one byte and re-scan. Adds isPlausibleRecordStart + 3 FramingTests; realigns instead of stalling. Co-Authored-By: Claude Opus 4.8 --- .../Sources/OuraProtocol/Framing.swift | 23 +++++++++++ .../OuraProtocolTests/FramingTests.swift | 39 ++++++++++++++++++- 2 files changed, 60 insertions(+), 2 deletions(-) diff --git a/Packages/OuraProtocol/Sources/OuraProtocol/Framing.swift b/Packages/OuraProtocol/Sources/OuraProtocol/Framing.swift index 226f9e1bb..3498e1d38 100644 --- a/Packages/OuraProtocol/Sources/OuraProtocol/Framing.swift +++ b/Packages/OuraProtocol/Sources/OuraProtocol/Framing.swift @@ -168,6 +168,7 @@ public final class OuraReassembler { buf.append(contentsOf: fragment) var out: [OuraRecord] = [] while buf.count >= 2 { + let type = buf[0] let len = Int(buf[1]) // A record must cover its 4 timestamp bytes. A len < 4 here is a misaligned byte: drop one // and resync rather than emit garbage (honest-data invariant). @@ -175,6 +176,18 @@ public final class OuraReassembler { buf.removeFirst(1) continue } + // RESYNC on an implausible type (OURA_PROTOCOL.md §2.4). The type byte must be a real record + // start: a known inner event tag (>= 0x41), or the GetEvents-summary (0x11) / battery (0x0D) + // OUTER responses that legitimately ride the same wire and round-trip as sized no-op records. + // Any other byte means we are byte-misaligned - typically landed INSIDE a 0x43 debug-text + // payload, where ASCII bytes that alias real tags ('p'=0x70 / 'I'=0x49 / 'L'=0x4C / 'Q'=0x51) + // would otherwise mint a PHANTOM "summary" whose bogus len swallows the following real records + // (e.g. the sleep-phase timeline). Drop one byte and re-scan instead of consuming 2+len of + // garbage, so the parser realigns to the next genuine record boundary. + if !Self.isPlausibleRecordStart(type) { + buf.removeFirst(1) + continue + } let total = 2 + len if buf.count < total { break // wait for the rest of this record @@ -187,6 +200,16 @@ public final class OuraReassembler { return out } + /// A byte that can legitimately BEGIN a record on the notify channel: a known inner event tag, or + /// the two OUTER command responses that share the wire and are consumed as sized no-op records - the + /// GetEvents summary (`0x11`) and battery (`0x0D`), both below the `0x41` tag range. Everything else + /// is a byte-misalignment to resync past (§2.4). Kept private + pure so it is exercised via `feed`. + static func isPlausibleRecordStart(_ type: UInt8) -> Bool { + OuraEventTag(rawValue: type) != nil + || type == OuraFraming.getEventsResponseOp + || type == OuraFraming.batteryResponseOp + } + /// Discard any buffered partial bytes (call on disconnect so a half-record does not bleed into the /// next session). Mirrors the StandardHRSource stop()/reset discipline. public func reset() { diff --git a/Packages/OuraProtocol/Tests/OuraProtocolTests/FramingTests.swift b/Packages/OuraProtocol/Tests/OuraProtocolTests/FramingTests.swift index 8117c5cbd..2ef905306 100644 --- a/Packages/OuraProtocol/Tests/OuraProtocolTests/FramingTests.swift +++ b/Packages/OuraProtocol/Tests/OuraProtocolTests/FramingTests.swift @@ -184,11 +184,46 @@ final class FramingTests: XCTestCase { } func testReassemblerLenBelowFourDoesNotEmitGarbageBeforeValidRecord() { - // A 2-byte garbage header whose len is < 4 is dropped one byte at a time; because TLV has no - // SOF this does not realign to the trailing valid record, but it must NOT emit a bogus record. + // A 2-byte garbage header whose len is < 4 is dropped one byte at a time. With the §2.4 type- + // resync it now REALIGNS to the trailing valid record (a byte whose type is a known tag) instead + // of stalling, and never emits a bogus record. let valid = bytes("4e0602000100006c") let r = OuraReassembler() let recs = r.feed([0x00, 0x01] + valid) // 00 01 = len 1 (< 4) XCTAssertTrue(recs.allSatisfy { $0.type != 0x00 }, "must never emit a type-0 garbage record") + XCTAssertEqual(recs.map { $0.type }, [0x4E], "resync recovers the trailing valid record") + } + + // MARK: - Reassembler: §2.4 type resync (phantom-summary prevention) + + func testIsPlausibleRecordStart() { + XCTAssertTrue(OuraReassembler.isPlausibleRecordStart(0x4E)) // known sleep-phase tag + XCTAssertTrue(OuraReassembler.isPlausibleRecordStart(0x70)) // known spo2-smoothed tag + XCTAssertTrue(OuraReassembler.isPlausibleRecordStart(0x11)) // GetEvents summary outer op + XCTAssertTrue(OuraReassembler.isPlausibleRecordStart(0x0D)) // battery outer op + XCTAssertFalse(OuraReassembler.isPlausibleRecordStart(0x30)) // ASCII '0' - misalignment + XCTAssertFalse(OuraReassembler.isPlausibleRecordStart(0x3B)) // ASCII ';' - misalignment + } + + func testReassemblerResyncsPastUnknownTypeToRecoverNextRecord() { + // Two junk bytes whose type is NOT a plausible record start (ASCII '0' then ';', each with a + // len >= 4 that would otherwise swallow the real record as a phantom body) precede a valid + // sleep-phase record. The parser must resync byte-by-byte and recover the real record. + let r = OuraReassembler() + let recs = r.feed(bytes("303b" + "4e0602000100006c")) + XCTAssertEqual(recs.map { $0.type }, [0x4E]) + XCTAssertEqual(r.bufferedByteCount, 0) + } + + func testReassemblerPreservesSummaryAndBatteryOuterResponses() { + // The GetEvents summary (0x11) and battery (0x0D) outer responses ride the same wire and must be + // consumed as sized no-op records (NOT resync'd away), so a real event packed behind them still + // emerges. Regression guard for the round-trip the notify-channel demux relies on. + let summary = bytes("1108ff00727202000300") // 0x11 len=8, 10 bytes total + let event = bytes("4e0602000100006c") // real sleep-phase record + let r = OuraReassembler() + let recs = r.feed(summary + event) + XCTAssertEqual(recs.map { $0.type }, [0x11, 0x4E]) + XCTAssertEqual(r.bufferedByteCount, 0) } } From 6f83358729042f069bcf7f2d922c5c01f3ce5291 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:01:28 +0200 Subject: [PATCH 06/21] Oura: gate SpO2 persistence to plausible [85,100] percentages A reassembler-misaligned phantom landing on a known spo2 tag byte decoded to impossible values and got persisted (integration build: 159 spo2Sample rows, -320..12,856,474), violating the honest-data invariant on a ring whose SpO2 feature 0x04 is gated OFF. OuraStreamMapping now persists an .spo2 sample only when its value is in the plausible oxygen-saturation band [85,100], aligned with open_oura tools/run_spo2.py (r-ratio calibration clamped to [85,100]) and Oura's own reporting floor. Values outside are DROPPED, never clamped - clamping garbage to "100%" would fabricate an oxygen reading. Only 0x6F yields direct percentages (~95-96, s6.5); 0x7B (unpinned uint16) and 0x77 dc_raw (PPG waveform) are not percentages, and nothing downstream reads spo2 red (AnalyticsEngine nulls spo2Pct), so this loses no real signal and yields zero rows on the gated-off ring. Mirrored in the Kotlin twin for parity. Co-Authored-By: Claude Opus 4.8 --- .../WhoopStore/OuraStreamMapping.swift | 27 +++++++++++-- .../OuraStreamMappingTests.swift | 38 ++++++++++++++++--- .../java/com/noop/data/OuraStreamMapping.kt | 28 ++++++++++++-- .../com/noop/data/OuraStreamMappingTest.kt | 34 +++++++++++++++++ 4 files changed, 114 insertions(+), 13 deletions(-) diff --git a/Packages/WhoopStore/Sources/WhoopStore/OuraStreamMapping.swift b/Packages/WhoopStore/Sources/WhoopStore/OuraStreamMapping.swift index f84901f72..a0209ed87 100644 --- a/Packages/WhoopStore/Sources/WhoopStore/OuraStreamMapping.swift +++ b/Packages/WhoopStore/Sources/WhoopStore/OuraStreamMapping.swift @@ -31,12 +31,22 @@ public enum OuraStreamMapping { /// WhoopEvent.kind for a decoded sleep-phase code (2-bit: awake/light/deep/rem). public static let sleepPhaseEventKind = "OURA_SLEEP_PHASE" + /// Plausible SpO2 oxygen-saturation percentage band. Aligned with open_oura `tools/run_spo2.py`, + /// which computes SpO2 from the r-ratio (tag 0x8b) and CLAMPS the result to `[85, 100]` — Oura's own + /// reporting floor and the physiologically plausible band for a worn ring. NOOP uses it as a REJECT + /// gate at the persist boundary (drop outside), NEVER a clamp: an out-of-band value is either a raw + /// sub-channel that is not a percentage (0x77 `dc_raw` PPG waveform, 0x7B unpinned uint16) or a + /// reassembler-misaligned phantom (the −63…4.7M garbage seen on a SpO2-gated-off Gen 3 Horizon), so + /// there is no genuine reading to clamp — forcing garbage to "100 %" would fabricate an oxygen value, + /// violating the honest-data invariant. Must match the Kotlin twin (OuraStreamMapping.kt). + public static let plausibleSpO2Percent = 85...100 + /// Build a `Streams` from a batch of decoded Oura events, all stamped at the arrival wall-clock `ts` /// (unix seconds). Pure → unit-testable. Section-4 table: /// - `.hr` (0x55 live-HR push) → `hr:[HRSample]` /// - `.ibi` (0x44/0x60 IBI) → `rr:[RRInterval]` /// - `.hrv` (0x5D HRV tag, raw int8 b1/b2) → `events:[WhoopEvent(kind: OURA_HRV)]` - /// - `.spo2` (0x6F/0x70/0x77) → `spo2:[SpO2Sample(raw_adc)]` + /// - `.spo2` (0x6F direct %) → `spo2:[SpO2Sample]` (only plausible % [85,100]) /// - `.temp` (0x46/0x75) → `skinTemp:[SkinTempSample(raw_adc)]` /// - `.sleepPhase` (0x4E/0x5A 2-bit codes) → `events:[WhoopEvent(kind: OURA_SLEEP_PHASE)]` /// - `.battery` → `battery:[BatterySample]` @@ -74,9 +84,18 @@ public enum OuraStreamMapping { ])) case .spo2(let v): - // Oura reports a single SpO2 channel; `SpO2Sample` is the WHOOP-shaped two-channel raw row, - // so we record the decoded value on `red` and leave `ir` at 0 (no second channel). `unit` - // carries the decoder's own scale tag ("raw"/"dc_raw") so downstream never assumes a %. + // Persist only PLAUSIBLE SpO2 percentages (`plausibleSpO2Percent`, open_oura's [85,100]). + // The ONLY decoder with established percentage semantics is 0x6F (direct per-second %, + // ~95-96; OURA_PROTOCOL.md s6.5); the 0x7B uint16 (unpinned scale) and 0x77 `dc_raw` PPG + // waveform are NOT oxygen-saturation percentages, so any value outside the band is either + // one of those raw sub-channels or a reassembler-misaligned phantom (the −63…4.7M garbage + // seen on a SpO2-gated-off Gen 3 Horizon). We DROP it rather than persist an impossible + // reading: nothing downstream reads spo2 `red` (AnalyticsEngine nulls spo2Pct), so this + // loses no real signal and yields ZERO rows on a ring with SpO2 feature 0x04 OFF. A genuine + // 0x6F ~95-96 still passes. `SpO2Sample` is the WHOOP-shaped two-channel raw row: decoded + // value on `red`, `ir` at 0 (no second channel), `unit` carries the decoder's own scale + // tag. PARITY: mirror this gate in the Kotlin twin (OuraStreamMapping.kt) exactly. + guard plausibleSpO2Percent.contains(v.value) else { continue } out.spo2.append(SpO2Sample(ts: ts, red: v.value, ir: 0, unit: v.unit)) case .temp(let v): diff --git a/Packages/WhoopStore/Tests/WhoopStoreTests/OuraStreamMappingTests.swift b/Packages/WhoopStore/Tests/WhoopStoreTests/OuraStreamMappingTests.swift index 271c8d2d8..d62cdd066 100644 --- a/Packages/WhoopStore/Tests/WhoopStoreTests/OuraStreamMappingTests.swift +++ b/Packages/WhoopStore/Tests/WhoopStoreTests/OuraStreamMappingTests.swift @@ -53,16 +53,44 @@ final class OuraStreamMappingTests: XCTestCase { // MARK: - SpO2 -> spo2:[SpO2Sample] func testSpO2MapsToSpO2StreamPreservingUnit() { + // A genuine 0x6F direct percentage (~95-96) passes the plausibility gate and keeps its unit tag. let s = OuraStreamMapping.streams(from: [ - .spo2(OuraSpO2(ringTimestamp: 100, value: 970, unit: "raw")), - .spo2(OuraSpO2(ringTimestamp: 101, value: 12345, unit: "dc_raw")), + .spo2(OuraSpO2(ringTimestamp: 100, value: 96, unit: "raw")), + .spo2(OuraSpO2(ringTimestamp: 101, value: 88, unit: "raw")), ], at: ts) - XCTAssertEqual(s.spo2.map { $0.red }, [970, 12345]) + XCTAssertEqual(s.spo2.map { $0.red }, [96, 88]) XCTAssertEqual(s.spo2.map { $0.ir }, [0, 0]) - XCTAssertEqual(s.spo2.map { $0.unit }, ["raw", "dc_raw"]) + XCTAssertEqual(s.spo2.map { $0.unit }, ["raw", "raw"]) XCTAssertEqual(s.spo2.map { $0.ts }, [ts, ts]) } + func testSpO2ImplausiblePercentagesAreDropped() { + // Everything outside open_oura's [85,100] band is a raw sub-channel (0x7B unpinned uint16 / 0x77 + // dc_raw waveform) or a reassembler-misaligned phantom - never persisted, never clamped. Covers + // the real garbage range observed on a SpO2-gated-off Gen 3 Horizon: negatives, zero, >100 %, and + // the multi-million dc_raw accumulator. The lone in-band 96 survives so the batch is not rejected + // wholesale. + let s = OuraStreamMapping.streams(from: [ + .spo2(OuraSpO2(ringTimestamp: 1, value: -320, unit: "dc_raw")), + .spo2(OuraSpO2(ringTimestamp: 2, value: 0, unit: "raw")), + .spo2(OuraSpO2(ringTimestamp: 3, value: 84, unit: "raw")), // just below the floor + .spo2(OuraSpO2(ringTimestamp: 4, value: 96, unit: "raw")), // genuine reading + .spo2(OuraSpO2(ringTimestamp: 5, value: 103, unit: "raw")), // impossible % + .spo2(OuraSpO2(ringTimestamp: 6, value: 970, unit: "raw")), // 0x7B raw, not a % + .spo2(OuraSpO2(ringTimestamp: 7, value: 12_856_474, unit: "dc_raw")), + ], at: ts) + XCTAssertEqual(s.spo2.map { $0.red }, [96], "only the in-band 96% survives") + } + + func testSpO2BandEndpointsAreInclusive() { + // 85 and 100 are the accepted bounds (open_oura run_spo2.py clamp endpoints). + let s = OuraStreamMapping.streams(from: [ + .spo2(OuraSpO2(ringTimestamp: 1, value: 85, unit: "raw")), + .spo2(OuraSpO2(ringTimestamp: 2, value: 100, unit: "raw")), + ], at: ts) + XCTAssertEqual(s.spo2.map { $0.red }, [85, 100]) + } + // MARK: - Temp 0x46/0x75 -> skinTemp:[SkinTempSample] (centi-degree-C, parity with Kotlin) func testTempMapsToSkinTempAsCentiC() { @@ -142,7 +170,7 @@ final class OuraStreamMappingTests: XCTestCase { .hr(OuraHR(ringTimestamp: 1, bpm: 55, ibiMs: 1090)), .ibi(OuraIBI(ringTimestamp: 1, ibiMs: 1090)), .hrv(OuraHRV(ringTimestamp: 1, timeMs: 0, b1: 40, b2: 1)), - .spo2(OuraSpO2(ringTimestamp: 1, value: 965)), + .spo2(OuraSpO2(ringTimestamp: 1, value: 96)), .temp(OuraTemp(ringTimestamp: 1, celsius: 34.0)), .sleepPhase(OuraSleepPhase(ringTimestamp: 1, index: 0, stage: .light)), .battery(OuraBattery(percent: 88)), diff --git a/android/app/src/main/java/com/noop/data/OuraStreamMapping.kt b/android/app/src/main/java/com/noop/data/OuraStreamMapping.kt index a78db2ad8..9c887696c 100644 --- a/android/app/src/main/java/com/noop/data/OuraStreamMapping.kt +++ b/android/app/src/main/java/com/noop/data/OuraStreamMapping.kt @@ -37,6 +37,18 @@ object OuraStreamMapping { /** The event `kind` recorded for the ring's own open sleep-phase (0x49.../0x58) tags. */ const val EVENT_SLEEP_PHASE = "OURA_SLEEP_PHASE" + /** + * Plausible SpO2 oxygen-saturation percentage band. Aligned with open_oura `tools/run_spo2.py`, + * which computes SpO2 from the r-ratio (tag 0x8b) and CLAMPS the result to [85, 100] - Oura's own + * reporting floor and the physiologically plausible band for a worn ring. Used as a REJECT gate at + * the persist boundary (drop outside), NEVER a clamp: an out-of-band value is either a raw + * sub-channel that is not a percentage (0x77 dc_raw PPG waveform, 0x7B unpinned uint16) or a + * reassembler-misaligned phantom (the -63..4.7M garbage seen on a SpO2-gated-off Gen 3 Horizon), so + * there is no genuine reading to clamp - forcing garbage to "100%" would fabricate an oxygen value, + * violating the honest-data invariant. Must match the Swift twin (OuraStreamMapping.plausibleSpO2Percent). + */ + val PLAUSIBLE_SPO2_PERCENT = 85..100 + /** * Fold a batch of decoded [events] into a protocol [Streams] for one flush. [anchor] maps a * ring-clock timestamp to wall-clock unix seconds (null => drop the sample). Pure: no BLE, no DB, @@ -76,10 +88,18 @@ object OuraStreamMapping { } is OuraEvent.Spo2 -> { - // The ring exposes ONE combined SpO2 reading (not separate red/ir channels): its - // raw value goes in `red`; `ir` stays 0 (an unread channel, never a fabricated - // second reading). `unit` carries the decoder's own scale tag so downstream never - // assumes a percentage, mirroring the Swift twin's SpO2Sample(unit:). + // Persist only PLAUSIBLE SpO2 percentages (PLAUSIBLE_SPO2_PERCENT, open_oura's + // [85,100]). The ONLY decoder with established percentage semantics is 0x6F (direct + // per-second %, ~95-96; OURA_PROTOCOL.md s6.5); the 0x7B uint16 (unpinned scale) and + // 0x77 dc_raw PPG waveform are NOT oxygen-saturation percentages, so any value outside + // the band is either one of those raw sub-channels or a reassembler-misaligned phantom + // (the -63..4.7M garbage seen on a SpO2-gated-off Gen 3 Horizon). DROP it rather than + // persist an impossible reading: nothing downstream reads spo2 red, so this loses no + // real signal and yields ZERO rows on a ring with SpO2 feature 0x04 OFF. A genuine 0x6F + // ~95-96 still passes. The ring exposes ONE combined channel: value in `red`, `ir` 0 + // (unread channel, never fabricated), `unit` carries the decoder's own scale tag. + // PARITY: mirror the Swift twin's [85,100] gate exactly. + if (ev.value.value !in PLAUSIBLE_SPO2_PERCENT) continue val ts = anchor(ev.value.ringTimestamp) ?: continue out.spo2.add(Spo2Sample(ts = ts, red = ev.value.value, ir = 0, unit = ev.value.unit)) } diff --git a/android/app/src/test/java/com/noop/data/OuraStreamMappingTest.kt b/android/app/src/test/java/com/noop/data/OuraStreamMappingTest.kt index 000ad0760..64e53e70f 100644 --- a/android/app/src/test/java/com/noop/data/OuraStreamMappingTest.kt +++ b/android/app/src/test/java/com/noop/data/OuraStreamMappingTest.kt @@ -95,6 +95,40 @@ class OuraStreamMappingTest { assertEquals(base + 1, s.spo2.first().ts) } + @Test + fun spo2ImplausiblePercentagesAreDropped() { + // PARITY with Swift testSpO2ImplausiblePercentagesAreDropped: only values in open_oura's + // [85,100] band survive. Everything outside is a raw sub-channel (0x7B unpinned uint16 / 0x77 + // dc_raw waveform) or a reassembler-misaligned phantom - never persisted, never clamped. Covers + // the garbage range seen on a SpO2-gated-off Gen 3 Horizon (negatives, zero, >100%, multi-million + // dc_raw accumulator); the lone in-band 96 survives so the batch is not rejected wholesale. + val s = OuraStreamMapping.streams( + listOf( + OuraEvent.Spo2(OuraSpO2(ringTimestamp = 1, value = -320, unit = "dc_raw")), + OuraEvent.Spo2(OuraSpO2(ringTimestamp = 2, value = 0)), + OuraEvent.Spo2(OuraSpO2(ringTimestamp = 3, value = 84)), // just below the floor + OuraEvent.Spo2(OuraSpO2(ringTimestamp = 4, value = 96)), // genuine reading + OuraEvent.Spo2(OuraSpO2(ringTimestamp = 5, value = 103)), // impossible % + OuraEvent.Spo2(OuraSpO2(ringTimestamp = 6, value = 970)), // 0x7B raw, not a % + OuraEvent.Spo2(OuraSpO2(ringTimestamp = 7, value = 12_856_474, unit = "dc_raw")), + ), + anchor, + ) + assertEquals(listOf(96), s.spo2.map { it.red }) // only the in-band 96% survives + } + + @Test + fun spo2BandEndpointsAreInclusive() { + val s = OuraStreamMapping.streams( + listOf( + OuraEvent.Spo2(OuraSpO2(ringTimestamp = 1, value = 85)), + OuraEvent.Spo2(OuraSpO2(ringTimestamp = 2, value = 100)), + ), + anchor, + ) + assertEquals(listOf(85, 100), s.spo2.map { it.red }) + } + @Test fun tempPersistsAsHundredthsOfDegree() { val s = OuraStreamMapping.streams( From 2e7a6ff535ab7952a04ce2608f6ec3a38b3bf905 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 16:09:34 +0200 Subject: [PATCH 07/21] =?UTF-8?q?=20=20=20Docs:=20SleepNet=20is=20cloud-lo?= =?UTF-8?q?cked=20=E2=80=94=20NOOP=20builds=20its=20own=20sleep=20session?= =?UTF-8?q?=20(=C2=A76.12.1)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Investigated open_oura docs/algorithms/sleepnet.md. The Oura app's 4-stage hypnogram is NOT on the BLE wire: it is SleepNet, an on-PHONE PyTorch model (per-30s epochs) shipped AES-256-GCM-encrypted with a server-delivered, cloud-cached key — the one Oura metric not reproducible without Oura's cloud. Documents the consequence for NOOP's honest-data invariant: we do not chase SleepNet. A NOOP sleep session is built from signals that cross the BLE boundary — the ring's Tier-A 2-bit 0x4E/0x5A phase codes (OURA_SLEEP_PHASE), its 0x49/0x76 boundaries and 0x43 debug in_bed/s:/e: narration, plus NOOP's own staging over raw HR/HRV/temp/motion — never relabelled as "Oura sleep stages". Adds a [sleepnet] citation key. --- docs/OURA_PROTOCOL.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/OURA_PROTOCOL.md b/docs/OURA_PROTOCOL.md index 05d5b0736..b61e66ea7 100644 --- a/docs/OURA_PROTOCOL.md +++ b/docs/OURA_PROTOCOL.md @@ -12,6 +12,7 @@ - **[open_oura-feat]** - Th0rgal/open_oura `docs/ring-features.md` (feature gating). - **[relue]** - relue/oura_ring_reverse `docs/.../heartbeat_replication_guide.md` and `heartbeat_complete_flow.md` (no-license; Ring 3 live-HR). - **[oura-rs]** - Th0rgal/open_oura `crates/oura-protocol/src/events.rs` (no-license Rust clean-room decoder; facts cited only, no code copied). Its event tags marked `"_status": "unvalidated"` are treated the same as our Tier B - plausible, not ground-truth-confirmed. +- **[sleepnet]** - Th0rgal/open_oura `docs/algorithms/sleepnet.md` (no-license; describes Oura's on-phone sleep-staging model + its encryption, NOT a BLE protocol layout). See §6.12.1. > **CONFLICT NOTE (resolution rule):** The relue archive file `event_data_definition.md` describes events as **protobuf varint** records (e.g. `0x55` SLEEP_HR with field tags). This contradicts the **byte-for-byte verified TLV framing** in [open_ring] and [ringverse]. The TLV/bit-packed model from [open_ring]/[ringverse] is authoritative for our decoders; the protobuf description is treated as unverified/likely AI-fabricated and is NOT used. Where a layout is only attested by a single no-license, AI-generated doc, it is marked **(UNVERIFIED)** and our decoder must gate it behind a fixture test before trusting it. @@ -358,6 +359,28 @@ bits 14–15 : qual_b - **`0x76` `bedtime_period`**: start/end as uint32 LE ringTimestamps → map to UTC (§5.5). [ringverse] - Tags `0x48,0x4A–0x4D,0x4F,0x57,0x58` are additional sleep summary/feature variants in the dictionary; layouts **(UNVERIFIED)** - decode only after fixtures. [ringverse] **Beware phantom sightings:** a `0x4C`/`0x49` "sleep_summary" whose payload is ASCII (e.g. contains `in_bed=…`, readable words) is almost certainly misframed `0x43` debug text, not a real summary — see §2.4 desync rule and §6.15. Confirm a candidate summary is binary and length-consistent before treating it as a fixture. +#### 6.12.1 The polished hypnogram is cloud-locked (SleepNet) — NOOP does not chase it +The 4-stage hypnogram the **Oura app** shows is **not** produced on the ring and is **not** on the BLE +wire. It is the output of **SleepNet**, a PyTorch model that runs **on the phone** (in Oura's app, not +in the ring's firmware/`ecore`) and classifies sleep in **per-30-second epochs**. The model ships as +`oura_models.apk/assets/sleepstaging_2_6_0.pt.enc` (~114 KB) **encrypted AES-256-GCM** +(`[12-byte IV][ciphertext + 16-byte tag]`, `AES/GCM/NoPadding`, `GCMParameterSpec(128, iv)`); the GCM key +is **fetched from Oura's servers** (per-model, labelled, rotatable) and only cached on a logged-in device. +So the finished hypnogram is **the one Oura metric that is not reproducible without Oura's cloud**. [sleepnet] + +**Consequence for NOOP (honest-data invariant, §1):** we neither can nor do reproduce SleepNet — the key +is cloud-gated, and even with it, surfacing Oura's encrypted proprietary score is exactly what NOOP does +not do. NOOP builds its **own** sleep session from signals that genuinely cross the BLE boundary: +- the ring's own **2-bit `0x4E`/`0x5A` phase codes** (§6.12, **Tier A** — the ring's coarse on-device + awake/light/deep/REM classification, decoded and persisted as `OURA_SLEEP_PHASE` events); +- the ring's own **sleep-window boundaries** — `0x49`/`0x76` summary/bedtime (Tier-B until fixtured) and + the **`0x43` debug narration** `check_sleep` / `in_bed` / `s: ` / `e: ` (§6.15), which on + live Gen 3 (2026-07-08) reported a ~552-min window corroborating the `0x49` summary; +- plus NOOP's **own** staging over raw HR / HRV / skin-temp / motion. + +These are leads for a NOOP-side sleep-session builder, **not** a path to Oura's hypnogram. Do not label a +NOOP-derived hypnogram as "Oura sleep stages": it is NOOP's own, from the ring's raw + coarse signals. + ### 6.13 Motion / activity - **`0x47` `motion_events`** (variable): byte6 bits`[7:5]`=field_a, `[4:0]`=field_b; bytes7–9 = three **int8 × 8** axis magnitudes; optional bytes10–11. [ringverse] - **`0x6B` `motion_period`** (19–31 B): 12-bit period `((b6<<8)|(b6>>6)) & 0xFFF`; byte6 bits`[5:4]`=leading-symbol count; then 2-bit codes, 4 per byte (MSB-first). MOTION_STATE enum: `0 NO_MOTION, 1 RESTLESS, 2 TOSSING, 3 ACTIVE`. [ringverse][open_ring] From df261a34f4060a164925de4aaba4a1379201f628 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 17:03:00 +0200 Subject: [PATCH 08/21] Oura: reject phantom ring timestamps far from the session anchor A framing desync mints a record with a garbage ring timestamp that, at 100 ms/tick, converts to a date days-to-years from the anchor yet still inside the loose 2020-2035 epoch gate (rt~1.9e9 -> +6yr/2033; rt~16.7M -> +19 days). On live Gen 3 this scattered ~25% of a night's skin-temp rows across 2020-2034, so they never landed on the correct calendar day - and the same phantoms carried the garbage SpO2 values. OuraDriver.unixSeconds now rejects any conversion more than +1 day after or -90 days before the session anchor (history is always recent-past relative to the 0x42 anchor). Returns nil so the caller drops/parks the sample instead of banking a mis-dated row (honest-data invariant) - the single chokepoint every history sample passes. Mirrored in the Kotlin twin; +4 tests (Swift+Kotlin), OURA_PROTOCOL.md 5.5 documents the guard. 83 OuraProtocol tests pass. Co-Authored-By: Claude Opus 4.8 --- .../Sources/OuraProtocol/OuraDriver.swift | 22 ++++++++++ .../OuraProtocolTests/OuraDriverTests.swift | 32 ++++++++++++++ .../src/main/java/com/noop/oura/OuraDriver.kt | 23 ++++++++++ .../test/java/com/noop/oura/OuraDriverTest.kt | 42 +++++++++++++++++++ docs/OURA_PROTOCOL.md | 3 +- 5 files changed, 121 insertions(+), 1 deletion(-) diff --git a/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift b/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift index 4c9343286..9541dc6ef 100644 --- a/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift +++ b/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift @@ -225,9 +225,31 @@ public final class OuraDriver { // 1970 or far-future sample. let seconds = ms / 1000 guard seconds >= Self.minPlausibleEpochSeconds, seconds <= Self.maxPlausibleEpochSeconds else { return nil } + // Phantom-record guard (anchor-relative). The absolute 2020-2035 gate above is far too loose to + // catch a MISFRAMED record: a framing desync on a 0x43 debug byte mints a record with a GARBAGE + // ring timestamp that, at 100 ms/tick, lands days-to-years from the anchor yet still INSIDE + // 2020-2035 - e.g. rt ~= 1.9e9 converts to +6 years (2033), rt ~= 16.7M to +19 days. Every such + // sample was persisted with a bogus date, scattering real streams across the calendar (observed: + // ~25% of a night's skin-temp rows landed in 2020-2034). A genuine history-fetched sample is + // ALWAYS in the recent past relative to the session anchor (which reflects ~now via the 0x42 + // time-sync): at most a small clock-skew margin AFTER it, and at most the ring's history depth + // BEFORE it. Reject anything outside that window so the caller drops/parks it instead of banking a + // mis-dated row (honest-data invariant). This is the single chokepoint every history sample passes. + let anchorSeconds = anchorUtcMs / 1000 + guard seconds <= anchorSeconds + Self.maxFutureAnchorOffsetSeconds, + seconds >= anchorSeconds - Self.maxPastAnchorOffsetSeconds else { return nil } return Int(seconds) } + /// Anchor-relative plausibility window for a history-fetched sample (phantom-record guard, above). + /// History is always in the recent past relative to the session anchor (≈ now): a real sample can be + /// at most a clock-skew/timezone margin AFTER the anchor (+1 day) and at most the ring's history depth + /// BEFORE it (−90 days, generous - the ring's flash cannot bank months of multi-stream 30 s data). A + /// misframed record's garbage ring timestamp lands far outside this and is dropped. Tunable if a real + /// deep-backlog capture ever proves the past bound too tight. + private static let maxFutureAnchorOffsetSeconds: Int64 = 86_400 // +1 day + private static let maxPastAnchorOffsetSeconds: Int64 = 7_776_000 // −90 days + /// Bounds for a plausible anchor epoch (unix seconds): 2020-01-01 to 2035-01-01. A decoded 0x42/0x85 /// value outside this range is a corrupt/misaligned record (seen on real hardware: a full cursor=0 /// history dump hit one deep in the backlog) and is never trusted as an anchor (honest-data invariant). diff --git a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift index 681c6fe62..123685d0c 100644 --- a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift +++ b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift @@ -197,6 +197,38 @@ final class OuraDriverTests: XCTestCase { XCTAssertEqual(d.unixSeconds(forRingTimestamp: anchorRt + 100), Int(anchorEpochSeconds) + 10) } + func testPhantomRingTimestampFarFromAnchorIsRejected() { + // A misframed record's garbage ring timestamp converts to a date days-to-years from the anchor but + // still INSIDE the loose 2020-2035 absolute window. The anchor-relative guard must reject it so it + // is never banked with a bogus date (the ~25% of a night's rows that scattered across 2020-2034). + let d = OuraDriver(ringGen: .gen3, authKey: key) + let anchorEpochSeconds: Int64 = 1_700_000_000 + let anchorRt: UInt32 = 100_000_000 // large so we can also probe a far-PAST rt (rt=0) + _ = d.ingest(record: OuraRecord(type: OuraEventTag.timeSync.rawValue, ringTimestamp: anchorRt, + payload: le8(anchorEpochSeconds) + [0x00])) + + // Recent-past sample (-2.78 h) is kept - a normal history-fetched record. + XCTAssertEqual(d.unixSeconds(forRingTimestamp: anchorRt - 100_000), Int(anchorEpochSeconds) - 10_000) + // Phantom FUTURE (~+3.17 years): inside 2020-2035 absolutely, but far beyond +1 day from anchor. + XCTAssertNil(d.unixSeconds(forRingTimestamp: anchorRt + 1_000_000_000)) + // Phantom PAST (~-115 days via rt=0): beyond the -90 day history window. + XCTAssertNil(d.unixSeconds(forRingTimestamp: 0)) + } + + func testAnchorRelativeWindowEndpointsAreInclusive() { + let d = OuraDriver(ringGen: .gen3, authKey: key) + let anchorEpochSeconds: Int64 = 1_700_000_000 + let anchorRt: UInt32 = 100_000_000 + _ = d.ingest(record: OuraRecord(type: OuraEventTag.timeSync.rawValue, ringTimestamp: anchorRt, + payload: le8(anchorEpochSeconds) + [0x00])) + // Exactly +1 day (864_000 ticks) is kept; a tick further into the future is rejected. + XCTAssertEqual(d.unixSeconds(forRingTimestamp: anchorRt + 864_000), Int(anchorEpochSeconds) + 86_400) + XCTAssertNil(d.unixSeconds(forRingTimestamp: anchorRt + 864_010)) + // Exactly -90 days (77_760_000 ticks) is kept; a tick further into the past is rejected. + XCTAssertEqual(d.unixSeconds(forRingTimestamp: anchorRt - 77_760_000), Int(anchorEpochSeconds) - 7_776_000) + XCTAssertNil(d.unixSeconds(forRingTimestamp: anchorRt - 77_760_100)) + } + func testRtcBeaconOnlyAnchorsWhenNoTimeSyncSeenYet() { let d = OuraDriver(ringGen: .gen3, authKey: key) let beaconRt: UInt32 = 5_000 diff --git a/android/app/src/main/java/com/noop/oura/OuraDriver.kt b/android/app/src/main/java/com/noop/oura/OuraDriver.kt index 1f1d66cfc..fec8b4137 100644 --- a/android/app/src/main/java/com/noop/oura/OuraDriver.kt +++ b/android/app/src/main/java/com/noop/oura/OuraDriver.kt @@ -301,6 +301,19 @@ class OuraDriver( // 1970 or far-future sample. Byte-identical to the Swift twin. val seconds = ms / 1000 if (seconds < MIN_PLAUSIBLE_EPOCH_SECONDS || seconds > MAX_PLAUSIBLE_EPOCH_SECONDS) return null + // Phantom-record guard (anchor-relative). The absolute 2020-2035 gate above is far too loose to + // catch a MISFRAMED record: a framing desync on a 0x43 debug byte mints a record with a garbage + // ring timestamp that, at 100 ms/tick, lands days-to-years from the anchor yet still inside + // 2020-2035 (e.g. rt ~= 1.9e9 -> +6 years). A genuine history-fetched sample is always in the + // recent past relative to the session anchor (~now via 0x42): at most a clock-skew margin AFTER it + // (+1 day) and at most the ring's history depth BEFORE it (-90 days). Reject anything outside so the + // caller drops/parks it instead of banking a mis-dated row. Byte-identical to the Swift twin. + val anchorSeconds = anchorMs / 1000 + if (seconds > anchorSeconds + MAX_FUTURE_ANCHOR_OFFSET_SECONDS || + seconds < anchorSeconds - MAX_PAST_ANCHOR_OFFSET_SECONDS + ) { + return null + } return seconds } @@ -554,5 +567,15 @@ class OuraDriver( */ private const val MIN_PLAUSIBLE_EPOCH_SECONDS = 1_577_836_800L private const val MAX_PLAUSIBLE_EPOCH_SECONDS = 2_051_222_400L + + /** + * Anchor-relative plausibility window for a history-fetched sample (phantom-record guard). History + * is always in the recent past relative to the session anchor (~now): a real sample can be at most a + * clock-skew/timezone margin AFTER the anchor (+1 day) and at most the ring's history depth BEFORE + * it (-90 days, generous). A misframed record's garbage ring timestamp lands far outside this and is + * dropped. Byte-identical to Swift's maxFuture/maxPastAnchorOffsetSeconds. + */ + private const val MAX_FUTURE_ANCHOR_OFFSET_SECONDS = 86_400L // +1 day + private const val MAX_PAST_ANCHOR_OFFSET_SECONDS = 7_776_000L // -90 days } } diff --git a/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt b/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt index 781bbdbc8..72ec8e880 100644 --- a/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt +++ b/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt @@ -234,6 +234,48 @@ class OuraDriverTest { assertEquals(anchorEpochSeconds + 10, d.unixSeconds(forRingTimestamp = anchorRt + 100)) } + @Test + fun testPhantomRingTimestampFarFromAnchorIsRejected() { + // Parity with Swift testPhantomRingTimestampFarFromAnchorIsRejected. A misframed record's garbage + // ring timestamp converts to a date days-to-years from the anchor but still INSIDE 2020-2035; the + // anchor-relative guard must reject it so it is never banked with a bogus date. + val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) + val anchorEpochSeconds = 1_700_000_000L + val anchorRt = 100_000_000L // large so we can also probe a far-PAST rt (rt=0) + d.ingest( + OuraRecord( + type = OuraEventTag.TIME_SYNC.raw, ringTimestamp = anchorRt, + payload = le8(anchorEpochSeconds) + intArrayOf(0x00), + ), + ) + + // Recent-past sample (-2.78 h) is kept - a normal history-fetched record. + assertEquals(anchorEpochSeconds - 10_000, d.unixSeconds(forRingTimestamp = anchorRt - 100_000)) + // Phantom FUTURE (~+3.17 years): inside 2020-2035 absolutely, but far beyond +1 day from anchor. + assertNull(d.unixSeconds(forRingTimestamp = anchorRt + 1_000_000_000)) + // Phantom PAST (~-115 days via rt=0): beyond the -90 day history window. + assertNull(d.unixSeconds(forRingTimestamp = 0L)) + } + + @Test + fun testAnchorRelativeWindowEndpointsAreInclusive() { + val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) + val anchorEpochSeconds = 1_700_000_000L + val anchorRt = 100_000_000L + d.ingest( + OuraRecord( + type = OuraEventTag.TIME_SYNC.raw, ringTimestamp = anchorRt, + payload = le8(anchorEpochSeconds) + intArrayOf(0x00), + ), + ) + // Exactly +1 day (864_000 ticks) is kept; a tick further into the future is rejected. + assertEquals(anchorEpochSeconds + 86_400, d.unixSeconds(forRingTimestamp = anchorRt + 864_000)) + assertNull(d.unixSeconds(forRingTimestamp = anchorRt + 864_010)) + // Exactly -90 days (77_760_000 ticks) is kept; a tick further into the past is rejected. + assertEquals(anchorEpochSeconds - 7_776_000, d.unixSeconds(forRingTimestamp = anchorRt - 77_760_000)) + assertNull(d.unixSeconds(forRingTimestamp = anchorRt - 77_760_100)) + } + @Test fun testRtcBeaconOnlyAnchorsWhenNoTimeSyncSeenYet() { val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) diff --git a/docs/OURA_PROTOCOL.md b/docs/OURA_PROTOCOL.md index b61e66ea7..61d44abfc 100644 --- a/docs/OURA_PROTOCOL.md +++ b/docs/OURA_PROTOCOL.md @@ -262,7 +262,8 @@ Response: `13 05 00`. [open_ring] ### 5.5 Ring-time → UTC anchoring - The ring clock is in **ticks**: default **100 ms/tick** (10 Hz); burst mode **1 ms/tick** (`factor_flag=1`). [open_ring] - Anchor from event `0x42` (time-sync ind, §6.11): set `anchor.utc_ms` from the event's epoch and `anchor.ring_time` from current `ringTimestamp`. -- Conversion: `utc_ms = anchor.utc_ms + factor × (target_rt − anchor.ring_time)`, `factor ∈ {100,1}`. [open_ring] +- Conversion: `utc_ms = anchor.utc_ms + factor × (target_rt − anchor.ring_time)`, `factor ∈ {100,1}`. NOOP v1 uses `factor = 100` only (burst mode not yet modeled; the low-rate temp/SpO2/sleep streams are 100 ms/tick, confirmed by a real night's `s:`/`e:` window converting to ~9 h). [open_ring] +- **Phantom-record guard (anchor-relative plausibility).** A framing desync (§2.4) can mint a record with a garbage `ring_time` that, at 100 ms/tick, converts to a date **days-to-years from the anchor yet still inside the loose 2020–2035 epoch gate** — e.g. `rt ≈ 1.9e9 → +6 years (2033)`, `rt ≈ 16.7M → +19 days`. On live Gen 3 (2026-07-08) this scattered ~25% of a night's skin-temp rows across 2020–2034, so they never landed on the correct calendar day. Rule: a history-fetched sample is always in the **recent past** relative to the session anchor (≈ now via `0x42`); reject any conversion more than **+1 day** after or **−90 days** before the anchor (`OuraDriver.unixSeconds` returns nil → the caller drops/parks it, honest-data invariant). This is the single chokepoint every history sample passes; it complements the value-level gates (SpO2 §7.1, skin-temp funnel). - On `0x41` (ring start) with `rt` regression → invalidate anchor (zero it). [open_ring] - `0x85` RTC beacon gives 1-second-granularity `unix_s` as a secondary source. [open_ring] From ff49d7b1d18269fb379c75f5cfb954b51db2ff82 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 22:32:16 +0200 Subject: [PATCH 09/21] Oura: build a NOOP sleep session from the ring's phase timeline AnalyticsEngine derives sleep from the gravity-driven SleepStager, and an Oura ring streams no accelerometer - so Sleep + Skin Temp stayed blank even with a night of correctly-dated OURA_SLEEP_PHASE events. NOOP now builds its OWN session from the ring's coarse phase codes (the polished hypnogram is SleepNet, cloud-gated, not on the wire - OURA_PROTOCOL.md 6.12.1). - OuraSleepSessionBuilder (pure): (ts, 2-bit stage) -> [SleepSession] with merged StageSegments + efficiency; splits on gaps, drops sub-minimum/all-wake runs. - analyzeDay gains providedSleepSessions: when set it REPLACES the gravity detector, so the ring's night flows through the same funnels (sleep totals + skin-temp window). nil default keeps every WHOOP caller byte-identical. - 11 tests incl. an end-to-end analyzeDay-with-no-gravity proof. Co-Authored-By: Claude Opus 4.8 --- .../StrandAnalytics/AnalyticsEngine.swift | 19 ++- .../OuraSleepSessionBuilder.swift | 110 +++++++++++++++ .../AnalyticsEngineTests.swift | 28 ++++ .../OuraSleepSessionBuilderTests.swift | 129 ++++++++++++++++++ 4 files changed, 285 insertions(+), 1 deletion(-) create mode 100644 Packages/StrandAnalytics/Sources/StrandAnalytics/OuraSleepSessionBuilder.swift create mode 100644 Packages/StrandAnalytics/Tests/StrandAnalyticsTests/OuraSleepSessionBuilderTests.swift diff --git a/Packages/StrandAnalytics/Sources/StrandAnalytics/AnalyticsEngine.swift b/Packages/StrandAnalytics/Sources/StrandAnalytics/AnalyticsEngine.swift index 431988651..2ed9977ec 100644 --- a/Packages/StrandAnalytics/Sources/StrandAnalytics/AnalyticsEngine.swift +++ b/Packages/StrandAnalytics/Sources/StrandAnalytics/AnalyticsEngine.swift @@ -316,6 +316,15 @@ public enum AnalyticsEngine { // measured night. Trace-only: never alters the DayResult. nil/default keeps // pure-function callers/tests byte-identical (still emits `measured`). sleepProvenance: SleepProvenance = .measured, + // Ring-supplied sleep sessions (Oura). `SleepStager.detectSleep` is + // gravity-driven and a ring streams no accelerometer, so it returns + // nothing for an Oura night; the caller instead builds sessions from the + // ring's OWN anchored phase timeline (OuraSleepSessionBuilder) and passes + // them here. When non-nil these REPLACE the gravity detector's output as + // the day's sessions, so the ring's night flows through the SAME funnels + // (sleep totals, the skin-temp window, rest) the WHOOP path uses. nil (the + // default) keeps every existing gravity-based caller/test byte-identical. + providedSleepSessions: [SleepSession]? = nil, // Sleep & Rest test-mode trace sink (zero-cost default nil = byte-identical). // When non-nil, the gate trace from detectSleep and the Rest sub-score line // are forwarded line-by-line. Side-effect-only; never alters the DayResult. @@ -332,11 +341,19 @@ public enum AnalyticsEngine { func tsInDay(_ ts: Int) -> Bool { (ts + tzOffsetSeconds) >= dayStartUtc && (ts + tzOffsetSeconds) < dayEndUtc } // ── Sleep detection + staging ───────────────────────────────────────── - let allSessions = SleepStager.detectSleep(hr: hr, rr: rr, resp: resp, gravity: gravity, + // A ring supplies its own sessions (built from its phase timeline); everything else derives them + // from the gravity-driven stager. `providedSleepSessions` REPLACES detection wholesale — a device + // that hands us a hypnogram has no accelerometer for the stager to work from. + let allSessions: [SleepSession] + if let providedSleepSessions { + allSessions = providedSleepSessions + } else { + allSessions = SleepStager.detectSleep(hr: hr, rr: rr, resp: resp, gravity: gravity, tzOffsetSeconds: tzOffsetSeconds, wristOff: wristOff, bandSleepState: bandSleepState, useSleepStagerV2: useSleepStagerV2, traceSink: traceSink) + } // Sessions attributed to `day` = those whose end falls on `day` (LOCAL day, #277). `day` is // the caller's local-day key; attribute by the same offset so the bucket and the key agree. let matched = allSessions.filter { tsInDay($0.end) } diff --git a/Packages/StrandAnalytics/Sources/StrandAnalytics/OuraSleepSessionBuilder.swift b/Packages/StrandAnalytics/Sources/StrandAnalytics/OuraSleepSessionBuilder.swift new file mode 100644 index 000000000..6ca7cd643 --- /dev/null +++ b/Packages/StrandAnalytics/Sources/StrandAnalytics/OuraSleepSessionBuilder.swift @@ -0,0 +1,110 @@ +import Foundation + +/// Build a NOOP sleep session from the Oura ring's OWN anchored sleep-phase timeline +/// (`OURA_SLEEP_PHASE` events, Tier-A 2-bit codes; OURA_PROTOCOL.md §6.12). +/// +/// Why this exists (OURA_PROTOCOL.md §6.12.1): the polished 4-stage hypnogram the Oura APP shows is +/// produced by SleepNet, an encrypted, cloud-key-gated PyTorch model on the phone — it is NOT on the BLE +/// wire and NOOP neither can nor does reproduce it. What DOES cross BLE is the ring's own coarse +/// per-epoch phase classification (awake/light/deep/REM). NOOP builds its OWN session from that, the +/// same honest-data stance as every other NOOP metric. +/// +/// This ALSO bridges an architectural gap: `AnalyticsEngine.analyzeDay` derives sleep from +/// `SleepStager.detectSleep`, which is **gravity-driven** — and an Oura ring streams no accelerometer, so +/// the detector returns nothing (`grav.count < 2 → []`). The sessions built here are injected into +/// `analyzeDay` as `providedSleepSessions`, taking the place the gravity detector fills for WHOOP, so the +/// ring's night flows through the SAME funnels (dailyMetric sleep totals, the skin-temp window, rest). +/// +/// Pure and platform-neutral: input is `(ts, stage)` pairs (wall-clock unix seconds + the ring's 2-bit +/// code), so this file needs no OuraProtocol dependency and is unit-tested in the fast package loop. +public enum OuraSleepSessionBuilder { + + /// The ring's 2-bit sleep-phase code (OURA_PROTOCOL.md §6.12: `0=awake, 1=light, 2=deep, 3=REM`) + /// mapped to the `StageSegment.stage` string the rest of analytics uses. Returns nil for an + /// unknown code (a corrupt/misframed value), so it is dropped rather than guessed (honest-data). + static func stageName(forPhaseCode code: Int) -> String? { + switch code { + case 0: return "wake" + case 1: return "light" + case 2: return "deep" + case 3: return "rem" + default: return nil + } + } + + /// Build sleep session(s) from the ring's anchored phase timeline. + /// + /// Each phase event marks a stage that HOLDS until the next event, so consecutive events + /// `[tsᵢ, tsᵢ₊₁)` form one `StageSegment` with `stageᵢ`; the session spans `first → last` event. + /// Adjacent same-stage segments are merged for a clean hypnogram. Efficiency = asleep / in-bed + /// (asleep = non-wake duration). A large inter-event gap splits into separate sessions (a nap vs the + /// overnight), and a session shorter than `minSessionMinutes` or with no asleep time is dropped as + /// noise. `restingHR`/`avgHRV` are left nil here — they are enriched by the caller/engine from the + /// night's HR/RR streams, not fabricated from the phase codes. + /// + /// - Parameters: + /// - phases: `(ts, stage)` pairs — wall-clock unix seconds and the ring's 2-bit phase code. Need + /// not be pre-sorted; duplicate timestamps keep the first-seen stage. + /// - minSessionMinutes: shortest span kept (default 60 — drops stray fragments; a real nap the ring + /// staged still clears this, an isolated blip does not). + /// - splitGapMinutes: an inter-event gap longer than this starts a new session (default 120). + public static func sessions(fromPhases phases: [(ts: Int, stage: Int)], + minSessionMinutes: Int = 60, + splitGapMinutes: Int = 120) -> [SleepSession] { + // Sort by time; collapse duplicate timestamps (keep first) so a doubled event can't make a + // zero-length segment. + let sorted = phases.sorted { $0.ts < $1.ts } + var events: [(ts: Int, stage: Int)] = [] + for e in sorted where e.ts != events.last?.ts { events.append(e) } + guard events.count >= 2 else { return [] } + + // Split into contiguous runs on a large gap (nap vs overnight). + let splitGap = splitGapMinutes * 60 + var runs: [[(ts: Int, stage: Int)]] = [] + var current: [(ts: Int, stage: Int)] = [events[0]] + for e in events.dropFirst() { + if e.ts - current[current.count - 1].ts > splitGap { + runs.append(current) + current = [e] + } else { + current.append(e) + } + } + runs.append(current) + + let minSpan = minSessionMinutes * 60 + var out: [SleepSession] = [] + for run in runs where run.count >= 2 { + let start = run[0].ts + let end = run[run.count - 1].ts + guard end - start >= minSpan else { continue } + + // One segment per [tsᵢ, tsᵢ₊₁); drop segments whose code is unknown (never guess a stage). + var segments: [StageSegment] = [] + for i in 0..<(run.count - 1) { + guard let name = stageName(forPhaseCode: run[i].stage) else { continue } + let seg = StageSegment(start: run[i].ts, end: run[i + 1].ts, stage: name) + // Merge with the previous segment when the stage is identical and they abut. + if var last = segments.last, last.stage == seg.stage, last.end == seg.start { + last.end = seg.end + segments[segments.count - 1] = last + } else { + segments.append(seg) + } + } + guard !segments.isEmpty else { continue } + + var asleepSeconds = 0 + for seg in segments where seg.stage != "wake" { + asleepSeconds += seg.end - seg.start + } + guard asleepSeconds > 0 else { continue } // an all-wake run is not a sleep session + let inBed = end - start + let efficiency = inBed > 0 ? min(1.0, Double(asleepSeconds) / Double(inBed)) : 0 + + out.append(SleepSession(start: start, end: end, efficiency: efficiency, + stages: segments, restingHR: nil, avgHRV: nil)) + } + return out + } +} diff --git a/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/AnalyticsEngineTests.swift b/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/AnalyticsEngineTests.swift index 4d6676c1b..56c16bafe 100644 --- a/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/AnalyticsEngineTests.swift +++ b/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/AnalyticsEngineTests.swift @@ -112,6 +112,34 @@ final class AnalyticsEngineTests: XCTestCase { XCTAssertEqual(result.cachedSleep[0].restingHr, 50) } + func testAnalyzeDayUsesProvidedSleepSessionsWithoutGravity() { + // The Oura path: a ring streams NO accelerometer, so the gravity-driven detector yields nothing. + // The caller instead builds sessions from the ring's own phase timeline and passes them via + // `providedSleepSessions`; dailyMetric's sleep fields must populate from them, gravity empty. + let day = "2021-06-15" + let dayStart = 1_623_715_200 // 2021-06-15 00:00:00 UTC + let t = dayStart + 3_600 // night starts 01:00 UTC, ends 08:00 (~7h, on `day`) + let phases: [(ts: Int, stage: Int)] = [ + (t + 0, 1), // light + (t + 3_600, 2), // deep + (t + 7_200, 3), // rem + (t + 10_800, 1), // light + (t + 25_200, 0), // wake (end marker) + ] + let sessions = OuraSleepSessionBuilder.sessions(fromPhases: phases) + XCTAssertEqual(sessions.count, 1) + + let result = AnalyticsEngine.analyzeDay( + day: day, profile: UserProfile(age: 30), providedSleepSessions: sessions) + + XCTAssertEqual(result.sleepSessions.count, 1, "the ring session flows through with no gravity") + XCTAssertNotNil(result.daily.totalSleepMin) + XCTAssertGreaterThan(result.daily.totalSleepMin!, 0) + XCTAssertNotNil(result.daily.deepMin) // stage minutes come from the ring's hypnogram + XCTAssertNotNil(result.daily.remMin) + XCTAssertNotNil(result.cachedSleep[0].stagesJSON) + } + func testAnalyzeDayColdStartRecoveryNil() { // No baselines supplied → recovery is nil (cold-start gate). let day = "2021-06-16" diff --git a/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/OuraSleepSessionBuilderTests.swift b/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/OuraSleepSessionBuilderTests.swift new file mode 100644 index 000000000..f8d17b85d --- /dev/null +++ b/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/OuraSleepSessionBuilderTests.swift @@ -0,0 +1,129 @@ +import XCTest +@testable import StrandAnalytics + +/// Tests for building a NOOP sleep session from the ring's OWN anchored phase timeline +/// (OURA_SLEEP_PHASE events), the bridge that lets an Oura night flow through the gravity-free path. +final class OuraSleepSessionBuilderTests: XCTestCase { + private let base = 1_700_000_000 + + private func build(_ phases: [(ts: Int, stage: Int)], + minSessionMinutes: Int = 60, + splitGapMinutes: Int = 120) -> [SleepSession] { + OuraSleepSessionBuilder.sessions(fromPhases: phases, + minSessionMinutes: minSessionMinutes, + splitGapMinutes: splitGapMinutes) + } + + // MARK: - Stage mapping + + func testStageCodeMapping() { + XCTAssertEqual(OuraSleepSessionBuilder.stageName(forPhaseCode: 0), "wake") + XCTAssertEqual(OuraSleepSessionBuilder.stageName(forPhaseCode: 1), "light") + XCTAssertEqual(OuraSleepSessionBuilder.stageName(forPhaseCode: 2), "deep") + XCTAssertEqual(OuraSleepSessionBuilder.stageName(forPhaseCode: 3), "rem") + XCTAssertNil(OuraSleepSessionBuilder.stageName(forPhaseCode: 7), "unknown code is never guessed") + } + + // MARK: - Core build + + func testBuildsOneSessionWithStagesAndEfficiency() { + // light 30m | wake 30m | deep 60m | rem 30m(end) -> 150m in bed, 120m asleep -> eff 0.8. + let s = build([ + (base + 0, 1), // light + (base + 1800, 0), // wake + (base + 3600, 2), // deep + (base + 7200, 3), // rem + (base + 9000, 0), // wake (end marker) + ]) + XCTAssertEqual(s.count, 1) + let night = s[0] + XCTAssertEqual(night.start, base) + XCTAssertEqual(night.end, base + 9000) + XCTAssertEqual(night.stages.map { $0.stage }, ["light", "wake", "deep", "rem"]) + XCTAssertEqual(night.stages.map { $0.end - $0.start }, [1800, 1800, 3600, 1800]) + XCTAssertEqual(night.efficiency, 0.8, accuracy: 0.0001) + XCTAssertNil(night.restingHR) // enriched downstream, never fabricated from phase codes + XCTAssertNil(night.avgHRV) + } + + func testAdjacentSameStageSegmentsAreMerged() { + // Two consecutive light epochs collapse into one [0, 3600) light segment. + let s = build([ + (base + 0, 1), // light + (base + 1800, 1), // light (merges) + (base + 3600, 2), // deep + (base + 5400, 0), // wake (end) + ]) + XCTAssertEqual(s.count, 1) + XCTAssertEqual(s[0].stages.map { $0.stage }, ["light", "deep"]) + XCTAssertEqual(s[0].stages[0].start, base) + XCTAssertEqual(s[0].stages[0].end, base + 3600) // merged 60m of light + } + + func testUnknownPhaseCodeSegmentIsDropped() { + // A misframed/unknown code (7) contributes no segment and no asleep time - never guessed. + let s = build([ + (base + 0, 1), // light + (base + 1800, 7), // unknown -> dropped + (base + 3600, 2), // deep + (base + 5400, 0), // wake (end) + ]) + XCTAssertEqual(s.count, 1) + XCTAssertEqual(s[0].stages.map { $0.stage }, ["light", "deep"]) + } + + // MARK: - Session splitting / gating + + func testLargeGapSplitsIntoSeparateSessions() { + let s = build([ + (base + 0, 1), // run A: light + (base + 3600, 0), // run A: wake (end) - 60m span + (base + 3600 + 7260, 2), // >120m gap -> run B: deep + (base + 3600 + 7260 + 3600, 0), // run B: wake (end) - 60m span + ]) + XCTAssertEqual(s.count, 2) + XCTAssertEqual(s[0].stages.first?.stage, "light") + XCTAssertEqual(s[1].stages.first?.stage, "deep") + } + + func testTooShortRunIsDropped() { + // 30m span < the 60m floor -> not a session. + XCTAssertTrue(build([(base + 0, 1), (base + 1800, 0)]).isEmpty) + } + + func testAllWakeRunIsNotASession() { + // A full-length run with zero asleep time is not sleep. + XCTAssertTrue(build([(base + 0, 0), (base + 3600, 0)]).isEmpty) + } + + func testDuplicateTimestampsCollapseKeepingFirst() { + // A doubled event at the same ts must not create a zero-length segment or shift the stage. + let s = build([ + (base + 0, 1), // light (kept) + (base + 0, 2), // duplicate ts -> ignored + (base + 3600, 0), // wake (end) + ]) + XCTAssertEqual(s.count, 1) + XCTAssertEqual(s[0].stages.map { $0.stage }, ["light"]) + } + + func testFewerThanTwoEventsYieldsNoSession() { + XCTAssertTrue(build([]).isEmpty) + XCTAssertTrue(build([(base, 1)]).isEmpty) + } + + func testUnsortedInputIsHandled() { + // Same night as the core test but shuffled - result must be identical (sorted internally). + let s = build([ + (base + 9000, 0), + (base + 3600, 2), + (base + 0, 1), + (base + 7200, 3), + (base + 1800, 0), + ]) + XCTAssertEqual(s.count, 1) + XCTAssertEqual(s[0].start, base) + XCTAssertEqual(s[0].end, base + 9000) + XCTAssertEqual(s[0].efficiency, 0.8, accuracy: 0.0001) + } +} From ae554b4ee3a8949acb9eefe71ea9999077324788 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 22:32:16 +0200 Subject: [PATCH 10/21] Docs: digest open_oura crates+tools into OURA_PROTOCOL.md (section 9) Deep-dived Th0rgal/open_oura (Rust workspace RE'd from libringeventparser.so + the Android app) and folded the findings in as section 9 (leads to fixture-validate, per the doc's trust rule): - canonical connect->sync sequence + the "no RData 0x03 on a normal pull" rule; - authoritative opcode / 0x2F sub-op set; cursor = deciseconds; legacy vs extended; - feature-gating mechanism (server flags default false client-side) explaining the SpO2 / real_steps OFF defaults; - undocumented event tags (0x62 respiration spot, 0x8B spo2_r_pi, 0x71/0x6E IBI, 0x59/0x56/0x6C/0x74/0x84/0x86, 0x87/0x88 backend-gated, 0x80 gen conflict); - RData raw sampler (0x03) + the raw-accelerometer lead; batched timestamping; - reference SpO2 quadratic + skin-temp algorithm. Flags two actions in section 8: the SyncTime (0x12) send-layout mismatch and the skinTempFunnel 300-sample floor vs open_oura's ~120. Co-Authored-By: Claude Opus 4.8 --- docs/OURA_PROTOCOL.md | 58 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/docs/OURA_PROTOCOL.md b/docs/OURA_PROTOCOL.md index 61d44abfc..de66f6b0a 100644 --- a/docs/OURA_PROTOCOL.md +++ b/docs/OURA_PROTOCOL.md @@ -13,6 +13,9 @@ - **[relue]** - relue/oura_ring_reverse `docs/.../heartbeat_replication_guide.md` and `heartbeat_complete_flow.md` (no-license; Ring 3 live-HR). - **[oura-rs]** - Th0rgal/open_oura `crates/oura-protocol/src/events.rs` (no-license Rust clean-room decoder; facts cited only, no code copied). Its event tags marked `"_status": "unvalidated"` are treated the same as our Tier B - plausible, not ground-truth-confirmed. - **[sleepnet]** - Th0rgal/open_oura `docs/algorithms/sleepnet.md` (no-license; describes Oura's on-phone sleep-staging model + its encryption, NOT a BLE protocol layout). See §6.12.1. +- **[oura-proto]** - Th0rgal/open_oura `crates/oura-protocol/src/{protocol,events,auth}.rs` (no-license clean-room Rust decoder). Opcodes, framing, event-body layouts. Treated as Tier-B leads (fixture-validate), same as [oura-rs]; see §9. +- **[oura-link]** - Th0rgal/open_oura `crates/oura-link/src/client.rs` (no-license; the BLE connect→sync state machine). See §9.1. +- **[oura-sync]** - Th0rgal/open_oura `docs/{sync-orchestration,data-recovery-map,native-decoder,ring-features}.md` + `crates/oura-analysis/src/ported/*.rs` (no-license). Sync recipe, recoverable-data map, native-lib decode provenance, reference scoring. See §9. > **CONFLICT NOTE (resolution rule):** The relue archive file `event_data_definition.md` describes events as **protobuf varint** records (e.g. `0x55` SLEEP_HR with field tags). This contradicts the **byte-for-byte verified TLV framing** in [open_ring] and [ringverse]. The TLV/bit-packed model from [open_ring]/[ringverse] is authoritative for our decoders; the protobuf description is treated as unverified/likely AI-fabricated and is NOT used. Where a layout is only attested by a single no-license, AI-generated doc, it is marked **(UNVERIFIED)** and our decoder must gate it behind a fixture test before trusting it. @@ -457,3 +460,58 @@ NOOP-derived hypnogram as "Oura sleep stages": it is NOOP's own, from the ring's - Resolve the `0x0D` battery percent-vs-voltage offset per generation via captured fixtures (§6.10). - Validate all Tier-B sleep/activity/step layouts against real captures before enabling in scoring. - Confirm live-HR `0x02` path on actual Gen-4/Gen-5 hardware (only Gen-3 is verified in the corpus). +- **⚠ SyncTime (`0x12`) send layout mismatch (§9.2).** [oura-proto] gives `12 09 `; NOOP's `OuraCommands.syncTime` emits a different 24-bit `unixSeconds/256` + `0xF6`-trailer body. Verify against a capture and align — a malformed SyncTime may be setting the ring clock wrongly or being ignored (the ring's own RTC currently still yields a ~correct `0x42` anchor, masking it, but this is fragile). +- **Reconsider the `skinTempFunnel` sample floor for Oura nights (§9.10).** [oura-sync]'s temperature algorithm accepts a night at ≥ 4 valid 30-sample windows (~120 samples); NOOP requires 300 kept samples/night — likely too strict for a ring (fewer temp samples than a WHOOP strap). + +--- + +## 9. open_oura clean-room digest (crates + tools, 2026-07-08) + +Digest of the `Th0rgal/open_oura` Rust workspace + Python tools ([oura-proto], [oura-link], [oura-sync], plus [oura-rs] `events.rs`). open_oura reverse-engineered the event bodies from Oura's own native library `libringeventparser.so` (ARM64) and the Android app. **Trust rule (unchanged, §1):** every layout below is a **lead to fixture-validate**, NOT ground truth, EXCEPT where it merely confirms an item NOOP has already verified. Nothing here changes a decoder without a real capture. + +### 9.1 Canonical connect → sync sequence [oura-link, oura-sync] +The client state machine runs, in order: **1** connect+bond, subscribe notify · **2** authenticate (nonce → AES → `Authenticate`) · **3** **GetCapabilities** (`2f 02 01 ` → resp `ext 0x02`; decides **extended-vs-legacy** event path) · **4** app-level auth · **5** **SyncTime** (`0x12`, write phone UTC) · **6** SetNotification (`0x1c`) · **7** battery/product/firmware reads · **8** conditional feature `SetFeatureMode` enables · **9** **SYNC_EVENTS** (history drain) · **10** SYNC_R_DATA (only if `r_data_autosync`). +- **Hard rule: do NOT issue any RData (`0x03`) for a normal pull** (§9.7). +- **NOOP gap:** NOOP skips step 3 (**GetCapabilities**) entirely — it never negotiates extended-vs-legacy, so it always uses the legacy `0x10` drain. Sending SyncTime (step 5) is the fix already landed on `oura-*` branches; GetCapabilities remains an untried lead for the `0x43`-debug-heavy / thin-structured stream. + +### 9.2 SyncTime (`0x12`) exact layout — ⚠ NOOP DISCREPANCY [oura-proto, oura-link] +Authoritative build: `12 09 `. Unit is **seconds** (`req_sync_time(now.as_secs(), 0)`), 8-byte little-endian, one signed tz byte. This is the SEND side; §6.11 covers decoding the ring's `0x42` reply. +- **NOOP currently emits a DIFFERENT body**: `12 09 00 00 00 00 F6` — a 24-bit coarse time (÷256 ≈ 4.3-min resolution) with a trailing `0xF6` (resembling the `0x85` RTC-beacon trailer), NOT 8-byte LE seconds. **Almost certainly wrong.** It may set the ring clock incorrectly or be silently rejected; the anchor still lands today only because the ring's own RTC drives a ~correct `0x42` regardless. Fixture-verify and align (see §8). Cross-ref §5.5, §6.11. + +### 9.3 Command + `0x2F` sub-op set (authoritative) [oura-proto] +Outer opcodes: `0x03` RData · `0x06` realtime measurement · `0x08` firmware info · `0x0C` battery request (→ `0x0D` reply) · `0x10` GetEvent · `0x12` SyncTime · `0x18` product info · `0x1C` notification flags · `0x24` set-auth-key · `0x28` sleep-analysis check · `0x2F` extended. +`0x2F` sub-ops: `0x01` GetCapabilities · `0x20` FeatureStatus · `0x22` SetFeatureMode · **`0x24` GetFeatureLatestValues** (poll a capability's last value — e.g. last IBI/SpO2 — WITHOUT streaming) · `0x26` SetFeatureSubscription · `0x27` subscription-resp · `0x2B` AuthNonce (→ `0x2C`) · `0x2D` Authenticate (→ `0x2E`). +Feature **modes**: `0x00` off · `0x01` automatic · `0x02` requested · `0x03` connected-live. (Matches §7.1; `0x24` GetFeatureLatestValues is the notable addition — a cheap poll path NOOP does not use.) + +### 9.4 History fetch — cursor unit + legacy/extended [oura-proto, oura-link, oura-sync] +`GetEvent 0x10`: `10 09 `. **The cursor is in deciseconds = the ring's 100 ms tick** — this pins §5.5's tick and confirms NOOP's cursor is a ring-timestamp in 100 ms units. Response `0x11` → `EventBatchSummary { bytes_left }`; loop until `bytes_left == 0`, advancing cursor to `max_event_ts + 1`, persisting after each batch. **Extended path** (`ExtGetEvent`, via `0x2F`): `start_ms = cursor × 100`, `max = 65535`, NORMAL buffer — chosen only when GetCapabilities (§9.1 step 3) reports it. NOOP uses the legacy `0x10` path. + +### 9.5 Feature-gating mechanism [oura-sync ring-features] +**All `FeatureDefinitions.*` server flags default FALSE client-side; the effective value arrives in the per-user `ClientConfiguration` Oura's cloud delivers.** So a NOOP-style cloudless client sees the OFF defaults: **SpO2** (`health/spo2`) and **REAL_STEPS** (`activity/real_steps`, feature `0x0B`) are OFF → this is the mechanism behind §7.1's "SpO2 confirmed OFF" and the empty Steps card, not a decode gap. `RESEARCH_DATA` (`0x01`) and `RAW_DATA_SAMPLER` (`0x12`) are **entitlement-locked firmware-side** (a `SetFeatureMode` is a no-op / `NOT_AVAILABLE`). `DAYTIME_HR` (`0x02`) is always-on Gen3+. Confirms + explains §7.1. + +### 9.6 Event tags not yet in §6 (leads — fixture-gate) [oura-rs, oura-sync] +- **IBI family:** `0x44` ibi, `0x60` ibi_and_amplitude (14 B, 6 IBIs + PPG amps, bit-packed), `0x6E` spo2_ibi, **`0x71` green_ibi_and_amplitude**. HR = `60000 / ibi_ms`, plausible 300–2000 ms. +- **`0x62` on_demand_meas** — a spot measurement bundling **breathing rate** + HR/HRV/temp. A NEW respiration source (NOOP has no respiration from Oura today). +- **`0x5D` HRV** — [oura-rs] decodes as pairs `(u8 avg HR bpm, u8 RMSSD ms)` per 5 min. This is a **lead to promote** NOOP's `0x5D` (§6.9 keeps `b1`/`b2` raw, scale unpinned): `b2` may be RMSSD in **ms directly**. Validate before minting `rmssd_ms`. +- **`0x8B` spo2_r_pi_event** — the REAL SpO2 source (Gen4/Cooper): header + 3-byte samples `(R-ratio: u16 BE / 16384, perfusion index: u8/255 × 0.05)`; feeds the quadratic in §9.10. A Gen-3 with SpO2 gated OFF never emits `0x8B` — consistent with NOOP's `[85,100]` persist gate dropping the misframed `0x6F/0x77` garbage. +- **Sleep:** `0x4B`/`0x4E`/`0x5A` all carry the 2-bit hypnogram (`0 wake,1 light,2 deep,3 REM`); summaries `0x49/0x4C/0x4F/0x58` carry bedtime / stage durations / lowest-HR / contributors (all UNVERIFIED — need a capture *after* a processed sleep period). Each daily sleep doc carries a `sleep_algorithm_version`. +- **Misc:** `0x59` eda (u16 LE @5 min), `0x56` alert (1 byte), `0x6C` feature_session (feature_id/status/±u16 value), `0x74` ehr_acm_intensity (u16 LE ×7), `0x61` debug_data (ASCII, or binary subtypes `0x11` charging_time u32 / `0x24` battery %+u16 mV), `0x84` ambient (i16 @5 min), `0x86` aohr (bpm/quality @1920 ms), `0x82`/`0x83` scan telemetry. +- **`0x80` — generation conflict:** [oura-native] documents `0x80` as **green_ibi_quality (Ring 5)**: `ibi_ms = (b1 & 7) | (b0 << 3)`, `quality = (b1>>3)&3`. NOOP/[ringverse] treat `0x80` as an IBI amplitude tag. **Verify which layout applies per generation before trusting `0x80` on Ring 5.** +- **`0x87`/`0x88` atlas metadata / raw_bioz — BACKEND-GATED** (need a cloud-delivered `FeatureDefinition`, like SleepNet §6.12.1); inaccessible to an independent client. + +### 9.7 RData raw sampler (`0x03`) — bulk flash sensor download [oura-proto, oura-sync] +An opt-in research path (`r_data_autosync` pref, default off; feature `0x12` entitlement-locked). Sub-ops: `0x01` get-page, `0x02` configure (`03 02 `), `0x03` stop, `0x04` clear, `0x05` state. `DataType`: PPG 250/125/50 Hz · ACM 8g/2g/4g @50/10 Hz · Gyro 2000/500/125 dps @50/10 Hz · Temp 1 min/10 s/10 Hz. +- **It does NOT self-stop** — the caller MUST issue `stop` + `clear` or the ring keeps sampling to flash. And **never issue RData on a normal pull** (§9.1). +- **Sleep-relevant lead:** RData `ACM` is the ONLY way to pull **raw accelerometer** off the ring. NOOP's sleep pipeline is gravity-driven and Oura yields no gravity, which is exactly why NOOP builds sessions from the ring's own phase tags instead (the honest path, `OuraSleepSessionBuilder`). RData ACM could in principle feed the gravity stager, but it is heavy (flash, entitlement-locked, teardown-critical) — **not a v1 path.** + +### 9.8 Batched-sample timestamping + intervals [oura-native] +Confirms NOOP's backward-walk: a batched event packs N samples under one event-time; per-sample time steps BACKWARD `t = utc_ms − (n−1)×interval`. Sampling intervals: **sleep-temp 30 s · HRV & ambient-temp 5 min · measurement-quality 3 min · SpO2 1 s · AOHR 1920 ms.** + +### 9.9 Cloud-only vs on-device (honest-data alignment) [oura-sync] +- **On-wire / on-device** (NOOP may consume): raw HR/IBI/PPG, HRV (`0x5D`), SpO2 samples (when gated on), skin temp + its on-device baseline, MET bins + step counts, coarse sleep-phase tags, respiration spot (`0x62`). +- **Cloud-ONLY, no local path:** Readiness / Sleep / Activity **scores** (0–100), workout auto-classification, SpO2 OVI/BDI indices, and the SleepNet hypnogram (§6.12.1). +- Reinforces the invariant: NOOP computes its OWN scores from the raw + coarse signals and never needs Oura's cloud scores. + +### 9.10 Reference algorithms — compare to NOOP funnels [oura-sync ported] +- **SpO2:** `SpO2% = a + b·r + c·r²` (quadratic on the `0x8B` R-ratio); coefficients are delivered **by the ring** (per-device), raw result clamped `[0,120]` (the `run_spo2.py` display tool further clamps `[85,100]`). Aligns NOOP's `[85,100]` SpO2 persist gate. +- **Skin temp:** 7-sample sliding median → consecutive 30-sample windows → keep a window only if `(max − min) < 2.50 °C` and non-zero → **nightly value = min of the window maxima** → needs **≥ 4 valid windows** (~120 samples). Deviation = nightly − personal-baseline mean. **Lead:** NOOP's `skinTempFunnel` demands 300 kept samples/night — ~2.5× open_oura's floor; likely too strict for a ring (§8). From ba3a26e5e45e2eacaac24007aad9c92a9a4e8837 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 11:27:59 +0200 Subject: [PATCH 11/21] Surface Oura activity/steps by promoting the Tier-B MET stream. Threaded a live body-weight provider from the profile down to the estimator, matching the codebase's existing closure-injection style: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AppModel.profile.weightKg → SourceCoordinator(bodyweightKg:) [new param] → OuraLiveSource(bodyweightKg:) [new param] → OuraActivityEstimator.summarize(bodyweightKg: bodyweightKg()) - Read live (a closure, not a captured value), so if you edit your weight in Settings it takes effect on the next flush. - Removed the activityBodyweightKg = 75.0 constant entirely. - Default fallback is 70 kg (only used if profile weight is unset/≤0) — deliberately matching the production calorie path's own 70 kg fallback, so there's one consistent default across the app rather than introducing a second (75). --- .../OuraProtocol/OuraActivityEstimator.swift | 114 ++++++++++++++++++ .../OuraActivityEstimatorTests.swift | 77 ++++++++++++ Strand/App/AppModel.swift | 5 +- Strand/BLE/OuraLiveSource.swift | 87 +++++++++++-- Strand/BLE/SourceCoordinator.swift | 11 +- 5 files changed, 284 insertions(+), 10 deletions(-) create mode 100644 Packages/OuraProtocol/Sources/OuraProtocol/OuraActivityEstimator.swift create mode 100644 Packages/OuraProtocol/Tests/OuraProtocolTests/OuraActivityEstimatorTests.swift diff --git a/Packages/OuraProtocol/Sources/OuraProtocol/OuraActivityEstimator.swift b/Packages/OuraProtocol/Sources/OuraProtocol/OuraActivityEstimator.swift new file mode 100644 index 000000000..c7e4f3cc9 --- /dev/null +++ b/Packages/OuraProtocol/Sources/OuraProtocol/OuraActivityEstimator.swift @@ -0,0 +1,114 @@ +import Foundation + +// MARK: - Phase-1 Oura activity estimate (Tier-B INVESTIGATION — never persisted, never scored) + +/// PHASE 1 of promoting the Oura `0x50 activity_info` (MET) stream out of Tier B. This pure roll-up +/// folds decoded MET samples into a per-local-day estimate that `OuraLiveSource` LOGS (never stores) +/// so the numbers can be eyeballed against the WHOOP band's own steps / active-kcal on the following +/// day (the user wears both). It exists ONLY to decide whether 0x50 is worth promoting to Tier A; it +/// never touches `OuraStreamMapping` / `Streams` / scoring, and it never mints an honest step count +/// (MET ≠ steps — the `stepProxy` below is explicitly a labelled proxy, not a stored value). +/// +/// HONEST UNCERTAINTY: the MET sample *spacing* is undocumented (docs/OURA_PROTOCOL.md §6.13 marks the +/// layout "UNVERIFIED - partial"). So the cadence-independent facts (sample counts per MET band, mean / +/// max MET) are reported as-is, while every minute / kcal / step figure is derived under an EXPLICIT +/// `assumedIntervalSec` and labelled as such. The WHOOP cross-check is precisely what calibrates that +/// unknown: if Oura active-minutes at the assumed interval read ~2× WHOOP, the true spacing is ~2×. +public struct OuraActivitySample: Equatable, Sendable { + /// Wall-clock unix seconds. In Phase 1 this is the live *arrival* time (history-backlog anchoring is + /// future work, and blocked today by the history-fetch cursor regression — see CLAUDE.md). + public let ts: Int + public let met: Double + public init(ts: Int, met: Double) { self.ts = ts; self.met = met } +} + +/// The per-day roll-up. Cadence-independent fields (`*Samples`, `meanMet`, `maxMet`) carry no interval +/// assumption; the `active*` / `est*` / `stepProxy` fields do (see `assumedIntervalSec`). +public struct OuraActivityEstimate: Equatable, Sendable { + public let day: String + public let sampleCount: Int + public let firstTs: Int? + public let lastTs: Int? + public let meanMet: Double + public let maxMet: Double + + // Cadence-INDEPENDENT: how many MET samples fell in each standard activity band. + public let sedentarySamples: Int // met < 1.5 + public let lightSamples: Int // 1.5 <= met < 3 + public let moderateSamples: Int // 3 <= met < 6 + public let vigorousSamples: Int // met >= 6 + + // Cadence-DEPENDENT: everything below scales linearly with `assumedIntervalSec`. + public let assumedIntervalSec: Double + public let activeMinutes: Double // (light+moderate+vigorous) samples * interval / 60 + public let estActiveKcal: Double // Σ max(met-1,0) * bodyweightKg * interval/3600 (net-of-resting) + public let stepProxy: Int // PROXY ONLY: activeMinutes * stepsPerActiveMin (never stored) + + /// A single-line, grep-friendly log string. `final == true` means the local day rolled over (a + /// complete day for whatever coverage we got); `false` is a running "so far" snapshot. + public func logLine(final: Bool) -> String { + let tag = final ? "FINAL" : "so far" + func f(_ v: Double, _ p: Int = 1) -> String { String(format: "%.\(p)f", v) } + return "activity estimate \(tag) day=\(day) samples=\(sampleCount) " + + "meanMET=\(f(meanMet, 2)) maxMET=\(f(maxMet, 2)) " + + "bands[sed/light/mod/vig]=\(sedentarySamples)/\(lightSamples)/\(moderateSamples)/\(vigorousSamples) " + + "activeMin≈\(f(activeMinutes)) estKcal≈\(f(estActiveKcal, 0)) stepProxy≈\(stepProxy) " + + "[assumed \(f(assumedIntervalSec, 0))s/sample — PROXY, not stored]" + } +} + +/// Pure, deterministic summariser. No I/O, no clock, no persistence — fully unit-testable. +public enum OuraActivityEstimator { + // Standard compendium-of-physical-activities MET band edges. + public static let lightFloor = 1.5 // below this = sedentary + public static let moderateFloor = 3.0 + public static let vigorousFloor = 6.0 + + /// Fold MET samples for a single local `day` into an estimate. `bodyweightKg` feeds the (secondary) + /// active-kcal figure; `assumedIntervalSec` is the UNVERIFIED per-sample spacing that all time-based + /// outputs scale with; `stepsPerActiveMin` is the labelled step-proxy cadence (never an honest count). + public static func summarize(_ samples: [OuraActivitySample], + day: String, + bodyweightKg: Double, + assumedIntervalSec: Double, + stepsPerActiveMin: Double) -> OuraActivityEstimate { + guard !samples.isEmpty else { + return OuraActivityEstimate(day: day, sampleCount: 0, firstTs: nil, lastTs: nil, + meanMet: 0, maxMet: 0, + sedentarySamples: 0, lightSamples: 0, moderateSamples: 0, + vigorousSamples: 0, assumedIntervalSec: assumedIntervalSec, + activeMinutes: 0, estActiveKcal: 0, stepProxy: 0) + } + var sum = 0.0, maxMet = 0.0 + var sed = 0, light = 0, mod = 0, vig = 0 + var activeKcal = 0.0 + let hours = assumedIntervalSec / 3600.0 + for s in samples { + sum += s.met + if s.met > maxMet { maxMet = s.met } + switch s.met { + case .. OuraActivitySample { + OuraActivitySample(ts: ts, met: met) + } + + func testEmptyIsAllZero() { + let e = OuraActivityEstimator.summarize([], day: day, bodyweightKg: 75, + assumedIntervalSec: 30, stepsPerActiveMin: 100) + XCTAssertEqual(e.sampleCount, 0) + XCTAssertEqual(e.activeMinutes, 0) + XCTAssertEqual(e.estActiveKcal, 0) + XCTAssertEqual(e.stepProxy, 0) + XCTAssertNil(e.firstTs) + } + + func testBandClassificationEdges() { + // Exact edges land in the HIGHER band (half-open [floor, ceil)). + let samples = [0.9, 1.5, 2.9, 3.0, 5.9, 6.0, 7.4].map { sample($0) } + let e = OuraActivityEstimator.summarize(samples, day: day, bodyweightKg: 75, + assumedIntervalSec: 30, stepsPerActiveMin: 100) + XCTAssertEqual(e.sedentarySamples, 1) // 0.9 + XCTAssertEqual(e.lightSamples, 2) // 1.5, 2.9 + XCTAssertEqual(e.moderateSamples, 2) // 3.0, 5.9 + XCTAssertEqual(e.vigorousSamples, 2) // 6.0, 7.4 + XCTAssertEqual(e.maxMet, 7.4, accuracy: 1e-9) + } + + func testActiveMinutesScaleWithAssumedInterval() { + // 4 active samples (>=1.5 MET) + 1 sedentary. At 30 s/sample → 4*30/60 = 2.0 active min. + let samples = [0.9, 2.0, 2.0, 4.0, 7.0].map { sample($0) } + let e30 = OuraActivityEstimator.summarize(samples, day: day, bodyweightKg: 75, + assumedIntervalSec: 30, stepsPerActiveMin: 100) + XCTAssertEqual(e30.activeMinutes, 2.0, accuracy: 1e-9) + XCTAssertEqual(e30.stepProxy, 200) // 2.0 * 100 + + // Doubling the assumed cadence doubles every time-based figure (the WHOOP-calibration lever). + let e60 = OuraActivityEstimator.summarize(samples, day: day, bodyweightKg: 75, + assumedIntervalSec: 60, stepsPerActiveMin: 100) + XCTAssertEqual(e60.activeMinutes, 4.0, accuracy: 1e-9) + XCTAssertEqual(e60.stepProxy, 400) + } + + func testActiveKcalIsNetOfResting() { + // One 5.0-MET sample for 3600 s (interval) at 70 kg → (5-1)*70*1.0 = 280 kcal. A resting 1.0-MET + // sample adds nothing (net-of-resting), and a sub-resting 0.5 never goes negative. + let samples = [sample(5.0), sample(1.0), sample(0.5)] + let e = OuraActivityEstimator.summarize(samples, day: day, bodyweightKg: 70, + assumedIntervalSec: 3600, stepsPerActiveMin: 100) + XCTAssertEqual(e.estActiveKcal, 280, accuracy: 1e-6) + } + + func testMeanAndCoverageTimestamps() { + let samples = [sample(1.0, ts: 100), sample(3.0, ts: 500), sample(2.0, ts: 300)] + let e = OuraActivityEstimator.summarize(samples, day: day, bodyweightKg: 75, + assumedIntervalSec: 30, stepsPerActiveMin: 100) + XCTAssertEqual(e.meanMet, 2.0, accuracy: 1e-9) + XCTAssertEqual(e.firstTs, 100) + XCTAssertEqual(e.lastTs, 500) + } + + func testLogLineIsGreppableAndLabelled() { + let e = OuraActivityEstimator.summarize([sample(4.0)], day: day, bodyweightKg: 75, + assumedIntervalSec: 30, stepsPerActiveMin: 100) + let line = e.logLine(final: true) + XCTAssertTrue(line.contains("activity estimate FINAL")) + XCTAssertTrue(line.contains("day=\(day)")) + XCTAssertTrue(line.contains("PROXY, not stored")) + } +} diff --git a/Strand/App/AppModel.swift b/Strand/App/AppModel.swift index a8ce24fc4..2d45cb033 100644 --- a/Strand/App/AppModel.swift +++ b/Strand/App/AppModel.swift @@ -425,7 +425,10 @@ final class AppModel: ObservableObject { // path. Timestamp matches BLEManager.log()'s "HH:mm:ss" so the lines read consistently. straplog: { [weak self] line in self?.live.append(log: "[\(AppModel.logTimeFormatter.string(from: Date()))] \(line)") - }) + }, + // Live user body weight for the Oura source's Phase-1 activity-estimate kcal — the SAME + // `Profile.weightKg` the production calorie path uses, never a hardcoded value. + bodyweightKg: { [weak self] in self?.profile.weightKg ?? 70.0 }) coordinator.start() self.deviceRegistry = registry self.sourceCoordinator = coordinator diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index 1eba36b13..51f1d9b49 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -115,6 +115,10 @@ public final class OuraLiveSource: NSObject, ObservableObject { // MARK: - Dependencies (injected - no BLEManager / WhoopBleClient reference) private let live: LiveState + /// Provider for the user's body weight (kg), read live so a profile edit takes effect next flush. + /// Wired to `Profile.weightKg` at the composition root (the same source the production calorie path + /// uses); Phase-1 Oura activity-estimate kcal only. + private let bodyweightKg: () -> Double private let deviceId: String private let persist: (Streams) -> Void private let log: (String) -> Void @@ -158,6 +162,27 @@ public final class OuraLiveSource: NSObject, ObservableObject { /// ONLY (see the `allowTierB: true` comment at driver construction) - the log is how we collect raw /// captures to validate these layouts; nothing here ever persists or scores. Reset on stop/disconnect. private var loggedTierBKinds: Set = [] + + // MARK: Phase-1 Oura activity estimate (Tier-B INVESTIGATION — logged, NEVER persisted/scored) + // Accumulates decoded 0x50 MET samples for the CURRENT local day and logs a per-day estimate + // (OuraActivityEstimator) so the numbers can be manually cross-checked against the WHOOP band's + // steps / active-kcal the following day. This buffer intentionally SURVIVES reconnects within the + // same local day (so a day accumulates across BLE sessions); it is flushed+cleared on day rollover + // and logged as a running "so far" snapshot on disconnect and on a throttle while streaming. See + // docs/OURA_PROTOCOL.md §6.13 and CLAUDE.md ("promoting steps out of Tier-B"). + private var activityDayKey: String? + private var activitySamples: [OuraActivitySample] = [] + private var lastActivityLogAt: Date = .distantPast + /// Phase-1 constants. The body weight for the (secondary) kcal figure comes from the live user + /// profile via the injected `bodyweightKg` provider — the SAME `Profile.weightKg` the production + /// calorie path uses (Calories.estimateDayCalories), never a hardcoded value. `assumedIntervalSec` + /// is the UNVERIFIED MET spacing every time-based figure scales with (the WHOOP compare calibrates + /// it); `stepsPerActiveMin` is a labelled PROXY cadence, never an honest step count. + private static let activityAssumedIntervalSec = 30.0 + private static let activityStepsPerActiveMin = 100.0 + /// Throttle for the running "so far" activity log while streaming (seconds). + private static let activityLogThrottleSec: TimeInterval = 300 + /// History-fetched events decoded BEFORE a ring-time -> UTC anchor exists this session, held here /// (with their own ring timestamp) until the anchor lands (`drainPendingAnchorEvents`), so they get /// their real historical time instead of a premature wall-clock guess. The ring's 0x42 time-sync can @@ -334,7 +359,8 @@ public final class OuraLiveSource: NSObject, ObservableObject { log: @escaping (String) -> Void = { _ in }, onBattery: @escaping (Int) -> Void = { _ in }, feedsLive: Bool = true, - adoptIntent: Bool = false) { + adoptIntent: Bool = false, + bodyweightKg: @escaping () -> Double = { 70.0 }) { self.live = live self.deviceId = deviceId self.ringGen = ringGen @@ -344,6 +370,7 @@ public final class OuraLiveSource: NSObject, ObservableObject { self.onBattery = onBattery self.feedsLive = feedsLive self.adoptIntent = adoptIntent + self.bodyweightKg = bodyweightKg super.init() // Dedicated queue-less central -> callbacks arrive on the main queue, matching @MainActor. self.central = CBCentralManager(delegate: self, queue: nil) @@ -603,6 +630,7 @@ public final class OuraLiveSource: NSObject, ObservableObject { private func ingest(_ events: [OuraEvent]) { guard !events.isEmpty, let driver else { return } let now = Int(Date().timeIntervalSince1970) + rolloverActivityDayIfNeeded(now: now) for e in events { switch e { case .hr(let hr): @@ -687,12 +715,19 @@ public final class OuraLiveSource: NSObject, ObservableObject { } case .activityInfo(let info): - // INVESTIGATION ONLY (0x50 activity/MET, Tier B - a plausible third-party formula, NOT - // ground-truth-validated; see OuraActivityInfo). Logged with the DECODED state/MET values - // every time (not once-per-kind): this is the tag under active plausibility evaluation, so - // every real capture is evidence. Never persisted, never scored, and NEVER converted into - // steps (MET is not a step count; OuraStreamMapping drops .activityInfo unconditionally). - log("Oura: activity (Tier-B) state=\(info.state) met=\(info.met)") + // PHASE 1 (Tier-B INVESTIGATION): 0x50 activity/MET is a plausible third-party formula, + // NOT ground-truth-validated (see OuraActivityInfo). Instead of spamming one line per + // record, accumulate the decoded MET samples into a per-local-day estimate and log a + // rolled-up "so far" snapshot on a throttle (final on day rollover / disconnect). Still + // never persisted, never scored, and NEVER converted into an honest step count — the + // estimate's stepProxy is explicitly labelled, and OuraStreamMapping still drops + // .activityInfo unconditionally. This is how we decide whether 0x50 earns Tier A. + if activityDayKey == nil { activityDayKey = dayKeyLocal(now) } + for m in info.met { activitySamples.append(OuraActivitySample(ts: now, met: m)) } + if Date().timeIntervalSince(lastActivityLogAt) >= Self.activityLogThrottleSec { + logActivityEstimate(final: false) + lastActivityLogAt = Date() + } case .debugText(let ringTimestamp, let text): // 0x43 debug_event: the ring's OWN ASCII firmware diagnostics (state strings), one per TLV @@ -718,6 +753,41 @@ public final class OuraLiveSource: NSObject, ObservableObject { } } + // MARK: - Phase-1 activity estimate (Tier-B INVESTIGATION — logged, never persisted) + + /// Local-day key ("YYYY-MM-DD", device time zone) for a unix-seconds timestamp. Matches the + /// `dailyMetric.day` convention so a logged Oura estimate lines up with the WHOOP day it's compared to. + private func dayKeyLocal(_ ts: Int) -> String { + let f = DateFormatter() + f.calendar = Calendar(identifier: .gregorian) + f.timeZone = .current + f.dateFormat = "yyyy-MM-dd" + return f.string(from: Date(timeIntervalSince1970: TimeInterval(ts))) + } + + /// If the local day changed since the accumulator started, log the completed day as FINAL and reset + /// the buffer for the new day. Called at the top of every ingest batch (all events share `now`). + private func rolloverActivityDayIfNeeded(now: Int) { + let today = dayKeyLocal(now) + guard let prior = activityDayKey, prior != today else { return } + logActivityEstimate(final: true) + activitySamples.removeAll(keepingCapacity: true) + activityDayKey = today + lastActivityLogAt = .distantPast + } + + /// Summarise the accumulated MET samples for the current day and log the one-line estimate. `final` + /// marks a day-rollover (complete for whatever coverage we got) vs a running "so far" snapshot. + private func logActivityEstimate(final: Bool) { + guard let day = activityDayKey, !activitySamples.isEmpty else { return } + let est = OuraActivityEstimator.summarize( + activitySamples, day: day, + bodyweightKg: bodyweightKg(), + assumedIntervalSec: Self.activityAssumedIntervalSec, + stepsPerActiveMin: Self.activityStepsPerActiveMin) + log("Oura: " + est.logLine(final: final)) + } + // MARK: - Debug-text (0x43) filter — Phase-1 investigation logging /// Prefixes of the ring's high-rate firmware chatter to DROP from the strap log (case-sensitive, so @@ -919,6 +989,9 @@ extension OuraLiveSource: @preconcurrency CBCentralManagerDelegate { } else { log("Oura: disconnected (clean)") } + // Phase-1: log the day's running activity estimate on drop (buffer is NOT cleared here — it keeps + // accumulating across reconnects and is finalised on day rollover). + logActivityEstimate(final: false) stopReengageTimer() stopHistoryFetchTimer() // Drain BEFORE driver.stop() clears its anchor (same reasoning as stop()). diff --git a/Strand/BLE/SourceCoordinator.swift b/Strand/BLE/SourceCoordinator.swift index 16d977943..e52fb491a 100644 --- a/Strand/BLE/SourceCoordinator.swift +++ b/Strand/BLE/SourceCoordinator.swift @@ -56,6 +56,10 @@ final class SourceCoordinator: ObservableObject { /// previously invisible). Passed straight into `StandardHRSource`. Defaults to a no-op so existing /// call sites (and tests) compile unchanged. private let straplog: (String) -> Void + /// User body weight (kg) provider, wired to `Profile.weightKg` at the composition root — the SAME + /// source the production calorie path uses. Read live so a profile edit takes effect on the next + /// flush. Used only by the Oura source's Phase-1 activity-estimate kcal figure. + private let bodyweightKg: () -> Double // MARK: - State @@ -119,7 +123,8 @@ final class SourceCoordinator: ObservableObject { setWhoopPreferredPeripheral: @escaping (String?) -> Void, setWhoopActiveDeviceId: @escaping (String) -> Void, connectedPeripheralUUID: AnyPublisher, - straplog: @escaping (String) -> Void = { _ in }) { + straplog: @escaping (String) -> Void = { _ in }, + bodyweightKg: @escaping () -> Double = { 70.0 }) { self.registry = registry self.live = live self.storeHandle = storeHandle @@ -129,6 +134,7 @@ final class SourceCoordinator: ObservableObject { self.setWhoopActiveDeviceId = setWhoopActiveDeviceId self.connectedPeripheralUUID = connectedPeripheralUUID self.straplog = straplog + self.bodyweightKg = bodyweightKg } // MARK: - Wiring @@ -363,7 +369,8 @@ final class SourceCoordinator: ObservableObject { }, log: straplog, onBattery: { [live] pct in live.setBattery(Double(pct)) }, - adoptIntent: adoptIntent) + adoptIntent: adoptIntent, + bodyweightKg: bodyweightKg) if adoptIntent { straplog("Oura: adopt consent granted - this session may install NOOP's key") } if let pid = peripheralId(for: id), let uuid = UUID(uuidString: pid) { source.connect(uuid) From e465e10104175c4201d19182d59a4aa13a775329 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Wed, 8 Jul 2026 22:44:34 +0200 Subject: [PATCH 12/21] Oura SyncTime (0x12): emit authoritative u64-LE-seconds + tz layout MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The old syncTime built a 24-bit floor(unix/256) coarse time (~4.3-min resolution) with a bogus 0xF6 trailer copied from the 0x85 RTC-beacon. This did not match the native client's req_sync_time(secs, 0) and could set the ring clock wrongly / be silently ignored (masked only by the ring's own RTC still yielding a ~correct 0x42 anchor). Now emit `12 09 `, tz=0 (UTC) as the reference client sends; NOOP buckets local days downstream. Mirrored byte-for-byte in the Kotlin twin. Tests pin the byte layout and the signed tz byte on both sides. Doc §5.4/§8/§9.2 updated to FIXED. --- .../Sources/OuraProtocol/Commands.swift | 21 ++++++++++-------- .../OuraProtocolTests/OuraDriverTests.swift | 16 ++++++++++---- .../src/main/java/com/noop/oura/Commands.kt | 22 +++++++++---------- .../test/java/com/noop/oura/OuraDriverTest.kt | 16 ++++++++++---- docs/OURA_PROTOCOL.md | 22 +++++++++++++------ 5 files changed, 62 insertions(+), 35 deletions(-) diff --git a/Packages/OuraProtocol/Sources/OuraProtocol/Commands.swift b/Packages/OuraProtocol/Sources/OuraProtocol/Commands.swift index d5eaf24a7..406185bad 100644 --- a/Packages/OuraProtocol/Sources/OuraProtocol/Commands.swift +++ b/Packages/OuraProtocol/Sources/OuraProtocol/Commands.swift @@ -54,15 +54,18 @@ public enum OuraCommands { // MARK: - Time sync - /// SyncTime: `12 09 00 00 00 00 f6` where counter = floor(unix_s / 256) - /// and the trailer 0xf6 is fixed. Per OURA_PROTOCOL.md s5.4. `token` defaults to 0. - public static func syncTime(unixSeconds: Int, token: UInt8 = 0x00) -> OuraCommand { - let counter = unixSeconds / 256 - let c0 = UInt8(counter & 0xFF) - let c1 = UInt8((counter >> 8) & 0xFF) - let c2 = UInt8((counter >> 16) & 0xFF) - return OuraCommand(label: "sync_time", - bytes: [0x12, 0x09, token, c0, c1, c2, 0x00, 0x00, 0x00, 0x00, 0xF6]) + /// SyncTime (`0x12`): hand the ring the current wall-clock so it can emit a usable `0x42` UTC anchor + /// (§5.5). Layout `12 09 ` — unix **seconds**, 8-byte + /// little-endian, one signed timezone byte in 30-minute units. Matches the authoritative open_oura + /// `req_sync_time(secs, 0)` ([oura-proto]/[oura-link], OURA_PROTOCOL.md §5.4/§9.2). Supersedes an + /// earlier reverse-engineered guess (`token` + `unix_s/256` in 3 bytes + `0xF6` trailer) that did NOT + /// match the native client. `tzHalfHours` defaults to 0 (UTC), exactly as the reference client sends; + /// NOOP does its own LOCAL-day bucketing downstream regardless of what the ring is told here. + public static func syncTime(unixSeconds: Int, tzHalfHours: Int8 = 0) -> OuraCommand { + let secs = UInt64(bitPattern: Int64(unixSeconds)) + var body: [UInt8] = (0..<8).map { UInt8((secs >> (UInt64($0) * 8)) & 0xFF) } // u64 seconds, LE + body.append(UInt8(bitPattern: tzHalfHours)) // i8 tz (30-min units) + return OuraCommand(label: "sync_time", bytes: [0x12, UInt8(body.count)] + body) } // MARK: - Event fetch (cursor) diff --git a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift index 123685d0c..4a0d55edf 100644 --- a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift +++ b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift @@ -495,10 +495,18 @@ final class OuraDriverTests: XCTestCase { XCTAssertTrue(OuraRingGen.gen3.capabilities.contains(.hrv)) } - func testSyncTimeCommandCounter() { - // counter = floor(unix / 256). For unix = 256 -> counter 1 -> bytes 01 00 00, trailer 0xF6. - let cmd = OuraCommands.syncTime(unixSeconds: 256) - XCTAssertEqual(cmd.bytes, [0x12, 0x09, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xF6]) + func testSyncTimeCommandIsU64SecondsLEPlusTz() { + // Authoritative layout: 12 09 . 1_700_000_000 = 0x6553F100 + // -> LE 00 F1 53 65 00 00 00 00, tz 0. (Supersedes the old unix/256 + 0xF6-trailer guess.) + let cmd = OuraCommands.syncTime(unixSeconds: 1_700_000_000) + XCTAssertEqual(cmd.bytes, + [0x12, 0x09, 0x00, 0xF1, 0x53, 0x65, 0x00, 0x00, 0x00, 0x00, 0x00]) + } + + func testSyncTimeCommandTimezoneByte() { + // tz is a signed half-hour offset in the trailing byte: +4 (=UTC+2) -> 0x04; -4 -> 0xFC. + XCTAssertEqual(OuraCommands.syncTime(unixSeconds: 0, tzHalfHours: 4).bytes.last, 0x04) + XCTAssertEqual(OuraCommands.syncTime(unixSeconds: 0, tzHalfHours: -4).bytes.last, 0xFC) } // MARK: - Dangerous commands are isolated and labelled diff --git a/android/app/src/main/java/com/noop/oura/Commands.kt b/android/app/src/main/java/com/noop/oura/Commands.kt index 511b9b7c5..68f8e40f6 100644 --- a/android/app/src/main/java/com/noop/oura/Commands.kt +++ b/android/app/src/main/java/com/noop/oura/Commands.kt @@ -61,18 +61,18 @@ object OuraCommands { // MARK: - Time sync /** - * SyncTime: `12 09 00 00 00 00 f6` where counter = floor(unix_s / 256) - * and the trailer 0xf6 is fixed. Per OURA_PROTOCOL.md s5.4. `token` defaults to 0. + * SyncTime (0x12): hand the ring the current wall-clock so it can emit a usable 0x42 UTC anchor + * (s5.5). Layout `12 09 ` - unix SECONDS, 8-byte little- + * endian, one signed timezone byte in 30-minute units. Matches the authoritative open_oura + * req_sync_time(secs, 0) (OURA_PROTOCOL.md s5.4/s9.2). Supersedes an earlier reverse-engineered guess + * (token + unix_s/256 in 3 bytes + 0xF6 trailer) that did NOT match the native client. `tzHalfHours` + * defaults to 0 (UTC). Byte-identical to the Swift twin (OuraCommands.syncTime). */ - fun syncTime(unixSeconds: Long, token: Int = 0x00): OuraCommand { - val counter = unixSeconds / 256 - val c0 = (counter and 0xFFL).toInt() - val c1 = ((counter shr 8) and 0xFFL).toInt() - val c2 = ((counter shr 16) and 0xFFL).toInt() - return OuraCommand( - "sync_time", - intArrayOf(0x12, 0x09, token and 0xFF, c0, c1, c2, 0x00, 0x00, 0x00, 0x00, 0xF6), - ) + fun syncTime(unixSeconds: Long, tzHalfHours: Int = 0): OuraCommand { + val body = IntArray(9) + for (i in 0 until 8) body[i] = ((unixSeconds shr (i * 8)) and 0xFFL).toInt() // u64 seconds, LE + body[8] = tzHalfHours and 0xFF // i8 tz (30-min units) + return OuraCommand("sync_time", intArrayOf(0x12, body.size) + body) } // MARK: - Event fetch (cursor) diff --git a/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt b/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt index 72ec8e880..564d894f2 100644 --- a/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt +++ b/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt @@ -623,15 +623,23 @@ class OuraDriverTest { } @Test - fun testSyncTimeCommandCounter() { - // counter = floor(unix / 256). For unix = 256 -> counter 1 -> bytes 01 00 00, trailer 0xF6. - val cmd = OuraCommands.syncTime(unixSeconds = 256L) + fun testSyncTimeCommandIsU64SecondsLEPlusTz() { + // Authoritative layout: 12 09 . 1_700_000_000 = 0x6553F100 + // -> LE 00 F1 53 65 00 00 00 00, tz 0. Byte-identical to the Swift twin. + val cmd = OuraCommands.syncTime(unixSeconds = 1_700_000_000L) assertArrayEquals( - intArrayOf(0x12, 0x09, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xF6), + intArrayOf(0x12, 0x09, 0x00, 0xF1, 0x53, 0x65, 0x00, 0x00, 0x00, 0x00, 0x00), cmd.bytes, ) } + @Test + fun testSyncTimeCommandTimezoneByte() { + // tz is a signed half-hour offset in the trailing byte: +4 (=UTC+2) -> 0x04; -4 -> 0xFC. + assertEquals(0x04, OuraCommands.syncTime(unixSeconds = 0L, tzHalfHours = 4).bytes.last()) + assertEquals(0xFC, OuraCommands.syncTime(unixSeconds = 0L, tzHalfHours = -4).bytes.last()) + } + // MARK: - Dangerous commands are isolated and labelled @Test diff --git a/docs/OURA_PROTOCOL.md b/docs/OURA_PROTOCOL.md index de66f6b0a..efb1c63f8 100644 --- a/docs/OURA_PROTOCOL.md +++ b/docs/OURA_PROTOCOL.md @@ -257,10 +257,18 @@ Gen 5 example `0912 020100 020103 010001 090329 665544332211`. [open_oura-r5] ### 5.4 SyncTime (`0x12`) ``` -12 09 00 00 00 00 f6 +12 09 ``` -where `counter = floor(unix_seconds / 256)`, trailer `0xf6` fixed. [open_ring] -Response: `13 05 00`. [open_ring] +Unit is **unix seconds** (NOT ms/deciseconds), 8-byte little-endian, then one signed timezone byte in +30-minute units. Authoritative from open_oura `req_sync_time(now.as_secs(), 0)` ([oura-proto]/[oura-link]); +NOOP sends `tz = 0` (UTC) as the reference client does and buckets local days downstream itself. +> **Superseded RE guess (do not use):** an earlier layout `12 09 00 00 00 00 f6` +> was attributed to [ringverse]/[open_ring] but does NOT match the native client — it encodes a ~4.3-min- +> resolution 24-bit time with a bogus `0xf6` trailer (which looks copied from the `0x85` RTC-beacon trailer). +> `OuraCommands.syncTime` now emits the 8-byte-LE-seconds layout above (§9.2). Fixed 2026-07-08. + +Response: `13 xx …` ack (echo/format unverified; NOOP does not require it — the usable anchor is the ring's +subsequent `0x42` event, §5.5). [open_ring] ### 5.5 Ring-time → UTC anchoring - The ring clock is in **ticks**: default **100 ms/tick** (10 Hz); burst mode **1 ms/tick** (`factor_flag=1`). [open_ring] @@ -460,7 +468,7 @@ NOOP-derived hypnogram as "Oura sleep stages": it is NOOP's own, from the ring's - Resolve the `0x0D` battery percent-vs-voltage offset per generation via captured fixtures (§6.10). - Validate all Tier-B sleep/activity/step layouts against real captures before enabling in scoring. - Confirm live-HR `0x02` path on actual Gen-4/Gen-5 hardware (only Gen-3 is verified in the corpus). -- **⚠ SyncTime (`0x12`) send layout mismatch (§9.2).** [oura-proto] gives `12 09 `; NOOP's `OuraCommands.syncTime` emits a different 24-bit `unixSeconds/256` + `0xF6`-trailer body. Verify against a capture and align — a malformed SyncTime may be setting the ring clock wrongly or being ignored (the ring's own RTC currently still yields a ~correct `0x42` anchor, masking it, but this is fragile). +- **✓ SyncTime (`0x12`) send layout — FIXED 2026-07-08 (§9.2).** [oura-proto] gives `12 09 `; NOOP previously emitted a 24-bit `unixSeconds/256` + `0xF6`-trailer body. `OuraCommands.syncTime` (Swift + Kotlin twin) now emits the authoritative 8-byte-LE-seconds layout with a signed tz byte (`tz=0`/UTC), pinned by tests. - **Reconsider the `skinTempFunnel` sample floor for Oura nights (§9.10).** [oura-sync]'s temperature algorithm accepts a night at ≥ 4 valid 30-sample windows (~120 samples); NOOP requires 300 kept samples/night — likely too strict for a ring (fewer temp samples than a WHOOP strap). --- @@ -474,9 +482,9 @@ The client state machine runs, in order: **1** connect+bond, subscribe notify · - **Hard rule: do NOT issue any RData (`0x03`) for a normal pull** (§9.7). - **NOOP gap:** NOOP skips step 3 (**GetCapabilities**) entirely — it never negotiates extended-vs-legacy, so it always uses the legacy `0x10` drain. Sending SyncTime (step 5) is the fix already landed on `oura-*` branches; GetCapabilities remains an untried lead for the `0x43`-debug-heavy / thin-structured stream. -### 9.2 SyncTime (`0x12`) exact layout — ⚠ NOOP DISCREPANCY [oura-proto, oura-link] -Authoritative build: `12 09 `. Unit is **seconds** (`req_sync_time(now.as_secs(), 0)`), 8-byte little-endian, one signed tz byte. This is the SEND side; §6.11 covers decoding the ring's `0x42` reply. -- **NOOP currently emits a DIFFERENT body**: `12 09 00 00 00 00 F6` — a 24-bit coarse time (÷256 ≈ 4.3-min resolution) with a trailing `0xF6` (resembling the `0x85` RTC-beacon trailer), NOT 8-byte LE seconds. **Almost certainly wrong.** It may set the ring clock incorrectly or be silently rejected; the anchor still lands today only because the ring's own RTC drives a ~correct `0x42` regardless. Fixture-verify and align (see §8). Cross-ref §5.5, §6.11. +### 9.2 SyncTime (`0x12`) exact layout — ✓ FIXED [oura-proto, oura-link] +Authoritative build: `12 09 `. Unit is **seconds** (`req_sync_time(now.as_secs(), 0)`), 8-byte little-endian, one signed tz byte. This is the SEND side; §6.11 covers decoding the ring's `0x42` reply. Canonical layout now in §5.4. +- **Was wrong, now aligned (2026-07-08).** NOOP previously emitted `12 09 00 00 00 00 F6` — a 24-bit coarse time (÷256 ≈ 4.3-min resolution) with a bogus `0xF6` trailer (copied from the `0x85` RTC-beacon), NOT 8-byte LE seconds. `OuraCommands.syncTime` (Swift + Kotlin twin) now emits the authoritative layout above; `tz = 0` (UTC) like the reference, since NOOP buckets local days downstream. Tests pin the byte layout + the signed tz byte. Cross-ref §5.4, §5.5, §6.11. ### 9.3 Command + `0x2F` sub-op set (authoritative) [oura-proto] Outer opcodes: `0x03` RData · `0x06` realtime measurement · `0x08` firmware info · `0x0C` battery request (→ `0x0D` reply) · `0x10` GetEvent · `0x12` SyncTime · `0x18` product info · `0x1C` notification flags · `0x24` set-auth-key · `0x28` sleep-analysis check · `0x2F` extended. From e7b635990d3eee1419201ed70abe7b29a4ff0c57 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Thu, 9 Jul 2026 09:41:28 +0200 Subject: [PATCH 13/21] Oura: wire the ring's phase timeline into a real sleep session An Oura ring streams no accelerometer, so the gravity-driven SleepStager returns nothing for its nights: the Sleep card read blank AND the skin-temp funnel window (gated to a sleepSession) never opened. IntelligenceEngine now extracts the OURA_SLEEP_PHASE events from the night window's events (already read for the off-wrist backstop, no extra store read), builds sessions via OuraSleepSessionBuilder, and passes them to analyzeDay as providedSleepSessions. When non-nil these replace the gravity detector's output, so the ring's night flows through the SAME funnels the WHOOP path uses (sleep totals, skin-temp window, rest). Any non-Oura owner emits no such events, so the gravity stager stays byte-identical for every existing device. The built sessions land in res.cachedSleep, which the existing pipeline already persists. Kotlin twin: added providedSleepSessions to AnalyticsEngine.analyzeDay, the new OuraSleepSessionBuilder, and the IntelligenceEngine wiring (parsing `phase` from EventRow.payloadJSON). StrandAnalytics 964 tests pass, incl. the builder suite. --- Strand/Data/IntelligenceEngine.swift | 19 +++ .../com/noop/analytics/AnalyticsEngine.kt | 14 +- .../com/noop/analytics/IntelligenceEngine.kt | 22 +++- .../noop/analytics/OuraSleepSessionBuilder.kt | 123 ++++++++++++++++++ 4 files changed, 176 insertions(+), 2 deletions(-) create mode 100644 android/app/src/main/java/com/noop/analytics/OuraSleepSessionBuilder.kt diff --git a/Strand/Data/IntelligenceEngine.swift b/Strand/Data/IntelligenceEngine.swift index 4b8d2c541..b6c85da12 100644 --- a/Strand/Data/IntelligenceEngine.swift +++ b/Strand/Data/IntelligenceEngine.swift @@ -551,6 +551,24 @@ final class IntelligenceEngine: ObservableObject { // built ONLY when the mode is active. nil otherwise = analyzeDay's byte-identical default path. var sleepTrace: [String] = [] let traceSink: ((String) -> Void)? = sleepTraceActive ? { sleepTrace.append($0) } : nil + // Oura sleep bridge: an Oura ring streams no accelerometer, so the gravity-driven + // SleepStager returns nothing for its nights — the Sleep card reads blank AND the skin-temp + // funnel window (gated to a sleepSession) never opens. What the ring DOES emit is its own + // coarse per-epoch phase classification (OURA_SLEEP_PHASE, Tier-A 2-bit codes, §6.12), + // already persisted to the `event` table and read above as `wristEvents`. Build NOOP + // sessions from that anchored phase timeline (OuraSleepSessionBuilder) and hand them to + // analyzeDay as `providedSleepSessions`, so the ring's night flows through the SAME funnels + // the WHOOP path uses (sleep totals, skin-temp window, rest). Any non-Oura owner emits no + // such events → `sleepPhases` empty → pass nil, keeping the gravity stager byte-identical + // for every existing device. (SleepNet's polished hypnogram is cloud-locked and off-wire; + // this is the honest ring-native session — see OuraSleepSessionBuilder / OURA_PROTOCOL §6.12.1.) + let sleepPhases: [(ts: Int, stage: Int)] = wristEvents.compactMap { ev in + guard ev.kind == OuraStreamMapping.sleepPhaseEventKind, + let phase = ev.payload["phase"]?.intValue else { return nil } + return (ts: ev.ts, stage: phase) + } + let ouraSessions = sleepPhases.isEmpty + ? nil : OuraSleepSessionBuilder.sessions(fromPhases: sleepPhases) let res = AnalyticsEngine.analyzeDay(day: day, hr: hr, rr: rr, resp: resp, gravity: grav, steps: steps, dayHr: dayHr, daySteps: daySteps, dayGravity: dayGrav, @@ -563,6 +581,7 @@ final class IntelligenceEngine: ObservableObject { // #690: thread the V2 toggle into the NORMAL staging path so // it affects detected nights, not just the self-heal restage. useSleepStagerV2: useSleepStagerV2, + providedSleepSessions: ouraSessions, traceSink: traceSink) // ── Steps test mode: 5/MG raw-counter trace ────────────────────────────────────────────── // Only built when the Steps mode is on (the gate was read once before the loop). Recomputes diff --git a/android/app/src/main/java/com/noop/analytics/AnalyticsEngine.kt b/android/app/src/main/java/com/noop/analytics/AnalyticsEngine.kt index d77b0f1ec..48a1a1081 100644 --- a/android/app/src/main/java/com/noop/analytics/AnalyticsEngine.kt +++ b/android/app/src/main/java/com/noop/analytics/AnalyticsEngine.kt @@ -207,13 +207,25 @@ object AnalyticsEngine { // instead of V1. Default false keeps V1 the byte-identical default for pure-function callers/tests; // IntelligenceEngine threads PuffinExperiment.from(context).experimentalSleepV2. Mirrors Swift. (V7 / #690) useSleepStagerV2: Boolean = false, + // Sessions supplied by the caller INSTEAD of running the gravity-driven detector. detectSleep is + // gravity-driven and a ring streams no accelerometer, so it returns nothing for an Oura night; the + // caller instead builds sessions from the ring's OWN anchored phase timeline + // (OuraSleepSessionBuilder) and passes them here. When non-null these REPLACE the gravity + // detector's output as the day's sessions, so the ring's night flows through the SAME funnels + // (sleep totals, the skin-temp window, rest) the WHOOP path uses. null (the default) keeps every + // existing gravity-based caller/test byte-identical. Mirrors Swift (`[SleepSession]?`; the Kotlin + // analytics session shape is [DetectedSleep], which is Swift's analytics `SleepSession` one-to-one). + providedSleepSessions: List? = null, // Sleep & Rest test-mode trace sink (E11). null = byte-identical default. When non-null the gate // trace from detectSleep and the Rest sub-score line are forwarded line-by-line. Mirrors Swift. traceSink: ((String) -> Unit)? = null, ): DayResult { // ── Sleep detection + staging ───────────────────────────────────────── - val allSessions = SleepStager.detectSleep( + // A ring supplies its own sessions (built from its phase timeline); everything else derives them + // from the gravity-driven stager. `providedSleepSessions` REPLACES detection wholesale — a device + // that hands us a hypnogram has no accelerometer for the stager to work from. Mirrors Swift. + val allSessions = providedSleepSessions ?: SleepStager.detectSleep( hr = hr, rr = rr, resp = resp, gravity = gravity, tzOffsetSeconds = tzOffsetSeconds, wristOff = wristOff, bandSleepState = bandSleepState, useSleepStagerV2 = useSleepStagerV2, diff --git a/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt b/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt index bc8e6edef..56ea5db18 100644 --- a/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt +++ b/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt @@ -2,11 +2,13 @@ package com.noop.analytics import com.noop.data.DailyMetric import com.noop.data.MetricSeriesRow +import com.noop.data.OuraStreamMapping import com.noop.data.SleepSession import com.noop.data.WhoopRepository import com.noop.data.WorkoutRow import com.noop.protocol.DeviceFamily import kotlinx.coroutines.Dispatchers +import org.json.JSONObject import kotlinx.coroutines.sync.Mutex import kotlinx.coroutines.sync.withLock import kotlinx.coroutines.withContext @@ -452,7 +454,24 @@ object IntelligenceEngine { // only when its off-wrist coverage reaches maxOffWristSleepFraction, so a real night with a // short off-wrist tail survives. Pairing needs WRIST_ON too (to bound each interval); a span // still open at the window end closes at `to`. Empty when the strap emitted no wrist events. - val wristOff = AnalyticsEngine.offWristIntervals(repo.events(owner, from, to, STREAM_LIMIT), to) + val nightEvents = repo.events(owner, from, to, STREAM_LIMIT) + val wristOff = AnalyticsEngine.offWristIntervals(nightEvents, to) + // Oura sleep bridge: an Oura ring streams no accelerometer, so the gravity-driven SleepStager + // returns nothing for its nights — the Sleep card reads blank AND the skin-temp funnel window + // (gated to a session) never opens. What the ring DOES emit is its own coarse per-epoch phase + // classification (OURA_SLEEP_PHASE, Tier-A 2-bit codes, §6.12), already persisted to the event + // table and read above as `nightEvents`. Build NOOP sessions from that anchored phase timeline + // (OuraSleepSessionBuilder) and hand them to analyzeDay as `providedSleepSessions`, so the + // ring's night flows through the SAME funnels the WHOOP path uses (sleep totals, skin-temp + // window, rest). Any non-Oura owner emits no such events → `sleepPhases` empty → pass null, + // keeping the gravity stager byte-identical for every existing device. Mirrors Swift. + val sleepPhases = nightEvents.mapNotNull { ev -> + if (ev.kind != OuraStreamMapping.EVENT_SLEEP_PHASE) return@mapNotNull null + val phase = try { JSONObject(ev.payloadJSON).optInt("phase", -1) } catch (_: Throwable) { -1 } + if (phase < 0) null else OuraSleepSessionBuilder.Phase(ts = ev.ts, stage = phase) + } + val ouraSessions = if (sleepPhases.isEmpty()) null + else OuraSleepSessionBuilder.sessions(sleepPhases) // Calendar-day window for the ADDITIVE daily totals (steps + calories). The night window // above is anchored to the current time-of-day and ends at dayStart+12h, so for a PAST @@ -513,6 +532,7 @@ object IntelligenceEngine { // The Context-aware caller (AppViewModel/WhoopBleClient) supplied it from // PuffinExperiment.from(context).experimentalSleepV2. useSleepStagerV2 = useExperimentalSleepV2, + providedSleepSessions = ouraSessions, // Sleep & Rest test mode (Test Centre E5): thread the trace sink straight through. null (the // default) keeps analyzeDay's byte-identical untraced path; when the caller passed a non-null // sink (mode on), detectSleep's gate trace + the Rest sub-score line route to the .sleep-tagged diff --git a/android/app/src/main/java/com/noop/analytics/OuraSleepSessionBuilder.kt b/android/app/src/main/java/com/noop/analytics/OuraSleepSessionBuilder.kt new file mode 100644 index 000000000..e4d79b9db --- /dev/null +++ b/android/app/src/main/java/com/noop/analytics/OuraSleepSessionBuilder.kt @@ -0,0 +1,123 @@ +package com.noop.analytics + +/** + * Build a NOOP sleep session from the Oura ring's OWN anchored sleep-phase timeline + * (`OURA_SLEEP_PHASE` events, Tier-A 2-bit codes; OURA_PROTOCOL.md §6.12). + * + * Why this exists (OURA_PROTOCOL.md §6.12.1): the polished 4-stage hypnogram the Oura APP shows is + * produced by SleepNet, an encrypted, cloud-key-gated PyTorch model on the phone — it is NOT on the BLE + * wire and NOOP neither can nor does reproduce it. What DOES cross BLE is the ring's own coarse + * per-epoch phase classification (awake/light/deep/REM). NOOP builds its OWN session from that, the + * same honest-data stance as every other NOOP metric. + * + * This ALSO bridges an architectural gap: [AnalyticsEngine.analyzeDay] derives sleep from + * [SleepStager.detectSleep], which is **gravity-driven** — and an Oura ring streams no accelerometer, so + * the detector returns nothing. The sessions built here are injected into `analyzeDay` as + * `providedSleepSessions`, taking the place the gravity detector fills for WHOOP, so the ring's night + * flows through the SAME funnels (dailyMetric sleep totals, the skin-temp window, rest). + * + * Pure and platform-neutral: input is `(ts, stage)` pairs (wall-clock unix seconds + the ring's 2-bit + * code). Byte-for-byte twin of Swift `OuraSleepSessionBuilder`. + */ +object OuraSleepSessionBuilder { + + /** + * The ring's 2-bit sleep-phase code (OURA_PROTOCOL.md §6.12: `0=awake, 1=light, 2=deep, 3=REM`) + * mapped to the [StageSegment.stage] string the rest of analytics uses. Returns null for an + * unknown code (a corrupt/misframed value), so it is dropped rather than guessed (honest-data). + */ + internal fun stageName(forPhaseCode code: Int): String? = when (code) { + 0 -> "wake" + 1 -> "light" + 2 -> "deep" + 3 -> "rem" + else -> null + } + + /** + * A single anchored phase event: wall-clock unix seconds and the ring's 2-bit phase code. + */ + data class Phase(val ts: Long, val stage: Int) + + /** + * Build sleep session(s) from the ring's anchored phase timeline. + * + * Each phase event marks a stage that HOLDS until the next event, so consecutive events + * `[tsᵢ, tsᵢ₊₁)` form one [StageSegment] with `stageᵢ`; the session spans `first → last` event. + * Adjacent same-stage segments are merged for a clean hypnogram. Efficiency = asleep / in-bed + * (asleep = non-wake duration). A large inter-event gap splits into separate sessions (a nap vs the + * overnight), and a session shorter than [minSessionMinutes] or with no asleep time is dropped as + * noise. `restingHR`/`avgHRV` are left null here — they are enriched by the caller/engine from the + * night's HR/RR streams, not fabricated from the phase codes. + * + * @param phases `(ts, stage)` pairs. Need not be pre-sorted; duplicate timestamps keep the + * first-seen stage. + * @param minSessionMinutes shortest span kept (default 60 — drops stray fragments; a real nap the + * ring staged still clears this, an isolated blip does not). + * @param splitGapMinutes an inter-event gap longer than this starts a new session (default 120). + */ + fun sessions( + fromPhases: List, + minSessionMinutes: Int = 60, + splitGapMinutes: Int = 120, + ): List { + // Sort by time; collapse duplicate timestamps (keep first) so a doubled event can't make a + // zero-length segment. + val sorted = fromPhases.sortedBy { it.ts } + val events = ArrayList() + for (e in sorted) if (e.ts != events.lastOrNull()?.ts) events.add(e) + if (events.size < 2) return emptyList() + + // Split into contiguous runs on a large gap (nap vs overnight). + val splitGap = splitGapMinutes.toLong() * 60L + val runs = ArrayList>() + var current = mutableListOf(events[0]) + for (e in events.drop(1)) { + if (e.ts - current[current.size - 1].ts > splitGap) { + runs.add(current) + current = mutableListOf(e) + } else { + current.add(e) + } + } + runs.add(current) + + val minSpan = minSessionMinutes.toLong() * 60L + val out = ArrayList() + for (run in runs) { + if (run.size < 2) continue + val start = run[0].ts + val end = run[run.size - 1].ts + if (end - start < minSpan) continue + + // One segment per [tsᵢ, tsᵢ₊₁); drop segments whose code is unknown (never guess a stage). + val segments = ArrayList() + for (i in 0 until run.size - 1) { + val name = stageName(forPhaseCode = run[i].stage) ?: continue + val seg = StageSegment(start = run[i].ts, end = run[i + 1].ts, stage = name) + // Merge with the previous segment when the stage is identical and they abut. + val last = segments.lastOrNull() + if (last != null && last.stage == seg.stage && last.end == seg.start) { + last.end = seg.end + } else { + segments.add(seg) + } + } + if (segments.isEmpty()) continue + + var asleepSeconds = 0L + for (seg in segments) if (seg.stage != "wake") asleepSeconds += seg.end - seg.start + if (asleepSeconds <= 0L) continue // an all-wake run is not a sleep session + val inBed = end - start + val efficiency = if (inBed > 0L) minOf(1.0, asleepSeconds.toDouble() / inBed.toDouble()) else 0.0 + + out.add( + DetectedSleep( + start = start, end = end, efficiency = efficiency, + stages = segments, restingHR = null, avgHRV = null, + ) + ) + } + return out + } +} From 107d5b9f29e8ac57b22156395a88ed75f05b33d9 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Thu, 9 Jul 2026 09:44:19 +0200 Subject: [PATCH 14/21] Oura: drop phantom history records instead of banking them at wall-clock The phantom guard (unixSeconds anchor-relative window) already rejected a misframed record's garbage ring timestamp on the ingest path. But the pending-anchor DRAIN collapsed the two reasons unixSeconds returns nil -- "no anchor yet" and "anchor present but rt is phantom" -- into a single '?? now` fallback. So a misframed skin-temp/SpO2/phase parked before the anchor landed was stamped at wall-clock and BANKED, slipping past the guard and polluting "today" with a bogus sample (honest-data violation). Expose driver.hasAnchor so the drain can tell them apart: anchor present + rt rejected -> DROP (and log the count); no anchor this session -> keep the honest wall-clock fallback so a real last-night record is placed at ~arrival rather than silently lost. --- .../Sources/OuraProtocol/OuraDriver.swift | 6 +++++ .../OuraProtocolTests/OuraDriverTests.swift | 18 +++++++++++++ Strand/BLE/OuraLiveSource.swift | 20 +++++++++++++-- .../main/java/com/noop/ble/OuraLiveSource.kt | 19 ++++++++++++-- .../src/main/java/com/noop/oura/OuraDriver.kt | 8 ++++++ .../test/java/com/noop/oura/OuraDriverTest.kt | 25 +++++++++++++++++++ 6 files changed, 92 insertions(+), 4 deletions(-) diff --git a/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift b/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift index 9541dc6ef..df90bdf91 100644 --- a/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift +++ b/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift @@ -215,6 +215,12 @@ public final class OuraDriver { /// Convert a record's ring-clock timestamp to unix seconds using the current session's anchor /// (OURA_PROTOCOL.md s5.5). Returns nil when no anchor has arrived yet this session, so the caller /// can honestly fall back (e.g. to wall-clock arrival time) instead of guessing. + /// Whether a ring-time→UTC anchor has arrived this session. Lets a caller tell the TWO reasons + /// `unixSeconds(forRingTimestamp:)` returns nil apart: `false` ⇒ no anchor yet (park / honest + /// wall-clock fallback is OK); `true` ⇒ an anchor EXISTS but the ring timestamp was rejected as + /// phantom/out-of-window, so the record must be DROPPED, never stamped at wall-clock (honest-data). + public var hasAnchor: Bool { anchorUtcMs != nil && anchorRingTime != nil } + public func unixSeconds(forRingTimestamp rt: UInt32) -> Int? { guard let anchorUtcMs, let anchorRingTime else { return nil } let deltaTicks = Int64(rt) - Int64(anchorRingTime) diff --git a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift index 4a0d55edf..a580bc8f1 100644 --- a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift +++ b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift @@ -215,6 +215,24 @@ final class OuraDriverTests: XCTestCase { XCTAssertNil(d.unixSeconds(forRingTimestamp: 0)) } + func testHasAnchorDistinguishesNoAnchorFromPhantomRejection() { + // hasAnchor lets the live-source drain tell the two `unixSeconds == nil` cases apart: park/fallback + // when no anchor yet, vs DROP a phantom once an anchor exists. + let d = OuraDriver(ringGen: .gen3, authKey: key) + XCTAssertFalse(d.hasAnchor) // nothing anchored yet + XCTAssertNil(d.unixSeconds(forRingTimestamp: 12_345)) // → caller should PARK + + let anchorEpochSeconds: Int64 = 1_700_000_000 + let anchorRt: UInt32 = 100_000_000 + _ = d.ingest(record: OuraRecord(type: OuraEventTag.timeSync.rawValue, ringTimestamp: anchorRt, + payload: le8(anchorEpochSeconds) + [0x00])) + XCTAssertTrue(d.hasAnchor) // anchor present now + XCTAssertNil(d.unixSeconds(forRingTimestamp: 0)) // phantom rt rejected → caller should DROP + + d.stop() + XCTAssertFalse(d.hasAnchor) // stop clears the anchor + } + func testAnchorRelativeWindowEndpointsAreInclusive() { let d = OuraDriver(ringGen: .gen3, authKey: key) let anchorEpochSeconds: Int64 = 1_700_000_000 diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index 51f1d9b49..7154a1f29 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -611,9 +611,25 @@ public final class OuraLiveSource: NSObject, ObservableObject { private func drainPendingAnchorEvents() { guard !pendingAnchorEvents.isEmpty, let driver else { return } let now = Int(Date().timeIntervalSince1970) + // `unixSeconds` returns nil for TWO reasons; the phantom guard is only honest if we tell them apart: + // • an anchor EXISTS but the ring timestamp was rejected as phantom/out-of-window → DROP the + // record. The old `?? now` fallback stamped it at wall-clock and BANKED it, so a misframed + // skin-temp/SpO2/phase parked before the anchor landed slipped past the guard and polluted + // "today" with a bogus sample (honest-data violation). Dropping is the whole point of the guard. + // • NO anchor ever arrived this session (teardown drain) → keep the honest wall-clock fallback so + // a real last-night record is placed at ~arrival rather than silently lost. + var droppedPhantoms = 0 for pending in pendingAnchorEvents { - let ts = driver.unixSeconds(forRingTimestamp: pending.ringTimestamp) ?? now - enqueue([pending.event], ts: ts) + if let ts = driver.unixSeconds(forRingTimestamp: pending.ringTimestamp) { + enqueue([pending.event], ts: ts) + } else if driver.hasAnchor { + droppedPhantoms += 1 // anchor present but rt is phantom → drop, never bank + } else { + enqueue([pending.event], ts: now) // no anchor this session → honest wall-clock fallback + } + } + if droppedPhantoms > 0 { + log("Oura: dropped \(droppedPhantoms) phantom record(s) - ring timestamp outside the anchor window") } pendingAnchorEvents.removeAll() } diff --git a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt index e29d57fd9..68f93f4cd 100644 --- a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt +++ b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt @@ -638,9 +638,24 @@ class OuraLiveSource( if (pendingAnchorEvents.isEmpty()) return@guardedCallback val d = driver ?: return@guardedCallback val now = (System.currentTimeMillis() / 1000L).toInt() + // `unixSeconds` returns null for TWO reasons; the phantom guard is only honest if we tell them apart: + // - an anchor EXISTS but the ring timestamp was rejected as phantom/out-of-window -> DROP the + // record. The old `?: now` fallback stamped it at wall-clock and BANKED it, so a misframed + // skin-temp/SpO2/phase parked before the anchor landed slipped past the guard and polluted + // "today" with a bogus sample (honest-data violation). Dropping is the whole point of the guard. + // - NO anchor ever arrived this session (teardown drain) -> keep the honest wall-clock fallback so + // a real last-night record is placed at ~arrival rather than silently lost. Mirrors Swift. + var droppedPhantoms = 0 for ((event, ringTimestamp) in pendingAnchorEvents) { - val ts = d.unixSeconds(forRingTimestamp = ringTimestamp)?.toInt() ?: now - enqueue(listOf(event), ts) + val ts = d.unixSeconds(forRingTimestamp = ringTimestamp)?.toInt() + when { + ts != null -> enqueue(listOf(event), ts) + d.hasAnchor -> droppedPhantoms++ // anchor present but rt is phantom -> drop, never bank + else -> enqueue(listOf(event), now) // no anchor this session -> honest wall-clock fallback + } + } + if (droppedPhantoms > 0) { + log("Oura: dropped $droppedPhantoms phantom record(s) - ring timestamp outside the anchor window") } pendingAnchorEvents.clear() } diff --git a/android/app/src/main/java/com/noop/oura/OuraDriver.kt b/android/app/src/main/java/com/noop/oura/OuraDriver.kt index fec8b4137..c0fc0c8d3 100644 --- a/android/app/src/main/java/com/noop/oura/OuraDriver.kt +++ b/android/app/src/main/java/com/noop/oura/OuraDriver.kt @@ -284,6 +284,14 @@ class OuraDriver( // MARK: - Ring-time -> UTC anchor (s5.5) + /** + * Whether a ring-time->UTC anchor has arrived this session. Lets a caller tell the TWO reasons + * [unixSeconds] returns null apart: `false` => no anchor yet (park / honest wall-clock fallback is + * OK); `true` => an anchor EXISTS but the ring timestamp was rejected as phantom/out-of-window, so + * the record must be DROPPED, never stamped at wall-clock (honest-data). Twin of Swift `hasAnchor`. + */ + val hasAnchor: Boolean get() = anchorUtcMs != null && anchorRingTime != null + /** * Convert a record's ring-clock timestamp to unix seconds using the current session's anchor * (OURA_PROTOCOL.md s5.5). Returns null when no anchor has arrived yet this session, so the caller diff --git a/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt b/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt index 564d894f2..a06742d73 100644 --- a/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt +++ b/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt @@ -2,6 +2,7 @@ package com.noop.oura import org.junit.Assert.assertArrayEquals import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse import org.junit.Assert.assertNotNull import org.junit.Assert.assertNull import org.junit.Assert.assertTrue @@ -257,6 +258,30 @@ class OuraDriverTest { assertNull(d.unixSeconds(forRingTimestamp = 0L)) } + @Test + fun testHasAnchorDistinguishesNoAnchorFromPhantomRejection() { + // Parity with Swift testHasAnchorDistinguishesNoAnchorFromPhantomRejection. hasAnchor lets the + // live-source drain tell the two `unixSeconds == null` cases apart: park/fallback when no anchor + // yet, vs DROP a phantom once an anchor exists. + val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) + assertFalse(d.hasAnchor) // nothing anchored yet + assertNull(d.unixSeconds(forRingTimestamp = 12_345L)) // -> caller should PARK + + val anchorEpochSeconds = 1_700_000_000L + val anchorRt = 100_000_000L + d.ingest( + OuraRecord( + type = OuraEventTag.TIME_SYNC.raw, ringTimestamp = anchorRt, + payload = le8(anchorEpochSeconds) + intArrayOf(0x00), + ), + ) + assertTrue(d.hasAnchor) // anchor present now + assertNull(d.unixSeconds(forRingTimestamp = 0L)) // phantom rt rejected -> caller should DROP + + d.stop() + assertFalse(d.hasAnchor) // stop clears the anchor + } + @Test fun testAnchorRelativeWindowEndpointsAreInclusive() { val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) From b6d894004484e7dc4b077659929981b4ae49a05c Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Thu, 9 Jul 2026 10:06:00 +0200 Subject: [PATCH 15/21] Oura: parse the ring's own check_sleep window (PROTOTYPE, honest sleep source) The decoded OURA_SLEEP_PHASE (0x4E) events turn out to be sparse bursts the ring emits only at connection time, not a continuous overnight timeline -- a real 8 h night persisted ~30 phase events clustered at the sync moments, none during sleep, so a session built from them under-counts (9.2 h read as ~5 h). The ring's firmware instead narrates its OWN sleep decision as 0x43 debug text: `check_sleep`, then `s: ` (bedtime) / `e: ` (wake), where is a ring timestamp in the 0x42 anchor domain -- so unixSeconds(rt) converts each straight to UTC. OuraCheckSleepParser (pure, both platforms) extracts the window and OuraLiveSource logs the anchored bedtime->wake span. Validated on device: last night read 23:46 -> 07:43 (7h 56m) vs the wearer's actual 23:49 -> 07:41. A lone `e:` can arrive with no fresh `s:`, pairing the new wake against the previous night's stale start -- observed live as a phantom 33 h window. An 18 h max-window guard rejects it (honest-data). Also confirmed and documented (OURA_PROTOCOL.md 6.15): the Tier-B sleep_summary tags (0x49/4B/4C/57/58 = ASCII I/K/L/W/X) captured so far are framing-desync aliases of debug text ("DHR_state:2", "batt: 100"), not real summaries -- do not decode until a clean binary fixture. INVESTIGATION prototype: the window is LOGGED, not yet persisted as a sleepSession. --- .../OuraProtocol/CheckSleepParser.swift | 81 ++++++++++++++++++ .../CheckSleepParserTests.swift | 75 +++++++++++++++++ Strand/BLE/OuraLiveSource.swift | 26 ++++++ .../main/java/com/noop/ble/OuraLiveSource.kt | 35 +++++++- .../java/com/noop/oura/CheckSleepParser.kt | 82 +++++++++++++++++++ .../com/noop/oura/CheckSleepParserTest.kt | 77 +++++++++++++++++ docs/OURA_PROTOCOL.md | 2 + 7 files changed, 377 insertions(+), 1 deletion(-) create mode 100644 Packages/OuraProtocol/Sources/OuraProtocol/CheckSleepParser.swift create mode 100644 Packages/OuraProtocol/Tests/OuraProtocolTests/CheckSleepParserTests.swift create mode 100644 android/app/src/main/java/com/noop/oura/CheckSleepParser.kt create mode 100644 android/app/src/test/java/com/noop/oura/CheckSleepParserTest.kt diff --git a/Packages/OuraProtocol/Sources/OuraProtocol/CheckSleepParser.swift b/Packages/OuraProtocol/Sources/OuraProtocol/CheckSleepParser.swift new file mode 100644 index 000000000..76edb7ba5 --- /dev/null +++ b/Packages/OuraProtocol/Sources/OuraProtocol/CheckSleepParser.swift @@ -0,0 +1,81 @@ +import Foundation + +/// Parse the ring's OWN computed sleep window out of its `check_sleep` debug-text (`0x43`) stream +/// (OURA_PROTOCOL.md §6.15). This is a PROTOTYPE / INVESTIGATION source (OURA_PROTOCOL.md §6.12.1): +/// the ring's firmware periodically logs its sleep-detection state as plain ASCII lines — +/// +/// check_sleep +/// s: 114643 ← bedtime, a ring timestamp in the anchor's domain +/// e: 446340 ← wake, a ring timestamp in the anchor's domain +/// not needed +/// +/// — and those `s:` / `e:` values are ring timestamps in the SAME domain as the `0x42` UTC anchor +/// (§5.5), so `OuraDriver.unixSeconds(forRingTimestamp:)` converts them straight to bedtime/wake UTC. +/// +/// Why this matters: the decoded `OURA_SLEEP_PHASE` (`0x4E`) events turn out to be sparse bursts the +/// ring emits at connection time, NOT a continuous overnight timeline — so a session built from them +/// under-counts (a 9.2 h night read as ~5 h). The `s:`/`e:` window is the ring's own boundary decision +/// and is far more reliable for sleep DURATION. Unlike the Tier-B `sleep_summary` tags (`0x49/4B/4C/ +/// 57/58`, which are ASCII letters `I/K/L/W/X` a framing desync aliases out of debug text — verified as +/// junk, never a real summary), the `s:`/`e:` lines are unambiguous ASCII and safe to read. +/// +/// Honest-data stance: this only reads what the ring itself computed; it never fabricates stages. Kept +/// pure and platform-neutral (input is text lines) so it unit-tests in the fast package loop. It is an +/// INVESTIGATION prototype — the caller LOGS the anchored window, does not yet persist a session. +public struct OuraCheckSleepParser { + private var lastStartRt: UInt32? + private var lastEndRt: UInt32? + private var lastEmitted: Window? + + /// A ring-timestamp sleep window: `startRt` = bedtime, `endRt` = wake, both in the anchor domain. + public struct Window: Equatable, Sendable { + public let startRt: UInt32 + public let endRt: UInt32 + public init(startRt: UInt32, endRt: UInt32) { self.startRt = startRt; self.endRt = endRt } + } + + /// Longest plausible in-bed span, in ring ticks (100 ms/tick, §5.5): 18 h. A `check_sleep` block can + /// emit a lone `e:` (wake) with no fresh `s:`, which would otherwise pair the NEW wake against the + /// PREVIOUS night's stale `s:` — observed live as a phantom 33 h window (2026-07-09). No real sleep is + /// 18 h, so reject any window longer than this rather than emit a cross-block mis-pairing (honest-data). + private static let maxWindowTicks: UInt32 = 648_000 // 18 h × 3600 s × 10 ticks/s + + public init() {} + + /// Clear the accumulated state (call on stop/disconnect so a new session starts fresh). + public mutating func reset() { + lastStartRt = nil + lastEndRt = nil + lastEmitted = nil + } + + /// Feed ONE trimmed debug-text line. Returns a `Window` when a NEW, complete (`endRt > startRt`) + /// sleep window is recognized; nil otherwise (an unrelated line, an incomplete pair, or a window + /// identical to the last one emitted — so repeated `e:` refinements collapse to one emit each). + /// + /// Matches ONLY the exact lowercase boundary lines `s: ` / `e: ` the firmware emits + /// for sleep (so `tsc:…`, `bed:…`, `ns=…`, etc. are ignored), keeping the guess honest. + public mutating func ingest(line: String) -> Window? { + if let rt = Self.ringValue(line, prefix: "s:") { + lastStartRt = rt + } else if let rt = Self.ringValue(line, prefix: "e:") { + lastEndRt = rt + } else { + return nil // not a boundary line — nothing to update + } + guard let s = lastStartRt, let e = lastEndRt, e > s, e - s <= Self.maxWindowTicks else { return nil } + let window = Window(startRt: s, endRt: e) + guard window != lastEmitted else { return nil } // unchanged since last emit → don't re-log + lastEmitted = window + return window + } + + /// Parse `" "` (single ASCII space) into a ring timestamp, or nil if the line is + /// not exactly that shape. Rejects a non-numeric or overflowing tail rather than guessing. + private static func ringValue(_ line: String, prefix: String) -> UInt32? { + guard line.hasPrefix(prefix) else { return nil } + let rest = line.dropFirst(prefix.count).drop(while: { $0 == " " }) + guard !rest.isEmpty, rest.allSatisfy(\.isNumber) else { return nil } + return UInt32(rest) + } +} diff --git a/Packages/OuraProtocol/Tests/OuraProtocolTests/CheckSleepParserTests.swift b/Packages/OuraProtocol/Tests/OuraProtocolTests/CheckSleepParserTests.swift new file mode 100644 index 000000000..d775f3ace --- /dev/null +++ b/Packages/OuraProtocol/Tests/OuraProtocolTests/CheckSleepParserTests.swift @@ -0,0 +1,75 @@ +import XCTest +@testable import OuraProtocol + +/// Tests for the `check_sleep` s:/e: sleep-window parser (OURA_PROTOCOL.md §6.15 prototype). Fixtures +/// are the exact debug-text lines captured from a real Gen3 ring. +final class CheckSleepParserTests: XCTestCase { + + func testExtractsWindowFromRealCheckSleepSequence() { + var p = OuraCheckSleepParser() + // The real captured sequence: check_sleep, then s:/e: boundaries. + XCTAssertNil(p.ingest(line: "check_sleep")) + XCTAssertNil(p.ingest(line: "s: 114643")) // start alone → incomplete + XCTAssertEqual(p.ingest(line: "e: 446001"), .init(startRt: 114_643, endRt: 446_001)) + XCTAssertNil(p.ingest(line: "not needed")) // unrelated line + // A refined wake emits the new window once… + XCTAssertEqual(p.ingest(line: "e: 446340"), .init(startRt: 114_643, endRt: 446_340)) + // …but a repeat of the same window does not re-emit. + XCTAssertNil(p.ingest(line: "e: 446340")) + } + + func testIgnoresLookalikeAndNonBoundaryLines() { + var p = OuraCheckSleepParser() + _ = p.ingest(line: "s: 114643") + // `tsc:`, `bed:`, `ns=`, `e:` with a non-numeric tail, and empty tails must NOT be read as e:/s:. + XCTAssertNil(p.ingest(line: "tsc:60464")) + XCTAssertNil(p.ingest(line: "bed: 114643")) + XCTAssertNil(p.ingest(line: "ns=1025898")) + XCTAssertNil(p.ingest(line: "e: pp_stop")) + XCTAssertNil(p.ingest(line: "e:")) + // A valid e: still completes the window afterwards. + XCTAssertEqual(p.ingest(line: "e: 446340"), .init(startRt: 114_643, endRt: 446_340)) + } + + func testRejectsInvertedWindow() { + var p = OuraCheckSleepParser() + _ = p.ingest(line: "s: 500000") + XCTAssertNil(p.ingest(line: "e: 446340")) // wake before bedtime → not a window + } + + func testRejectsCrossBlockPhantomWindow() { + // Real 2026-07-09 capture: last night's wake `e: 1311598` arrived while the PREVIOUS night's + // `s: 114643` was still latched → a 33 h window. The max-duration guard must reject it… + var p = OuraCheckSleepParser() + _ = p.ingest(line: "s: 114643") + XCTAssertNil(p.ingest(line: "e: 1311598")) // 1_196_955 ticks ≈ 33.2 h → rejected + // …but a fresh `s:` completes the window against the latched `e:` (≈ 7.9 h) — and, matching the + // real log, the emit fires on THIS `s:` line (the `e:` is already latched). + XCTAssertEqual(p.ingest(line: "s: 1025598"), .init(startRt: 1_025_598, endRt: 1_311_598)) + } + + func testResetClearsState() { + var p = OuraCheckSleepParser() + _ = p.ingest(line: "s: 114643") + _ = p.ingest(line: "e: 446340") + p.reset() + // After reset, a lone e: cannot complete a window (start was cleared). + XCTAssertNil(p.ingest(line: "e: 446340")) + } + + func testAnchoredWindowConvertsToRealUtc() { + // End to end: feed the driver a UTC anchor, then anchor the parsed s:/e: to unix seconds. + let key = [UInt8](0..<16) + let d = OuraDriver(ringGen: .gen3, authKey: key) + let anchorEpoch: Int64 = 1_700_000_000 + let anchorRt: UInt32 = 500_000 + _ = d.ingest(record: OuraRecord(type: OuraEventTag.timeSync.rawValue, ringTimestamp: anchorRt, + payload: (0..<8).map { UInt8((UInt64(bitPattern: anchorEpoch) >> ($0 * 8)) & 0xFF) } + [0x00])) + var p = OuraCheckSleepParser() + _ = p.ingest(line: "s: 400000") + let w = p.ingest(line: "e: 450000")! + // 400000 ticks = 100_000 before anchor → -10_000 s; 450000 = 50_000 before → -5_000 s. + XCTAssertEqual(d.unixSeconds(forRingTimestamp: w.startRt), Int(anchorEpoch) - 10_000) + XCTAssertEqual(d.unixSeconds(forRingTimestamp: w.endRt), Int(anchorEpoch) - 5_000) + } +} diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index b0ba7d859..bf0fe674e 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -454,6 +454,7 @@ public final class OuraLiveSource: NSObject, ObservableObject { loggedFirstTemp = false loggedFirstSpo2 = false loggedAnchor = false + checkSleepParser.reset() loggedTierBKinds.removeAll() reachedStreaming = false pendingInstallKey = nil @@ -779,6 +780,7 @@ public final class OuraLiveSource: NSObject, ObservableObject { // lowercase `s:`/`e:`/`nb:` sleep boundaries), `auth_key_set`, `batt:…`, `orientation`. let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) logDebug(trimmed, rt: ringTimestamp) + logCheckSleepWindowIfAny(from: trimmed) // PROTOTYPE: ring's own sleep window (§6.15) default: break // motion / state / rtcBeacon: not a durable Streams row (see OuraStreamMapping) @@ -833,6 +835,11 @@ public final class OuraLiveSource: NSObject, ObservableObject { /// Last debug string we logged, to collapse consecutive identical lines (e.g. `DHR_info:neg t` ×20). private var lastDebugText: String? + /// PROTOTYPE (§6.15 / §6.12.1, INVESTIGATION): accumulates the ring's `check_sleep` `s:`/`e:` debug + /// boundaries into its OWN computed sleep window — a far more reliable duration signal than the sparse + /// `OURA_SLEEP_PHASE` bursts. Reset per session. The window is LOGGED (anchored to UTC), not yet persisted. + private var checkSleepParser = OuraCheckSleepParser() + /// Log a decoded 0x43 debug string unless it is empty, a known high-rate flood (drop-prefix), or an /// immediate duplicate of the previous logged line. Investigation only — never persisted or scored. private func logDebug(_ text: String, rt: UInt32) { @@ -843,6 +850,23 @@ public final class OuraLiveSource: NSObject, ObservableObject { log("Oura: debug (0x43) rt=\(rt) \"\(text)\"") } + /// PROTOTYPE (§6.15, INVESTIGATION): feed a debug line to the `check_sleep` parser and, when it yields + /// a NEW sleep window, anchor both boundaries to UTC (§5.5) and log the ring's own bedtime→wake span. + /// This is the honest sleep-duration signal (the ring's own decision); it is NOT yet turned into a + /// persisted `sleepSession`. `s:`/`e:` are ring timestamps in the anchor domain, so unresolved (no + /// anchor yet, or a phantom value) simply skips — never a guessed time. + private func logCheckSleepWindowIfAny(from line: String) { + guard let w = checkSleepParser.ingest(line: line), + let driver, + let bedtime = driver.unixSeconds(forRingTimestamp: w.startRt), + let wake = driver.unixSeconds(forRingTimestamp: w.endRt), + wake > bedtime else { return } + let mins = (wake - bedtime) / 60 + let bedStr = Self.cursorDateFormatter.string(from: Date(timeIntervalSince1970: TimeInterval(bedtime))) + let wakeStr = Self.cursorDateFormatter.string(from: Date(timeIntervalSince1970: TimeInterval(wake))) + log("Oura: ring sleep window (check_sleep) \(bedStr) → \(wakeStr) (\(mins / 60)h \(mins % 60)m) [PROTOTYPE — not persisted]") + } + // MARK: - Re-engagement timer (daytime-HR auto-reverts ~20s) private func startReengageTimer() { @@ -987,6 +1011,7 @@ extension OuraLiveSource: @preconcurrency CBCentralManagerDelegate { loggedFirstTemp = false loggedFirstSpo2 = false loggedAnchor = false + checkSleepParser.reset() loggedTierBKinds.removeAll() pendingAnchorEvents.removeAll() // a fresh session must never replay a stale-anchor guess pendingInstallKey = nil @@ -1037,6 +1062,7 @@ extension OuraLiveSource: @preconcurrency CBCentralManagerDelegate { loggedFirstTemp = false loggedFirstSpo2 = false loggedAnchor = false + checkSleepParser.reset() loggedTierBKinds.removeAll() reachedStreaming = false pendingInstallKey = nil diff --git a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt index 0c88dd59e..dda2a8091 100644 --- a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt +++ b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt @@ -40,6 +40,9 @@ import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.StateFlow import kotlinx.coroutines.flow.asStateFlow import java.security.SecureRandom +import java.text.SimpleDateFormat +import java.util.Date +import java.util.Locale import java.util.UUID import java.util.concurrent.ConcurrentHashMap @@ -225,6 +228,11 @@ class OuraLiveSource( private var loggedFirstSpo2 = false /** Logs the FIRST ring-time -> UTC anchor of this session only (s5.5); reset on stop/disconnect. */ private var loggedAnchor = false + /** PROTOTYPE (§6.15 / §6.12.1, INVESTIGATION): accumulates the ring's `check_sleep` `s:`/`e:` debug + * boundaries into its OWN computed sleep window - a far more reliable duration signal than the sparse + * OURA_SLEEP_PHASE bursts. Reset per session. The window is LOGGED (anchored to UTC), not yet persisted. + * Twin of the Swift [checkSleepParser]. */ + private val checkSleepParser = OuraCheckSleepParser() /** Tier-B (UNVERIFIED) kinds ("activity" / "real_steps" / "sleep_summary" / "spo2_smoothed") already * logged this session, so a repeated tag logs once per KIND, not once per record. INVESTIGATION * ONLY (see the `allowTierB = true` comment at driver construction) - the log is how we collect raw @@ -528,6 +536,7 @@ class OuraLiveSource( loggedFirstTemp = false loggedFirstSpo2 = false loggedAnchor = false + checkSleepParser.reset() loggedTierBKinds.clear() pendingAnchorEvents.clear() // Resume the GetEvents cursor from where the LAST connection to this ring left off (s5.1/5.3), so a @@ -577,6 +586,7 @@ class OuraLiveSource( loggedFirstTemp = false loggedFirstSpo2 = false loggedAnchor = false + checkSleepParser.reset() loggedTierBKinds.clear() reachedStreaming = false // A stop MID-install is an honest failure (no ack will come); a stop after streaming leaves the @@ -720,6 +730,7 @@ class OuraLiveSource( loggedFirstTemp = false loggedFirstSpo2 = false loggedAnchor = false + checkSleepParser.reset() loggedTierBKinds.clear() reachedStreaming = false // A disconnect MID-install is an honest failure (no 0x25 ack will arrive); a disconnect @@ -1142,11 +1153,33 @@ class OuraLiveSource( // every real capture is evidence. Never persisted, never scored, and NEVER converted into // steps (MET is not a step count; OuraStreamMapping drops ActivityInfo unconditionally). log("Oura: activity (Tier-B) state=${e.value.state} met=${e.value.met}") - // Motion / state / rtcBeacon / debugText: not a durable Streams row (see OuraStreamMapping). + is OuraEvent.DebugTextEvent -> + // PROTOTYPE (§6.15, INVESTIGATION): the ring logs its OWN computed sleep window as + // `check_sleep` s:/e: debug lines. Feed them to the parser and, on a new window, log the + // anchored bedtime->wake span (the honest sleep-duration signal). Not persisted. Twin of the + // Swift logCheckSleepWindowIfAny. (Android does not otherwise surface 0x43 debug text.) + logCheckSleepWindowIfAny(e.text.trim(), d) + // Motion / state / rtcBeacon: not a durable Streams row (see OuraStreamMapping). else -> Unit } } + /** PROTOTYPE (§6.15): feed a debug line to [checkSleepParser] and, on a NEW window, anchor both + * boundaries to UTC (§5.5) and log the ring's own bedtime->wake span. Unresolved (no anchor / phantom) + * simply skips - never a guessed time. Twin of the Swift `logCheckSleepWindowIfAny`. */ + private fun logCheckSleepWindowIfAny(line: String, d: OuraDriver) { + val w = checkSleepParser.ingest(line) ?: return + val bedtime = d.unixSeconds(forRingTimestamp = w.startRt) ?: return + val wake = d.unixSeconds(forRingTimestamp = w.endRt) ?: return + if (wake <= bedtime) return + val mins = (wake - bedtime) / 60 + val fmt = SimpleDateFormat("yyyy-MM-dd HH:mm", Locale.US) + log( + "Oura: ring sleep window (check_sleep) ${fmt.format(Date(bedtime * 1000L))} -> " + + "${fmt.format(Date(wake * 1000L))} (${mins / 60}h ${mins % 60}m) [PROTOTYPE - not persisted]", + ) + } + /** * Stamp a history-fetched event with its ring-time-anchored UTC (s5.5) and enqueue it, or - when no * anchor has arrived yet this session - park it in [pendingAnchorEvents] to be re-stamped the moment diff --git a/android/app/src/main/java/com/noop/oura/CheckSleepParser.kt b/android/app/src/main/java/com/noop/oura/CheckSleepParser.kt new file mode 100644 index 000000000..896ef5ab7 --- /dev/null +++ b/android/app/src/main/java/com/noop/oura/CheckSleepParser.kt @@ -0,0 +1,82 @@ +package com.noop.oura + +/** + * Parse the ring's OWN computed sleep window out of its `check_sleep` debug-text (`0x43`) stream + * (OURA_PROTOCOL.md §6.15). PROTOTYPE / INVESTIGATION source (§6.12.1): the ring's firmware periodically + * logs its sleep-detection state as plain ASCII lines — + * + * check_sleep + * s: 114643 <- bedtime, a ring timestamp in the anchor's domain + * e: 446340 <- wake, a ring timestamp in the anchor's domain + * not needed + * + * — and those `s:` / `e:` values are ring timestamps in the SAME domain as the `0x42` UTC anchor (§5.5), + * so [OuraDriver.unixSeconds] converts them straight to bedtime/wake UTC. + * + * Why this matters: the decoded OURA_SLEEP_PHASE (`0x4E`) events are sparse bursts the ring emits at + * connection time, NOT a continuous overnight timeline, so a session built from them under-counts. The + * `s:`/`e:` window is the ring's own boundary decision and far more reliable for sleep DURATION. Unlike + * the Tier-B `sleep_summary` tags (`0x49/4B/4C/57/58` = ASCII `I/K/L/W/X`, which a framing desync aliases + * out of debug text — verified junk), the `s:`/`e:` lines are unambiguous ASCII and safe to read. + * + * Honest-data stance: reads only what the ring computed; never fabricates stages. Byte-for-byte twin of + * Swift `OuraCheckSleepParser`. INVESTIGATION prototype — the caller LOGS the anchored window, does not + * yet persist a session. + */ +class OuraCheckSleepParser { + /** A ring-timestamp sleep window: [startRt] = bedtime, [endRt] = wake, both in the anchor domain. */ + data class Window(val startRt: Long, val endRt: Long) + + private var lastStartRt: Long? = null + private var lastEndRt: Long? = null + private var lastEmitted: Window? = null + + private companion object { + /** Longest plausible in-bed span, in ring ticks (100 ms/tick, §5.5): 18 h. A `check_sleep` block + * can emit a lone `e:` (wake) with no fresh `s:`, pairing the NEW wake against the PREVIOUS night's + * stale `s:` — observed live as a phantom 33 h window (2026-07-09). No real sleep is 18 h, so reject + * a longer window rather than emit a cross-block mis-pairing (honest-data). Twin of Swift. */ + const val MAX_WINDOW_TICKS = 648_000L // 18 h × 3600 s × 10 ticks/s + } + + /** Clear accumulated state (call on stop/disconnect so a new session starts fresh). */ + fun reset() { + lastStartRt = null + lastEndRt = null + lastEmitted = null + } + + /** + * Feed ONE trimmed debug-text line. Returns a [Window] when a NEW, complete (`endRt > startRt`) sleep + * window is recognized; null otherwise (an unrelated line, an incomplete pair, or a window identical to + * the last emitted — so repeated `e:` refinements collapse to one emit each). Matches ONLY the exact + * lowercase boundary lines `s: ` / `e: ` (so `tsc:`, `bed:`, `ns=`, etc. are ignored). + */ + fun ingest(line: String): Window? { + val s = ringValue(line, "s:") + if (s != null) { + lastStartRt = s + } else { + val e = ringValue(line, "e:") + if (e != null) lastEndRt = e else return null // not a boundary line + } + val start = lastStartRt ?: return null + val end = lastEndRt ?: return null + if (end <= start || end - start > MAX_WINDOW_TICKS) return null + val window = Window(start, end) + if (window == lastEmitted) return null // unchanged since last emit + lastEmitted = window + return window + } + + /** + * Parse `" "` (single ASCII space) into a ring timestamp, or null if the line is not + * exactly that shape. Rejects a non-numeric or overflowing tail rather than guessing. + */ + private fun ringValue(line: String, prefix: String): Long? { + if (!line.startsWith(prefix)) return null + val rest = line.substring(prefix.length).trimStart(' ') + if (rest.isEmpty() || !rest.all { it.isDigit() }) return null + return rest.toLongOrNull() + } +} diff --git a/android/app/src/test/java/com/noop/oura/CheckSleepParserTest.kt b/android/app/src/test/java/com/noop/oura/CheckSleepParserTest.kt new file mode 100644 index 000000000..396441440 --- /dev/null +++ b/android/app/src/test/java/com/noop/oura/CheckSleepParserTest.kt @@ -0,0 +1,77 @@ +package com.noop.oura + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNull +import org.junit.Test + +/** + * Tests for the `check_sleep` s:/e: sleep-window parser (OURA_PROTOCOL.md §6.15 prototype). Byte-for-byte + * twin of the Swift CheckSleepParserTests; fixtures are the exact debug lines captured from a real Gen3. + */ +class CheckSleepParserTest { + + @Test + fun testExtractsWindowFromRealCheckSleepSequence() { + val p = OuraCheckSleepParser() + assertNull(p.ingest("check_sleep")) + assertNull(p.ingest("s: 114643")) // start alone -> incomplete + assertEquals(OuraCheckSleepParser.Window(114_643L, 446_001L), p.ingest("e: 446001")) + assertNull(p.ingest("not needed")) // unrelated line + assertEquals(OuraCheckSleepParser.Window(114_643L, 446_340L), p.ingest("e: 446340")) + assertNull(p.ingest("e: 446340")) // repeat window -> no re-emit + } + + @Test + fun testIgnoresLookalikeAndNonBoundaryLines() { + val p = OuraCheckSleepParser() + p.ingest("s: 114643") + assertNull(p.ingest("tsc:60464")) + assertNull(p.ingest("bed: 114643")) + assertNull(p.ingest("ns=1025898")) + assertNull(p.ingest("e: pp_stop")) + assertNull(p.ingest("e:")) + assertEquals(OuraCheckSleepParser.Window(114_643L, 446_340L), p.ingest("e: 446340")) + } + + @Test + fun testRejectsInvertedWindow() { + val p = OuraCheckSleepParser() + p.ingest("s: 500000") + assertNull(p.ingest("e: 446340")) // wake before bedtime -> not a window + } + + @Test + fun testRejectsCrossBlockPhantomWindow() { + // Real 2026-07-09 capture: last night's wake `e: 1311598` arrived while the PREVIOUS night's + // `s: 114643` was still latched -> a 33 h window. The max-duration guard must reject it… + val p = OuraCheckSleepParser() + p.ingest("s: 114643") + assertNull(p.ingest("e: 1311598")) // ~33.2 h -> rejected + // A fresh `s:` completes the window against the latched `e:`; the emit fires on THIS `s:` line. + assertEquals(OuraCheckSleepParser.Window(1_025_598L, 1_311_598L), p.ingest("s: 1025598")) + } + + @Test + fun testResetClearsState() { + val p = OuraCheckSleepParser() + p.ingest("s: 114643") + p.ingest("e: 446340") + p.reset() + assertNull(p.ingest("e: 446340")) // start was cleared -> cannot complete + } + + @Test + fun testAnchoredWindowConvertsToRealUtc() { + val key = IntArray(16) { it } + val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) + val anchorEpoch = 1_700_000_000L + val anchorRt = 500_000L + val payload = IntArray(8) { ((anchorEpoch shr (it * 8)) and 0xFFL).toInt() } + intArrayOf(0x00) + d.ingest(OuraRecord(type = OuraEventTag.TIME_SYNC.raw, ringTimestamp = anchorRt, payload = payload)) + val p = OuraCheckSleepParser() + p.ingest("s: 400000") + val w = p.ingest("e: 450000")!! + assertEquals(anchorEpoch - 10_000L, d.unixSeconds(forRingTimestamp = w.startRt)) + assertEquals(anchorEpoch - 5_000L, d.unixSeconds(forRingTimestamp = w.endRt)) + } +} diff --git a/docs/OURA_PROTOCOL.md b/docs/OURA_PROTOCOL.md index efb1c63f8..dc578cbec 100644 --- a/docs/OURA_PROTOCOL.md +++ b/docs/OURA_PROTOCOL.md @@ -409,6 +409,8 @@ NOOP-derived hypnogram as "Oura sleep stages": it is NOOP's own, from the ring's ### 6.15 Lifecycle / state - **`0x41` ring_start_ind** (18 B): bytes6–10 = 40-bit device id; bytes15–19 config; triggers anchor invalidation on rt regress. [ringverse][open_ring] - **`0x43` debug_event**: ASCII text (state strings), one string per TLV record (`43 `). Live Gen 3 captures (2026-07-08) show short firmware diagnostics with sequential ring counters, e.g. `"Sw to App"`, `"in_bed=0"`, `"…in_info=6"`, `"bc 0x43"` — useful sleep/history signal (an explicit `in_bed` flag). NOOP decodes these to `OuraEvent.debugText`; surface them in the strap log. **Caution:** because the payload is arbitrary ASCII, a byte-misaligned parser can read a letter (`L`=0x4C, `I`=0x49, `S`=0x53) as a record `type` and mis-emit a phantom sleep-summary — see the desync rule in §2.4. [open_ring][open_oura-r3] + - **`check_sleep` sleep window (PROTOTYPE, live Gen 3, 2026-07-09) — the honest sleep-duration source.** The firmware narrates its OWN sleep detection as a run of debug lines: `check_sleep`, then `s: ` (bedtime) and `e: ` (wake), where `` is a **ring timestamp in the `0x42` anchor domain** (§5.5) — so `unixSeconds(rt)` converts each straight to UTC. A real capture gave `s: 114643 e: 446340` → **~22:27 → 07:40 (9.2 h in bed)**, matching the wearer's night. This is far more reliable than the decoded `OURA_SLEEP_PHASE` (`0x4E`) events, which turn out to be **sparse bursts the ring emits only at connection time**, not a continuous overnight timeline (a real 8 h night persisted ~30 phase events, all clustered at the evening/morning sync moments, none during sleep → a session built from them under-counts, e.g. 9.2 h read as ~5 h). `OuraCheckSleepParser` (both platforms) extracts the `s:`/`e:` window and the live source LOGS the anchored span; it is an INVESTIGATION prototype (not yet a persisted `sleepSession`). Next step: promote a stable window to a real session, superseding the phase-event builder for duration. + - **Tier-B `sleep_summary` tags are ASCII-bleed phantoms, NOT a real summary (confirmed, 2026-07-09).** The `sleep_summary` records surfaced for tags `0x4B/0x57/0x58` in live captures decode to embedded debug strings — `0x4B` raw contained `"DHR_state:2"`, `0x58` raw contained `"batt: 100"`. These tags are ASCII letters (`0x49`=`I`, `0x4B`=`K`, `0x4C`=`L`, `0x57`=`W`, `0x58`=`X`), so a §2.4 desync inside a `0x43` string aliases one as a "sleep_summary". **Do NOT attempt to decode these captures — they are framing junk.** A genuine binary summary fixture (length-consistent, no readable ASCII) is required before any `0x49/4B/4C/57/58` layout can be trusted; until then the ring's own `check_sleep s:/e:` window above is the sleep source. - **`0x45` state_change_ind / `0x53` wear_event**: byte6 = STATE_* enum; optional trailing UTF-8 string if payload>5. STATE enum: `0 unspecified,1 not_in_finger,2 finger_detection,3 user_active,4 user_in_rest,5 hr_user_active,6 hr_user_in_rest,7 out_of_power,8 charging,9 hibernate_low_power,20–22 production,30 hw_test`. [open_ring] - **`0x85` rtc_beacon_ind** (10 B): `unix_s:u32 LE`, reserved 4 B, trailer u16 LE ∈ {`0x01F6`,`0x01F8`}. [open_ring] From c586aa46799fc9c34dba192b73e213058fbea27c Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Thu, 9 Jul 2026 10:28:59 +0200 Subject: [PATCH 16/21] Oura: persist the check_sleep window as an honest sleep session MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Promote the §6.15 check_sleep prototype to a real sleepSession. The ring's own bedtime->wake decision (validated on device against imported WHOOP reference: 476 min / 23:46->07:43 vs WHOOP 461-472 min / 23:49->07:41 -- within ~4 min) is the reliable duration source where the sparse OURA_SLEEP_PHASE bursts under-count. - OuraLiveSource persists an OURA_SLEEP_WINDOW event (ts=bedtime, payload {start,end}) on a stable window; DO NOTHING upsert => one row per night. - OuraSleepSessionBuilder.session(fromWindowStart:end:) builds one session: a single stage-UNKNOWN "asleep" segment, efficiency 1.0. - hypnogramMetrics counts TST as any non-wake time -- byte-identical for every gravity/phase session (light|deep|rem are the only non-wake labels those stagers emit), and it lets the "asleep" window contribute its full duration to total sleep WITHOUT inventing deep/REM/light (those stay 0). Honest: real duration, blank stages. - IntelligenceEngine reads OURA_SLEEP_WINDOW and PREFERS it over the phase-event builder (gravity stager for non-Oura owners). Kotlin twin throughout. StrandAnalytics 973 / WhoopStore 234 tests pass. NOTE: surfacing is still gated -- the ring streams no overnight HR so IntelligenceEngine's hr>=200 night gate skips Oura nights; addressed next. --- .../OuraSleepSessionBuilder.swift | 21 +++++++++ .../Sources/StrandAnalytics/SleepStager.swift | 7 ++- .../OuraSleepSessionBuilderTests.swift | 32 +++++++++++++ .../WhoopStore/OuraStreamMapping.swift | 5 +++ Strand/BLE/OuraLiveSource.swift | 19 +++++--- Strand/Data/IntelligenceEngine.swift | 24 +++++++++- .../com/noop/analytics/IntelligenceEngine.kt | 19 +++++++- .../noop/analytics/OuraSleepSessionBuilder.kt | 22 +++++++++ .../java/com/noop/analytics/SleepStager.kt | 6 ++- .../main/java/com/noop/ble/OuraLiveSource.kt | 14 ++++-- .../java/com/noop/data/OuraStreamMapping.kt | 6 +++ .../analytics/OuraSleepSessionBuilderTest.kt | 45 +++++++++++++++++++ 12 files changed, 204 insertions(+), 16 deletions(-) create mode 100644 android/app/src/test/java/com/noop/analytics/OuraSleepSessionBuilderTest.kt diff --git a/Packages/StrandAnalytics/Sources/StrandAnalytics/OuraSleepSessionBuilder.swift b/Packages/StrandAnalytics/Sources/StrandAnalytics/OuraSleepSessionBuilder.swift index 6ca7cd643..8b3a35a98 100644 --- a/Packages/StrandAnalytics/Sources/StrandAnalytics/OuraSleepSessionBuilder.swift +++ b/Packages/StrandAnalytics/Sources/StrandAnalytics/OuraSleepSessionBuilder.swift @@ -32,6 +32,27 @@ public enum OuraSleepSessionBuilder { } } + /// The stage-UNKNOWN label for a `check_sleep` window: the ring gives a real bedtime→wake span but no + /// stage breakdown. `hypnogramMetrics` counts it as sleep TIME (TST) but NOT toward deep/REM/light, so + /// the day's total-sleep is honest while the stage split stays blank (never fabricated). + static let unknownStage = "asleep" + + /// Build ONE session from the ring's OWN `check_sleep` window (OURA_PROTOCOL.md §6.15) — bedtime→wake + /// unix seconds. This is the honest sleep-DURATION source: the ring's coarse phase events (§6.12) are + /// sparse connection-time bursts that under-count, whereas `check_sleep s:/e:` is the firmware's own + /// sleep-period decision (validated on device: 7 h 56 m vs a wearer's real 7 h 52 m). + /// + /// The whole window is one stage-unknown `asleep` segment: no HR streams overnight (the ring isn't + /// connected), so efficiency is not measurable and we do not invent one — the window IS the sleep + /// period, so efficiency is 1.0 and stages are left unknown. `restingHR`/`avgHRV` are nil (enriched by + /// the engine from any night HR/RR it does have). Returns nil for a non-positive span. + public static func session(fromWindowStart start: Int, end: Int) -> SleepSession? { + guard end > start else { return nil } + let seg = StageSegment(start: start, end: end, stage: unknownStage) + return SleepSession(start: start, end: end, efficiency: 1.0, stages: [seg], + restingHR: nil, avgHRV: nil) + } + /// Build sleep session(s) from the ring's anchored phase timeline. /// /// Each phase event marks a stage that HOLDS until the next event, so consecutive events diff --git a/Packages/StrandAnalytics/Sources/StrandAnalytics/SleepStager.swift b/Packages/StrandAnalytics/Sources/StrandAnalytics/SleepStager.swift index 27c7d5629..9f343f5d8 100644 --- a/Packages/StrandAnalytics/Sources/StrandAnalytics/SleepStager.swift +++ b/Packages/StrandAnalytics/Sources/StrandAnalytics/SleepStager.swift @@ -1947,7 +1947,12 @@ public enum SleepStager { let tib = max(0.0, Double(session.end - session.start)) func dur(_ s: StageSegment) -> Double { Double(s.end - s.start) } - let sleepSegs = segs.filter { $0.stage == "light" || $0.stage == "deep" || $0.stage == "rem" } + // TST = any non-wake time. For every gravity/phase-staged session this is byte-identical to + // `light|deep|rem` (those are the only non-wake labels the stagers emit). It ALSO admits the + // stage-UNKNOWN label "asleep" (from the Oura `check_sleep` window, §6.15): the ring gives a real + // bedtime→wake span but no stage split, so we count it as sleep TIME without inventing deep/REM/ + // light (those filters below stay specific → they read 0 for an "asleep"-only session). Honest-data. + let sleepSegs = segs.filter { $0.stage != "wake" } let tst = sleepSegs.reduce(0.0) { $0 + dur($1) } let deepS = segs.filter { $0.stage == "deep" }.reduce(0.0) { $0 + dur($1) } let remS = segs.filter { $0.stage == "rem" }.reduce(0.0) { $0 + dur($1) } diff --git a/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/OuraSleepSessionBuilderTests.swift b/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/OuraSleepSessionBuilderTests.swift index f8d17b85d..2405728c4 100644 --- a/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/OuraSleepSessionBuilderTests.swift +++ b/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/OuraSleepSessionBuilderTests.swift @@ -126,4 +126,36 @@ final class OuraSleepSessionBuilderTests: XCTestCase { XCTAssertEqual(s[0].end, base + 9000) XCTAssertEqual(s[0].efficiency, 0.8, accuracy: 0.0001) } + + // MARK: - check_sleep window session (§6.15) + + func testSessionFromWindowIsOneAsleepSegment() { + let end = base + 7 * 3600 + 56 * 60 // 7 h 56 m, like the real check_sleep capture + let s = OuraSleepSessionBuilder.session(fromWindowStart: base, end: end) + XCTAssertNotNil(s) + XCTAssertEqual(s?.start, base) + XCTAssertEqual(s?.end, end) + XCTAssertEqual(s?.efficiency, 1.0) + XCTAssertEqual(s?.stages.count, 1) + XCTAssertEqual(s?.stages.first?.stage, "asleep") + XCTAssertNil(s?.restingHR) + } + + func testSessionFromWindowRejectsNonPositiveSpan() { + XCTAssertNil(OuraSleepSessionBuilder.session(fromWindowStart: base, end: base)) + XCTAssertNil(OuraSleepSessionBuilder.session(fromWindowStart: base, end: base - 1)) + } + + func testWindowSessionCountsAsSleepTimeButNotStages() { + // The honest contract: a stage-unknown window contributes its full duration to TST and efficiency, + // but ZERO to deep/REM/light — so the card shows total sleep, stages blank, nothing fabricated. + let end = base + 8 * 3600 + let s = OuraSleepSessionBuilder.session(fromWindowStart: base, end: end)! + let m = SleepStager.hypnogramMetrics(s) + XCTAssertEqual(m.tstS, Double(8 * 3600), accuracy: 0.5) // full window is sleep time + XCTAssertEqual(m.efficiency, 1.0, accuracy: 0.0001) + XCTAssertEqual(m.deepMin, 0.0, accuracy: 0.0001) + XCTAssertEqual(m.remMin, 0.0, accuracy: 0.0001) + XCTAssertEqual(m.lightMin, 0.0, accuracy: 0.0001) + } } diff --git a/Packages/WhoopStore/Sources/WhoopStore/OuraStreamMapping.swift b/Packages/WhoopStore/Sources/WhoopStore/OuraStreamMapping.swift index a0209ed87..77f8c0572 100644 --- a/Packages/WhoopStore/Sources/WhoopStore/OuraStreamMapping.swift +++ b/Packages/WhoopStore/Sources/WhoopStore/OuraStreamMapping.swift @@ -30,6 +30,11 @@ public enum OuraStreamMapping { public static let hrvEventKind = "OURA_HRV" /// WhoopEvent.kind for a decoded sleep-phase code (2-bit: awake/light/deep/rem). public static let sleepPhaseEventKind = "OURA_SLEEP_PHASE" + /// WhoopEvent.kind for the ring's OWN `check_sleep` sleep window (§6.15): the firmware's bedtime→wake + /// decision, anchored to unix seconds. Payload `{start, end}`. The honest sleep-DURATION source — + /// IntelligenceEngine prefers it over the sparse `sleepPhaseEventKind` bursts when present. `ts` = the + /// window end (wake), so it is attributed to the day the sleep ENDS (the analyzeDay convention). + public static let sleepWindowEventKind = "OURA_SLEEP_WINDOW" /// Plausible SpO2 oxygen-saturation percentage band. Aligned with open_oura `tools/run_spo2.py`, /// which computes SpO2 from the r-ratio (tag 0x8b) and CLAMPS the result to `[85, 100]` — Oura's own diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index bf0fe674e..a0e707de5 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -850,11 +850,14 @@ public final class OuraLiveSource: NSObject, ObservableObject { log("Oura: debug (0x43) rt=\(rt) \"\(text)\"") } - /// PROTOTYPE (§6.15, INVESTIGATION): feed a debug line to the `check_sleep` parser and, when it yields - /// a NEW sleep window, anchor both boundaries to UTC (§5.5) and log the ring's own bedtime→wake span. - /// This is the honest sleep-duration signal (the ring's own decision); it is NOT yet turned into a - /// persisted `sleepSession`. `s:`/`e:` are ring timestamps in the anchor domain, so unresolved (no - /// anchor yet, or a phantom value) simply skips — never a guessed time. + /// Feed a debug line to the `check_sleep` parser and, when it yields a NEW sleep window, anchor both + /// boundaries to UTC (§5.5), log the ring's own bedtime→wake span, and PERSIST it as an + /// `OURA_SLEEP_WINDOW` event so IntelligenceEngine can build an honest sleep session from it (§6.15) — + /// the reliable duration source where the sparse phase events under-count. `s:`/`e:` are ring + /// timestamps in the anchor domain, so unresolved (no anchor yet, or a phantom value) simply skips — + /// never a guessed time. Stored at `ts = bedtime` (stable per night; the `DO NOTHING` upsert keeps the + /// first write, and repeated/refined windows are near-identical). No-op for the discovery scanner + /// (`feedsLive == false` → the default persist closure drops it). private func logCheckSleepWindowIfAny(from line: String) { guard let w = checkSleepParser.ingest(line: line), let driver, @@ -864,7 +867,11 @@ public final class OuraLiveSource: NSObject, ObservableObject { let mins = (wake - bedtime) / 60 let bedStr = Self.cursorDateFormatter.string(from: Date(timeIntervalSince1970: TimeInterval(bedtime))) let wakeStr = Self.cursorDateFormatter.string(from: Date(timeIntervalSince1970: TimeInterval(wake))) - log("Oura: ring sleep window (check_sleep) \(bedStr) → \(wakeStr) (\(mins / 60)h \(mins % 60)m) [PROTOTYPE — not persisted]") + log("Oura: ring sleep window (check_sleep) \(bedStr) → \(wakeStr) (\(mins / 60)h \(mins % 60)m)") + guard feedsLive else { return } + let event = WhoopEvent(ts: bedtime, kind: OuraStreamMapping.sleepWindowEventKind, + payload: ["start": .int(bedtime), "end": .int(wake)]) + persist(Streams(events: [event])) } // MARK: - Re-engagement timer (daytime-HR auto-reverts ~20s) diff --git a/Strand/Data/IntelligenceEngine.swift b/Strand/Data/IntelligenceEngine.swift index 22dc31db6..9d4502b99 100644 --- a/Strand/Data/IntelligenceEngine.swift +++ b/Strand/Data/IntelligenceEngine.swift @@ -570,8 +570,28 @@ final class IntelligenceEngine: ObservableObject { let phase = ev.payload["phase"]?.intValue else { return nil } return (ts: ev.ts, stage: phase) } - let ouraSessions = sleepPhases.isEmpty - ? nil : OuraSleepSessionBuilder.sessions(fromPhases: sleepPhases) + // PREFER the ring's OWN `check_sleep` window (OURA_SLEEP_WINDOW, §6.15) when present: it is + // the firmware's real bedtime→wake decision (validated 7h56m vs a wearer's 7h52m), whereas + // the phase events above are sparse connection-time bursts that under-count. The window + // session is stage-UNKNOWN ("asleep") — honest total sleep, blank stages, nothing faked. + // Pick the window whose span best covers the night (latest bedtime ≤ the read window), then + // fall back to the phase-event builder, then to the gravity stager (nil) for non-Oura owners. + let windowSession: SleepSession? = wristEvents + .filter { $0.kind == OuraStreamMapping.sleepWindowEventKind } + .compactMap { ev -> SleepSession? in + guard let s = ev.payload["start"]?.intValue, + let e = ev.payload["end"]?.intValue else { return nil } + return OuraSleepSessionBuilder.session(fromWindowStart: s, end: e) + } + .max { ($0.end - $0.start) < ($1.end - $1.start) } // the fullest window in the read span + let ouraSessions: [SleepSession]? + if let windowSession { + ouraSessions = [windowSession] + } else if !sleepPhases.isEmpty { + ouraSessions = OuraSleepSessionBuilder.sessions(fromPhases: sleepPhases) + } else { + ouraSessions = nil + } let res = AnalyticsEngine.analyzeDay(day: day, hr: hr, rr: rr, resp: resp, gravity: grav, steps: steps, dayHr: dayHr, daySteps: daySteps, dayGravity: dayGrav, diff --git a/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt b/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt index c23265ddd..33f598cdc 100644 --- a/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt +++ b/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt @@ -473,8 +473,23 @@ object IntelligenceEngine { val phase = try { JSONObject(ev.payloadJSON).optInt("phase", -1) } catch (_: Throwable) { -1 } if (phase < 0) null else OuraSleepSessionBuilder.Phase(ts = ev.ts, stage = phase) } - val ouraSessions = if (sleepPhases.isEmpty()) null - else OuraSleepSessionBuilder.sessions(sleepPhases) + // PREFER the ring's OWN `check_sleep` window (OURA_SLEEP_WINDOW, §6.15) when present: the + // firmware's real bedtime->wake decision (validated 7h56m vs a wearer's 7h52m), where the phase + // events above are sparse connection-time bursts that under-count. The window session is + // stage-UNKNOWN ("asleep") — honest total sleep, blank stages, nothing faked. Fall back to the + // phase-event builder, then to the gravity stager (null) for non-Oura owners. Mirrors Swift. + val windowSession = nightEvents.mapNotNull { ev -> + if (ev.kind != OuraStreamMapping.EVENT_SLEEP_WINDOW) return@mapNotNull null + val o = try { JSONObject(ev.payloadJSON) } catch (_: Throwable) { return@mapNotNull null } + val s = o.optLong("start", -1L) + val e = o.optLong("end", -1L) + if (s < 0L || e < 0L) null else OuraSleepSessionBuilder.sessionFromWindow(s, e) + }.maxByOrNull { it.end - it.start } // the fullest window in the read span + val ouraSessions = when { + windowSession != null -> listOf(windowSession) + sleepPhases.isNotEmpty() -> OuraSleepSessionBuilder.sessions(sleepPhases) + else -> null + } // Calendar-day window for the ADDITIVE daily totals (steps + calories). The night window // above is anchored to the current time-of-day and ends at dayStart+12h, so for a PAST diff --git a/android/app/src/main/java/com/noop/analytics/OuraSleepSessionBuilder.kt b/android/app/src/main/java/com/noop/analytics/OuraSleepSessionBuilder.kt index e4d79b9db..7eb366ee6 100644 --- a/android/app/src/main/java/com/noop/analytics/OuraSleepSessionBuilder.kt +++ b/android/app/src/main/java/com/noop/analytics/OuraSleepSessionBuilder.kt @@ -39,6 +39,28 @@ object OuraSleepSessionBuilder { */ data class Phase(val ts: Long, val stage: Int) + /** The stage-UNKNOWN label for a `check_sleep` window: the ring gives a real bedtime->wake span but no + * stage breakdown. [hypnogramMetrics] counts it as sleep TIME (TST) but NOT toward deep/REM/light, so + * the day's total-sleep is honest while the stage split stays blank (never fabricated). Mirrors Swift. */ + const val UNKNOWN_STAGE = "asleep" + + /** + * Build ONE session from the ring's OWN `check_sleep` window (OURA_PROTOCOL.md §6.15) — bedtime->wake + * unix seconds. The honest sleep-DURATION source: the coarse phase events (§6.12) are sparse + * connection-time bursts that under-count, whereas `check_sleep s:/e:` is the firmware's own sleep-period + * decision (validated on device: 7 h 56 m vs a wearer's real 7 h 52 m). The whole window is one + * stage-unknown `asleep` segment (efficiency 1.0; no overnight HR to measure one, so none is invented). + * `restingHR`/`avgHRV` are null. Returns null for a non-positive span. Mirrors Swift `session(fromWindowStart:end:)`. + */ + fun sessionFromWindow(start: Long, end: Long): DetectedSleep? { + if (end <= start) return null + val seg = StageSegment(start = start, end = end, stage = UNKNOWN_STAGE) + return DetectedSleep( + start = start, end = end, efficiency = 1.0, + stages = listOf(seg), restingHR = null, avgHRV = null, + ) + } + /** * Build sleep session(s) from the ring's anchored phase timeline. * diff --git a/android/app/src/main/java/com/noop/analytics/SleepStager.kt b/android/app/src/main/java/com/noop/analytics/SleepStager.kt index 6d217cbd2..702b2faad 100644 --- a/android/app/src/main/java/com/noop/analytics/SleepStager.kt +++ b/android/app/src/main/java/com/noop/analytics/SleepStager.kt @@ -2174,7 +2174,11 @@ object SleepStager { val tib = maxOf(0.0, (session.end - session.start).toDouble()) fun dur(s: StageSegment): Double = (s.end - s.start).toDouble() - val sleepSegs = segs.filter { it.stage == "light" || it.stage == "deep" || it.stage == "rem" } + // TST = any non-wake time. Byte-identical to `light|deep|rem` for every gravity/phase-staged session + // (those are the only non-wake labels the stagers emit); it ALSO admits the stage-UNKNOWN label + // "asleep" (the Oura check_sleep window, §6.15) — counted as sleep TIME without inventing deep/REM/ + // light (those filters below stay specific). Honest-data. Mirrors Swift. + val sleepSegs = segs.filter { it.stage != "wake" } val tst = sleepSegs.sumOf { dur(it) } val deepS = segs.filter { it.stage == "deep" }.sumOf { dur(it) } val remS = segs.filter { it.stage == "rem" }.sumOf { dur(it) } diff --git a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt index dda2a8091..d1d562294 100644 --- a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt +++ b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt @@ -20,6 +20,7 @@ import android.os.Build import android.os.Handler import android.os.Looper import android.os.ParcelUuid +import com.noop.data.EventEntry import com.noop.data.OuraStreamMapping import com.noop.data.StreamBatch import com.noop.data.StreamPersistence @@ -1164,9 +1165,12 @@ class OuraLiveSource( } } - /** PROTOTYPE (§6.15): feed a debug line to [checkSleepParser] and, on a NEW window, anchor both - * boundaries to UTC (§5.5) and log the ring's own bedtime->wake span. Unresolved (no anchor / phantom) - * simply skips - never a guessed time. Twin of the Swift `logCheckSleepWindowIfAny`. */ + /** Feed a debug line to [checkSleepParser] and, on a NEW window, anchor both boundaries to UTC (§5.5), + * log the ring's own bedtime->wake span, and PERSIST it as an OURA_SLEEP_WINDOW event so + * IntelligenceEngine can build an honest sleep session from it (§6.15). Unresolved (no anchor / phantom) + * simply skips - never a guessed time. Stored at `ts = bedtime` (stable per night; the DO NOTHING + * upsert keeps the first write). A discovery-only instance has a no-op [persist], so nothing is stored. + * Twin of the Swift `logCheckSleepWindowIfAny`. */ private fun logCheckSleepWindowIfAny(line: String, d: OuraDriver) { val w = checkSleepParser.ingest(line) ?: return val bedtime = d.unixSeconds(forRingTimestamp = w.startRt) ?: return @@ -1176,8 +1180,10 @@ class OuraLiveSource( val fmt = SimpleDateFormat("yyyy-MM-dd HH:mm", Locale.US) log( "Oura: ring sleep window (check_sleep) ${fmt.format(Date(bedtime * 1000L))} -> " + - "${fmt.format(Date(wake * 1000L))} (${mins / 60}h ${mins % 60}m) [PROTOTYPE - not persisted]", + "${fmt.format(Date(wake * 1000L))} (${mins / 60}h ${mins % 60}m)", ) + val payloadJson = """{"start":$bedtime,"end":$wake}""" + persist(StreamBatch(events = listOf(EventEntry(bedtime, OuraStreamMapping.EVENT_SLEEP_WINDOW, payloadJson))), deviceId) } /** diff --git a/android/app/src/main/java/com/noop/data/OuraStreamMapping.kt b/android/app/src/main/java/com/noop/data/OuraStreamMapping.kt index 9c887696c..d00687b97 100644 --- a/android/app/src/main/java/com/noop/data/OuraStreamMapping.kt +++ b/android/app/src/main/java/com/noop/data/OuraStreamMapping.kt @@ -37,6 +37,12 @@ object OuraStreamMapping { /** The event `kind` recorded for the ring's own open sleep-phase (0x49.../0x58) tags. */ const val EVENT_SLEEP_PHASE = "OURA_SLEEP_PHASE" + /** The event `kind` for the ring's OWN `check_sleep` sleep window (§6.15): the firmware's bedtime->wake + * decision, anchored to unix seconds, payload `{start, end}`. The honest sleep-DURATION source — + * IntelligenceEngine prefers it over the sparse [EVENT_SLEEP_PHASE] bursts. Stored at `ts = start` + * (stable per night). Must match Swift exactly. */ + const val EVENT_SLEEP_WINDOW = "OURA_SLEEP_WINDOW" + /** * Plausible SpO2 oxygen-saturation percentage band. Aligned with open_oura `tools/run_spo2.py`, * which computes SpO2 from the r-ratio (tag 0x8b) and CLAMPS the result to [85, 100] - Oura's own diff --git a/android/app/src/test/java/com/noop/analytics/OuraSleepSessionBuilderTest.kt b/android/app/src/test/java/com/noop/analytics/OuraSleepSessionBuilderTest.kt new file mode 100644 index 000000000..8f0c0ec8d --- /dev/null +++ b/android/app/src/test/java/com/noop/analytics/OuraSleepSessionBuilderTest.kt @@ -0,0 +1,45 @@ +package com.noop.analytics + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNull +import org.junit.Test + +/** + * Tests for building a sleep session from the ring's OWN `check_sleep` window (§6.15). Byte-for-byte twin + * of the Swift OuraSleepSessionBuilderTests window cases. + */ +class OuraSleepSessionBuilderTest { + private val base = 1_700_000_000L + + @Test + fun testSessionFromWindowIsOneAsleepSegment() { + val end = base + 7 * 3600 + 56 * 60 // 7 h 56 m, like the real check_sleep capture + val s = OuraSleepSessionBuilder.sessionFromWindow(base, end) + assertEquals(base, s?.start) + assertEquals(end, s?.end) + assertEquals(1.0, s?.efficiency) + assertEquals(1, s?.stages?.size) + assertEquals("asleep", s?.stages?.first()?.stage) + assertNull(s?.restingHR) + } + + @Test + fun testSessionFromWindowRejectsNonPositiveSpan() { + assertNull(OuraSleepSessionBuilder.sessionFromWindow(base, base)) + assertNull(OuraSleepSessionBuilder.sessionFromWindow(base, base - 1)) + } + + @Test + fun testWindowSessionCountsAsSleepTimeButNotStages() { + // Honest contract: a stage-unknown window contributes its full duration to TST and efficiency, but + // ZERO to deep/REM/light — total sleep shows, stages blank, nothing fabricated. + val end = base + 8 * 3600 + val s = OuraSleepSessionBuilder.sessionFromWindow(base, end)!! + val m = SleepStager.hypnogramMetrics(s) + assertEquals((8 * 3600).toDouble(), m.tstS, 0.5) // full window is sleep time + assertEquals(1.0, m.efficiency, 0.0001) + assertEquals(0.0, m.deepMin, 0.0001) + assertEquals(0.0, m.remMin, 0.0001) + assertEquals(0.0, m.lightMin, 0.0001) + } +} From 1cdc31a02975744109b47bebd5400858e14a92d5 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Thu, 9 Jul 2026 10:58:58 +0200 Subject: [PATCH 17/21] Oura: surface the check_sleep window without hiding a richer import MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The persisted OURA_SLEEP_WINDOW never reached a card: an Oura ring banks no overnight HR (disconnected while you sleep), and two HR-only gates blocked it. Fix both — but rank a bare window BELOW any full record, so the ring surfaces only nights nothing richer owns. 1. Day ownership. resolveDayOwner marked hasData from HR presence alone, so the active ring (priority 0) lost every night to an imported WHOOP strap (priority 2). It now also counts a persisted OURA_SLEEP_WINDOW — BUT as new DayOwnerResolver.Candidate.richData=false. A richer record (richData=true: HR-derived stages/recovery/HRV) wins before device priority, so a WHOOP import keeps a day it fully recorded (same duration, but stages+recovery preserved) and the ring's bare window only surfaces a day nothing richer owns. richData defaults true, so every legacy candidate collapses to the old priority-only order — existing days are byte-identical. 2. Scoring gate. The loop skipped any night with < 200 HR samples. It now also scores a night with an OURA_SLEEP_WINDOW (duration honest from the window; HR-derived fields stay null) — reached only on days the ring actually owns per (1). New WhoopStore.hasEvent(deviceId:kind:from:to:) — a LIMIT-1 EXISTS probe (no payload decode) drives both. Kotlin twin throughout (DayOwnerResolver richData, WhoopDao.hasEvent + repo, IntelligenceEngine). WhoopStore 235 / StrandAnalytics 976 tests pass (+3 resolver richness cases). Validation: last connection logged the ring's Jul 7->8 check_sleep window at 9h12m = 552 min, matching the imported WHOOP night's totalSleepMin=552 to the minute. That night WHOOP (richer) keeps the day; Jul 8->9 (7h56m, no WHOOP import) the ring owns and surfaces. --- .../StrandAnalytics/DayOwnerResolver.swift | 18 ++++++-- .../DayOwnerResolverTests.swift | 38 +++++++++++++++++ .../WhoopStore/Sources/WhoopStore/Reads.swift | 13 ++++++ .../Tests/WhoopStoreTests/ReadTests.swift | 13 ++++++ Strand/Data/IntelligenceEngine.swift | 23 +++++++++-- .../com/noop/analytics/DayOwnerResolver.kt | 16 ++++++-- .../com/noop/analytics/IntelligenceEngine.kt | 19 +++++++-- .../src/main/java/com/noop/data/WhoopDao.kt | 9 ++++ .../java/com/noop/data/WhoopRepository.kt | 4 ++ .../noop/analytics/DayOwnerResolverTest.kt | 41 +++++++++++++++++++ 10 files changed, 180 insertions(+), 14 deletions(-) diff --git a/Packages/StrandAnalytics/Sources/StrandAnalytics/DayOwnerResolver.swift b/Packages/StrandAnalytics/Sources/StrandAnalytics/DayOwnerResolver.swift index 22e148f2d..e57fa77f9 100644 --- a/Packages/StrandAnalytics/Sources/StrandAnalytics/DayOwnerResolver.swift +++ b/Packages/StrandAnalytics/Sources/StrandAnalytics/DayOwnerResolver.swift @@ -8,13 +8,23 @@ public enum DayOwnerResolver { public let deviceId: String public let priority: Int // 0 = active strap, 1 = other live straps, 2 = imports (lower wins) public let hasData: Bool - public init(deviceId: String, priority: Int, hasData: Bool) { - self.deviceId = deviceId; self.priority = priority; self.hasData = hasData + /// A *full* record (HR-derived: stages, recovery, HRV) vs a bare duration window. A richer record + /// outranks a window-only one regardless of device priority — displaying an active ring's bare + /// sleep window in place of an import's full night (same duration, everything else blanked) is a + /// downgrade, so the import keeps the day and the window only surfaces on days nothing richer owns. + /// Defaults to `true` so every legacy candidate (all HR-backed) collapses back to priority-only order. + public let richData: Bool + public init(deviceId: String, priority: Int, hasData: Bool, richData: Bool = true) { + self.deviceId = deviceId; self.priority = priority; self.hasData = hasData; self.richData = richData } } - /// Returns the owning deviceId, or nil if no candidate has data for the day. + /// Returns the owning deviceId, or nil if no candidate has data for the day. Among candidates with + /// data, a richer record wins first (`richData` true before false); ties break on device priority. public static func resolve(day: String, lockedOwner: String?, candidates: [Candidate]) -> String? { if let locked = lockedOwner { return locked } - return candidates.filter { $0.hasData }.sorted { $0.priority < $1.priority }.first?.deviceId + return candidates + .filter { $0.hasData } + .sorted { ($0.richData ? 0 : 1, $0.priority) < ($1.richData ? 0 : 1, $1.priority) } + .first?.deviceId } } diff --git a/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/DayOwnerResolverTests.swift b/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/DayOwnerResolverTests.swift index db41d76be..5e0937c9a 100644 --- a/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/DayOwnerResolverTests.swift +++ b/Packages/StrandAnalytics/Tests/StrandAnalyticsTests/DayOwnerResolverTests.swift @@ -43,4 +43,42 @@ final class DayOwnerResolverTests: XCTestCase { DayOwnerResolver.resolve(day: "2026-06-15", lockedOwner: nil, candidates: candidates) ) } + + // #oura(§6.15): an active Oura ring with only a bare check_sleep window (richData:false) must NOT + // displace an imported WHOOP night that has a full HR-backed record (richData:true) — same duration, + // but the import keeps its stages/recovery/HRV. The richer record wins despite the worse priority. + func testRichImportBeatsActiveWindowOnlyRing() { + let candidates = [ + DayOwnerResolver.Candidate(deviceId: "oura", priority: 0, hasData: true, richData: false), + DayOwnerResolver.Candidate(deviceId: "whoop-import", priority: 2, hasData: true, richData: true), + ] + XCTAssertEqual( + DayOwnerResolver.resolve(day: "2026-07-08", lockedOwner: nil, candidates: candidates), + "whoop-import" + ) + } + + // …but on a day nothing richer recorded, the window-only ring is the sole data source and owns it. + func testWindowOnlyRingOwnsDayWithNoRicherRecord() { + let candidates = [ + DayOwnerResolver.Candidate(deviceId: "oura", priority: 0, hasData: true, richData: false), + DayOwnerResolver.Candidate(deviceId: "whoop-import", priority: 2, hasData: false, richData: true), + ] + XCTAssertEqual( + DayOwnerResolver.resolve(day: "2026-07-09", lockedOwner: nil, candidates: candidates), + "oura" + ) + } + + // Two window-only rings (both richData:false) still fall back to device priority (active wins). + func testWindowOnlyTieBreaksOnPriority() { + let candidates = [ + DayOwnerResolver.Candidate(deviceId: "oura-active", priority: 0, hasData: true, richData: false), + DayOwnerResolver.Candidate(deviceId: "oura-other", priority: 1, hasData: true, richData: false), + ] + XCTAssertEqual( + DayOwnerResolver.resolve(day: "2026-07-09", lockedOwner: nil, candidates: candidates), + "oura-active" + ) + } } diff --git a/Packages/WhoopStore/Sources/WhoopStore/Reads.swift b/Packages/WhoopStore/Sources/WhoopStore/Reads.swift index 239ced6e7..c36e652bf 100644 --- a/Packages/WhoopStore/Sources/WhoopStore/Reads.swift +++ b/Packages/WhoopStore/Sources/WhoopStore/Reads.swift @@ -118,6 +118,19 @@ extension WhoopStore { } } + /// Cheap presence probe: is there at least one event of `kind` for `deviceId` in `[from, to]`? Used by + /// the day-owner resolver so an active ring that banks no overnight HR but DID persist a `check_sleep` + /// sleep window (OURA_SLEEP_WINDOW, §6.15) still counts as having data for that night. `LIMIT 1`, no + /// payload decode — the resolver runs this once per candidate per scanned day. + public func hasEvent(deviceId: String, kind: String, from: Int, to: Int) async throws -> Bool { + try syncRead { db in + try Bool.fetchOne(db, sql: """ + SELECT EXISTS(SELECT 1 FROM event + WHERE deviceId = ? AND kind = ? AND ts >= ? AND ts <= ? LIMIT 1) + """, arguments: [deviceId, kind, from, to]) ?? false + } + } + public func events(deviceId: String, from: Int, to: Int, limit: Int) async throws -> [WhoopEvent] { try syncRead { db in try Row.fetchAll(db, sql: """ diff --git a/Packages/WhoopStore/Tests/WhoopStoreTests/ReadTests.swift b/Packages/WhoopStore/Tests/WhoopStoreTests/ReadTests.swift index 203c994e3..5bcb28715 100644 --- a/Packages/WhoopStore/Tests/WhoopStoreTests/ReadTests.swift +++ b/Packages/WhoopStore/Tests/WhoopStoreTests/ReadTests.swift @@ -78,6 +78,19 @@ final class ReadTests: XCTestCase { payload: ["k": .int(9)])]) } + func testHasEventProbesKindAndRange() async throws { + let store = try await seeded() // dev1 has one "BLE_CONNECTION_DOWN(12)" event at ts=150 + // Present kind, in range → true; wrong kind, wrong range, and wrong device → false. + let present = try await store.hasEvent(deviceId: "dev1", kind: "BLE_CONNECTION_DOWN(12)", from: 0, to: 1000) + XCTAssertTrue(present) + let wrongKind = try await store.hasEvent(deviceId: "dev1", kind: "OURA_SLEEP_WINDOW", from: 0, to: 1000) + XCTAssertFalse(wrongKind) + let outOfRange = try await store.hasEvent(deviceId: "dev1", kind: "BLE_CONNECTION_DOWN(12)", from: 200, to: 1000) + XCTAssertFalse(outOfRange) + let wrongDevice = try await store.hasEvent(deviceId: "other", kind: "BLE_CONNECTION_DOWN(12)", from: 0, to: 1000) + XCTAssertFalse(wrongDevice) + } + func testBatterySamples() async throws { let store = try await seeded() let bat = try await store.batterySamples(deviceId: "dev1", from: 0, to: 1000, limit: 100) diff --git a/Strand/Data/IntelligenceEngine.swift b/Strand/Data/IntelligenceEngine.swift index 9d4502b99..d13240996 100644 --- a/Strand/Data/IntelligenceEngine.swift +++ b/Strand/Data/IntelligenceEngine.swift @@ -455,7 +455,15 @@ final class IntelligenceEngine: ObservableObject { registry: registry, fallbackDeviceId: ownerFallbackId) let hr = (try? await store.hrSamples(deviceId: owner, from: from, to: to, limit: 200_000)) ?? [] - guard hr.count >= 200 else { continue } // need real raw data, not a stray sample + // Normally a night needs real raw HR (not a stray sample) to score. EXCEPTION (§6.15): an + // Oura ring banks no overnight HR but persists its own `check_sleep` sleep window — score + // that night from the window (totalSleepMin honest; HR-derived fields stay nil) instead of + // skipping it. Only a persisted OURA_SLEEP_WINDOW opens this path, so every non-Oura night + // keeps the byte-identical HR gate. + let hasSleepWindow = (try? await store.hasEvent(deviceId: owner, + kind: OuraStreamMapping.sleepWindowEventKind, + from: from, to: to)) ?? false + guard hr.count >= 200 || hasSleepWindow else { continue } let rr = (try? await store.rrIntervals(deviceId: owner, from: from, to: to, limit: 200_000)) ?? [] let resp = (try? await store.respSamples(deviceId: owner, from: from, to: to, limit: 200_000)) ?? [] let grav = (try? await store.gravitySamples(deviceId: owner, from: from, to: to, limit: 200_000)) ?? [] @@ -1329,8 +1337,17 @@ final class IntelligenceEngine: ObservableObject { let priority = d.id == activeId ? 0 : (isImport ? 2 : 1) // Cheap presence check: a single HR row for this device in the night window is enough to // mark it a candidate. (LIMIT 1 , not the full pull the caller does once an owner is chosen.) - let hasData = !((try? await store.hrSamples(deviceId: d.id, from: from, to: to, limit: 1)) ?? []).isEmpty - candidates.append(DayOwnerResolver.Candidate(deviceId: d.id, priority: priority, hasData: hasData)) + // #oura(§6.15): an Oura ring banks NO overnight HR (it's disconnected while you sleep) but DOES + // persist its own `check_sleep` sleep window — so an HR-only probe made the active ring lose its + // night to an imported strap. Count a persisted OURA_SLEEP_WINDOW as data too, so the ring can own + // a night NOTHING RICHER recorded. But a bare window is NOT a full record: `richData: hasHr` keeps + // it below any HR-backed candidate (e.g. an imported WHOOP night with stages/recovery/HRV) — so the + // ring wins only days no full record exists. Non-Oura devices never have this event → no change. + let hasHr = !((try? await store.hrSamples(deviceId: d.id, from: from, to: to, limit: 1)) ?? []).isEmpty + let hasWindow = (try? await store.hasEvent(deviceId: d.id, kind: OuraStreamMapping.sleepWindowEventKind, + from: from, to: to)) ?? false + candidates.append(DayOwnerResolver.Candidate(deviceId: d.id, priority: priority, + hasData: hasHr || hasWindow, richData: hasHr)) } return DayOwnerResolver.resolve(day: day, lockedOwner: nil, candidates: candidates) ?? fallbackDeviceId } diff --git a/android/app/src/main/java/com/noop/analytics/DayOwnerResolver.kt b/android/app/src/main/java/com/noop/analytics/DayOwnerResolver.kt index 65d146e36..7fe4c2cc8 100644 --- a/android/app/src/main/java/com/noop/analytics/DayOwnerResolver.kt +++ b/android/app/src/main/java/com/noop/analytics/DayOwnerResolver.kt @@ -8,12 +8,20 @@ package com.noop.analytics */ object DayOwnerResolver { /** A device in contention for owning [day]. [priority]: 0 = active strap, 1 = other live straps, - * 2 = imports (lower wins). [hasData] = the device actually has data for the day. */ - data class Candidate(val deviceId: String, val priority: Int, val hasData: Boolean) + * 2 = imports (lower wins). [hasData] = the device actually has data for the day. [richData] = a + * FULL record (HR-derived: stages, recovery, HRV) vs a bare duration window; a richer record + * outranks a window-only one regardless of priority, so an active ring's bare sleep window never + * displaces an import's full night (same duration, everything else blanked). Defaults true so every + * legacy candidate (all HR-backed) collapses back to priority-only order. */ + data class Candidate( + val deviceId: String, val priority: Int, val hasData: Boolean, val richData: Boolean = true + ) /** The owning deviceId, or null if no candidate has data for the day. A non-null [lockedOwner] - * (an explicit dayOwnership decision) always wins, regardless of priority or data. */ + * (an explicit dayOwnership decision) always wins, regardless of priority or data. Among candidates + * with data, a richer record wins first ([richData] true before false); ties break on priority. */ fun resolve(day: String, lockedOwner: String?, candidates: List): String? = if (lockedOwner != null) lockedOwner - else candidates.filter { it.hasData }.minByOrNull { it.priority }?.deviceId + else candidates.filter { it.hasData } + .minWithOrNull(compareBy({ if (it.richData) 0 else 1 }, { it.priority }))?.deviceId } diff --git a/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt b/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt index 33f598cdc..bd06869bf 100644 --- a/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt +++ b/android/app/src/main/java/com/noop/analytics/IntelligenceEngine.kt @@ -434,7 +434,12 @@ object IntelligenceEngine { // scored-days loop, NOT here). Only when the universal sink is on. A day skipped below for too // few rows is never scored, so it emits no line, byte-identical to the iOS behaviour. if (universalSink != null) readOwnerByDay[day] = OwnerRead(owner, hr.size) - if (hr.size < MIN_HR_SAMPLES) continue // need real raw data, not a stray sample + // Normally a night needs real raw HR to score. EXCEPTION (§6.15): an Oura ring banks no overnight + // HR but persists its own check_sleep sleep window — score that night from the window + // (totalSleepMin honest; HR-derived fields stay null) instead of skipping it. Only a persisted + // OURA_SLEEP_WINDOW opens this path, so every non-Oura night keeps the byte-identical gate. Swift twin. + val hasSleepWindow = repo.hasEvent(owner, OuraStreamMapping.EVENT_SLEEP_WINDOW, from, to) + if (hr.size < MIN_HR_SAMPLES && !hasSleepWindow) continue // need real raw data, not a stray sample val rr = repo.rrIntervals(owner, from, to, STREAM_LIMIT) val resp = repo.respSamples(owner, from, to, STREAM_LIMIT) val grav = repo.gravitySamples(owner, from, to, STREAM_LIMIT) @@ -1482,8 +1487,16 @@ object IntelligenceEngine { val candidates = candidatePriorities.map { (id, priority) -> // Cheap presence check: a single HR row for this device in the night window marks it a // candidate. (LIMIT 1 , not the full pull the caller does once an owner is chosen.) - val hasData = repo.hrSamples(id, from, to, 1).isNotEmpty() - DayOwnerResolver.Candidate(deviceId = id, priority = priority, hasData = hasData) + // §6.15: an Oura ring banks NO overnight HR but persists its own check_sleep sleep window, so an + // HR-only probe made the active ring lose its night to an imported strap. Count a persisted + // OURA_SLEEP_WINDOW as data too, so the ring can own a night NOTHING RICHER recorded — but a bare + // window is not a full record: richData = hasHr keeps it below any HR-backed candidate (e.g. an + // imported WHOOP night with stages/recovery/HRV). Non-Oura devices never have this event -> no + // change. Mirrors Swift. + val hasHr = repo.hrSamples(id, from, to, 1).isNotEmpty() + val hasWindow = repo.hasEvent(id, OuraStreamMapping.EVENT_SLEEP_WINDOW, from, to) + DayOwnerResolver.Candidate( + deviceId = id, priority = priority, hasData = hasHr || hasWindow, richData = hasHr) } return DayOwnerResolver.resolve(day, lockedOwner = null, candidates = candidates) ?: importedDeviceId } diff --git a/android/app/src/main/java/com/noop/data/WhoopDao.kt b/android/app/src/main/java/com/noop/data/WhoopDao.kt index 960d11a49..b51e0e469 100644 --- a/android/app/src/main/java/com/noop/data/WhoopDao.kt +++ b/android/app/src/main/java/com/noop/data/WhoopDao.kt @@ -244,6 +244,15 @@ interface WhoopDao : DeviceRegistryDao { ) suspend fun events(deviceId: String, from: Long, to: Long, limit: Int): List + /** Cheap presence probe: is there an event of [kind] for [deviceId] in `[from, to]`? Used by the + * day-owner resolver so an active ring with no overnight HR but a persisted check_sleep window + * (OURA_SLEEP_WINDOW, §6.15) still counts as having data. Twin of Swift `hasEvent`. */ + @Query( + "SELECT EXISTS(SELECT 1 FROM event WHERE deviceId = :deviceId AND kind = :kind " + + "AND ts >= :from AND ts <= :to LIMIT 1)" + ) + suspend fun hasEvent(deviceId: String, kind: String, from: Long, to: Long): Boolean + @Query( "SELECT * FROM battery WHERE deviceId = :deviceId AND ts >= :from AND ts <= :to " + "ORDER BY ts ASC LIMIT :limit" diff --git a/android/app/src/main/java/com/noop/data/WhoopRepository.kt b/android/app/src/main/java/com/noop/data/WhoopRepository.kt index 81c97757c..0ee001793 100644 --- a/android/app/src/main/java/com/noop/data/WhoopRepository.kt +++ b/android/app/src/main/java/com/noop/data/WhoopRepository.kt @@ -603,6 +603,10 @@ class WhoopRepository(private val dao: WhoopDao) { suspend fun events(deviceId: String, from: Long, to: Long, limit: Int = DEFAULT_LIMIT) = dao.events(deviceId, from, to, limit) + /** Cheap presence probe for one event kind in a window (§6.15 day-owner). Twin of Swift `hasEvent`. */ + suspend fun hasEvent(deviceId: String, kind: String, from: Long, to: Long): Boolean = + dao.hasEvent(deviceId, kind, from, to) + suspend fun batterySamples(deviceId: String, from: Long, to: Long, limit: Int = DEFAULT_LIMIT) = dao.batterySamples(deviceId, from, to, limit) diff --git a/android/app/src/test/java/com/noop/analytics/DayOwnerResolverTest.kt b/android/app/src/test/java/com/noop/analytics/DayOwnerResolverTest.kt index f4d41d345..bf4eddf31 100644 --- a/android/app/src/test/java/com/noop/analytics/DayOwnerResolverTest.kt +++ b/android/app/src/test/java/com/noop/analytics/DayOwnerResolverTest.kt @@ -58,4 +58,45 @@ class DayOwnerResolverTest { ) assertNull(DayOwnerResolver.resolve("2026-06-15", lockedOwner = null, candidates = candidates)) } + + // §6.15: an active Oura ring with only a bare check_sleep window (richData=false) must NOT displace an + // imported WHOOP night with a full HR-backed record (richData=true) — the richer record wins despite + // the worse priority. Mirrors Swift testRichImportBeatsActiveWindowOnlyRing. + @Test + fun richImportBeatsActiveWindowOnlyRing() { + val candidates = listOf( + DayOwnerResolver.Candidate("oura", priority = 0, hasData = true, richData = false), + DayOwnerResolver.Candidate("whoop-import", priority = 2, hasData = true, richData = true), + ) + assertEquals( + "whoop-import", + DayOwnerResolver.resolve("2026-07-08", lockedOwner = null, candidates = candidates), + ) + } + + // …but on a day nothing richer recorded, the window-only ring is the sole source and owns it. + @Test + fun windowOnlyRingOwnsDayWithNoRicherRecord() { + val candidates = listOf( + DayOwnerResolver.Candidate("oura", priority = 0, hasData = true, richData = false), + DayOwnerResolver.Candidate("whoop-import", priority = 2, hasData = false, richData = true), + ) + assertEquals( + "oura", + DayOwnerResolver.resolve("2026-07-09", lockedOwner = null, candidates = candidates), + ) + } + + // Two window-only rings (both richData=false) still fall back to device priority (active wins). + @Test + fun windowOnlyTieBreaksOnPriority() { + val candidates = listOf( + DayOwnerResolver.Candidate("oura-active", priority = 0, hasData = true, richData = false), + DayOwnerResolver.Candidate("oura-other", priority = 1, hasData = true, richData = false), + ) + assertEquals( + "oura-active", + DayOwnerResolver.resolve("2026-07-09", lockedOwner = null, candidates = candidates), + ) + } } From dddbe952dc9937340b0133ffa449c5f716437b4f Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Thu, 9 Jul 2026 16:32:20 +0200 Subject: [PATCH 18/21] =?UTF-8?q?=20=20=20Oura:=20fix=20#91=20full=20re-du?= =?UTF-8?q?mp=20loop=20=E2=80=94=20client-managed=20ring-time=20resume=20c?= =?UTF-8?q?ursor?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The GetEvents (0x11) response was being misread: bytes 2-5 are bytes_left (a remaining-BYTE count that drains ~800KB→0), not a "last ring timestamp cursor". Persisting that byte-count and comparing it across sessions minted a phantom "ring-time regression" every connect → reset-to-0 → full history re-dump (~40k phantom-record drops + #91 log spam). Fix, matching open_oura's model (EventBatchSummary + nextEventToSync): - parseGetEventsResponse now returns (eventsReceived, bytesLeft, moreData); the drain loops while bytes_left > 0, and the resume position is CLIENT-managed (max stored ring-time), not read from the 0x11 body. - OuraLiveSource persists a real ring-time resume cursor, only when it resolves against the anchor; a plausibility ceiling (maxPlausibleResumeTicks = 500M ds) clamps any absurd persisted value back to a full pull so a pre-fix garbage cursor can't wedge the ring on a seek-past-end forever. - describeCursor renders the cursor as a human-readable date (all history / date / out-of-window / no-anchor). Proven on-device (Oura Ring Gen3): connect 1 clamps the leftover garbage cursor, does one full pull, acquires the anchor, and stores resume cursor 1624998 [2026-07-09 16:25:42]; connect 2 resumes from it and returns 55 events with bytes_left=0 — zero phantom drops, no re-dump. Android twin + docs/OURA_PROTOCOL.md §5.2/§5.3 updated to match. OuraProtocol tests 99/0. --- .../Sources/OuraProtocol/Framing.swift | 22 ++- .../OuraProtocolTests/FramingTests.swift | 30 +++- Strand/BLE/OuraLiveSource.swift | 147 ++++++++++++----- .../main/java/com/noop/ble/OuraLiveSource.kt | 151 +++++++++++++----- .../src/main/java/com/noop/oura/Framing.kt | 31 ++-- .../test/java/com/noop/oura/FramingTest.kt | 29 +++- docs/OURA_PROTOCOL.md | 21 ++- 7 files changed, 311 insertions(+), 120 deletions(-) diff --git a/Packages/OuraProtocol/Sources/OuraProtocol/Framing.swift b/Packages/OuraProtocol/Sources/OuraProtocol/Framing.swift index 3498e1d38..d2c3b21e3 100644 --- a/Packages/OuraProtocol/Sources/OuraProtocol/Framing.swift +++ b/Packages/OuraProtocol/Sources/OuraProtocol/Framing.swift @@ -47,15 +47,21 @@ public enum OuraFraming { /// caller fails to special-case it. public static let batteryResponseOp: UInt8 = 0x0D - /// Parse a 0x11 GetEvents response body: `status:1 sub_status:1 last_ring_timestamp:4LE pad:2` - /// (OURA_PROTOCOL.md s5.2). `status` 0x00 = empty/no more; any other value = data follows. The - /// `last_ring_timestamp` is the new cursor to resume the fetch from. Returns nil on a short body - /// (never guesses a cursor). - public static func parseGetEventsResponse(_ body: [UInt8]) -> (cursor: UInt32, moreData: Bool)? { + /// Parse a 0x11 GetEvents response body per open_oura's `EventBatchSummary` (events.rs): + /// `events_received:1 sleep_analysis_progress:1 bytes_left:4LE`. The drain loop runs until + /// `bytes_left == 0` (sync-orchestration.md). There is **no resume cursor in this packet** — the + /// resume position is a CLIENT-managed event-envelope ring-time (`nextEventToSync`), never read back + /// here. Returns nil on a short body. + /// + /// #91: NOOP previously decoded bytes[2..5] as a `last_ring_timestamp` cursor, persisted it, and + /// compared it across sessions. Those bytes are `bytes_left` — a remaining-byte count (~800 KB full → + /// 0). Two unrelated byte-counts compared as clocks minted a phantom "ring-time regression", which + /// reset the cursor to 0 and re-dumped the ring's entire banked history on every connect. + public static func parseGetEventsResponse(_ body: [UInt8]) -> (eventsReceived: UInt8, bytesLeft: UInt32, moreData: Bool)? { guard body.count >= 6 else { return nil } - let status = body[0] - let cursor = UInt32(body[2]) | (UInt32(body[3]) << 8) | (UInt32(body[4]) << 16) | (UInt32(body[5]) << 24) - return (cursor, status != 0x00) + let eventsReceived = body[0] + let bytesLeft = UInt32(body[2]) | (UInt32(body[3]) << 8) | (UInt32(body[4]) << 16) | (UInt32(body[5]) << 24) + return (eventsReceived, bytesLeft, bytesLeft > 0) } /// Parse one outer frame from the front of `bytes`. Returns nil on a short buffer (header or body diff --git a/Packages/OuraProtocol/Tests/OuraProtocolTests/FramingTests.swift b/Packages/OuraProtocol/Tests/OuraProtocolTests/FramingTests.swift index 2ef905306..7a1eba4e4 100644 --- a/Packages/OuraProtocol/Tests/OuraProtocolTests/FramingTests.swift +++ b/Packages/OuraProtocol/Tests/OuraProtocolTests/FramingTests.swift @@ -55,20 +55,34 @@ final class FramingTests: XCTestCase { // MARK: - GetEvents response (0x11, s5.2) func testParseGetEventsResponseMoreDataFollows() { - // 11 08 + // 11 08 let outer = OuraFraming.parseOuterFrame(bytes("1108ff00785634120000")) XCTAssertEqual(outer?.op, OuraFraming.getEventsResponseOp) let summary = OuraFraming.parseGetEventsResponse(outer!.body) - XCTAssertEqual(summary?.cursor, 0x1234_5678) - XCTAssertEqual(summary?.moreData, true) + XCTAssertEqual(summary?.eventsReceived, 0xff) + XCTAssertEqual(summary?.bytesLeft, 0x1234_5678) + XCTAssertEqual(summary?.moreData, true) // bytes_left > 0 ⇒ ring has more } - func testParseGetEventsResponseNoMoreData() { - // status 0x00 -> caught up, no more data. - let outer = OuraFraming.parseOuterFrame(bytes("11080000785634120000")) + func testParseGetEventsResponseTerminalIsBytesLeftZero() { + // The terminal packet zero-fills bytes_left (00 00 00 00). events_received may be 0 too. + let outer = OuraFraming.parseOuterFrame(bytes("11080000000000000000")) let summary = OuraFraming.parseGetEventsResponse(outer!.body) - XCTAssertEqual(summary?.cursor, 0x1234_5678) - XCTAssertEqual(summary?.moreData, false) + XCTAssertEqual(summary?.bytesLeft, 0) + XCTAssertEqual(summary?.moreData, false) // bytes_left == 0 ⇒ drain complete + } + + // #91: bytes_left is a remaining-BYTE count, not a cursor. A later summary reporting a SMALLER + // bytes_left than an earlier one is normal draining (not a "regression"), and while it stays > 0 the + // loop must keep going — the exact case NOOP used to misread as a session reset → full re-dump. + func testParseGetEventsResponseSmallerBytesLeftStillMeansMoreData() { + let first = OuraFraming.parseGetEventsResponse(bytes("ff00746e05000000")) // bytes_left 355956 + let later = OuraFraming.parseGetEventsResponse(bytes("ff000d0d00000000")) // bytes_left 3341 (< first) + XCTAssertEqual(first?.bytesLeft, 355_956) + XCTAssertEqual(later?.bytesLeft, 3_341) + XCTAssertLessThan(later!.bytesLeft, first!.bytesLeft) + XCTAssertTrue(first!.moreData) + XCTAssertTrue(later!.moreData) // still draining — NOT a regression/terminal } func testParseGetEventsResponseShortBodyReturnsNil() { diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index a0e707de5..8459545d0 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -102,14 +102,17 @@ public final class OuraLiveSource: NSObject, ObservableObject { return f }() - /// Decode a ring-tick cursor value to a human-readable local date/time via the driver's current - /// session anchor (s5.5), or "no anchor yet" when none has arrived yet this session (honest: never - /// guesses a time). Investigation/logging only. + /// Decode a ring-tick cursor to a human-readable local date/time via the current session anchor (s5.5). + /// Distinguishes the three honest states so a log line never misreports (investigation/logging only): + /// `all history` (the 0 sentinel = full pull), a real date, `out of anchor window` (an anchor EXISTS but + /// this ring-time is a phantom — e.g. a garbage resume cursor), or `no anchor yet` (none this session). private func describeCursor(_ cursor: UInt32) -> String { - guard let driver, let seconds = driver.unixSeconds(forRingTimestamp: cursor) else { - return "no anchor yet" + guard cursor != 0 else { return "all history" } + guard let driver else { return "no driver" } + if let seconds = driver.unixSeconds(forRingTimestamp: cursor) { + return Self.cursorDateFormatter.string(from: Date(timeIntervalSince1970: TimeInterval(seconds))) } - return Self.cursorDateFormatter.string(from: Date(timeIntervalSince1970: TimeInterval(seconds))) + return driver.hasAnchor ? "out of anchor window" : "no anchor yet" } // MARK: - Dependencies (injected - no BLEManager / WhoopBleClient reference) @@ -251,10 +254,29 @@ public final class OuraLiveSource: NSObject, ObservableObject { // arrive by. Neither temp nor SpO2 is ever pushed live on this hardware; both are banked overnight and // retrievable only by asking the ring for its history. - /// The GetEvents cursor to resume from, loaded from `OuraHistoryCursorStore` on connect and advanced - /// as `0x11` summaries arrive. 0 = fetch everything the ring has banked (first-ever connect for this - /// ring; OURA_PROTOCOL.md s5.1). + /// The GetEvents resume cursor — a CLIENT-managed event-envelope ring-time (open_oura's + /// `nextEventToSync`, sync-orchestration.md), loaded from `OuraHistoryCursorStore` on connect and + /// advanced to the newest STORED history sample's ring-time when a drain completes. 0 = fetch + /// everything the ring has banked (first-ever connect; OURA_PROTOCOL.md s5.1). + /// + /// #91: the GetEvents response carries NO cursor — only `bytes_left`. NOOP used to persist that + /// byte-count and, seeing next session's `bytes_left` smaller, declared a phantom "ring-time + /// regression" and re-dumped everything. The real resume point is the ring-time we last stored. private var historyCursor: UInt32 = 0 + /// The newest STORED (real, anchored) history-sample ring-time seen this session — becomes the next + /// persisted `historyCursor` when the drain completes. Only advanced from anchored stores, never from + /// the no-anchor wall-clock fallback (whose ring-times are meaningless), so it stays an honest resume point. + private var maxStoredRingTime: UInt32 = 0 + /// Absolute ceiling for a plausible ring-time (deciseconds since boot): ~1.6 years of uptime. A + /// wearable never runs this long between reboots (observed uptime is days), so any cursor above it is + /// misframe garbage and must never be seeked to. `UInt32` can hold ~13.6 y of deciseconds, so this + /// leaves generous headroom above real uptimes while still rejecting the 2e9-class garbage values. + private static let maxPlausibleResumeTicks: UInt32 = 500_000_000 + /// The cursor we resumed FROM at the start of the current fetch, kept so we can detect a genuine ring + /// reboot: if the ring hands back a real stored sample OLDER than where we sought, its clock reset (or + /// it ignored the seek), so the persisted cursor is stale and must fall back to a full pull. + private var resumeCursorAtFetchStart: UInt32 = 0 + private var sawPreResumeData = false /// Periodic re-fetch while connected, so an overnight-connected session (or one left open after a nap) /// picks up freshly-banked sleep data without needing a reconnect. Mirrors BLEManager's ~15 min /// periodic WHOOP history-offload floor. @@ -266,6 +288,8 @@ public final class OuraLiveSource: NSObject, ObservableObject { /// both right after reaching `.streaming` and from the periodic timer). private func fetchHistoryIfIdle() { guard let driver, driver.phase == .streaming else { return } + resumeCursorAtFetchStart = historyCursor + sawPreResumeData = false log("Oura: fetching history from cursor \(historyCursor) (\(describeCursor(historyCursor)))") advance(.startHistoryFetch(cursor: historyCursor)) } @@ -283,36 +307,53 @@ public final class OuraLiveSource: NSObject, ObservableObject { historyFetchTimer = nil } - /// Handle a `0x11` GetEvents response (OURA_PROTOCOL.md s5.2): persist the advanced cursor (so a LATER - /// connection resumes rather than re-fetching everything) and drive the driver's cursor-loop state - /// machine, which asks for another ack-fetch while `moreData` or returns to `.streaming` once caught up. - /// - /// The ring's terminal "no more data" response (moreData=false, status 0x00) zero-fills the cursor - /// field, whereas a mid-fetch response (moreData=true) carries a real advancing nonzero cursor. So the - /// cursor is only trusted/persisted while the response is actually carrying new data - persisting the - /// terminal zero would reset the cursor to 0 on every fetch and force a full backlog re-fetch forever. + /// Handle a `0x11` GetEvents response (OURA_PROTOCOL.md s5.2 / open_oura `EventBatchSummary`). The + /// summary carries `bytes_left`, NOT a resume cursor: while `bytes_left > 0` (`summary.moreData`) the + /// drain continues (ack-fetch); at `bytes_left == 0` it is complete. The response's byte-count is NEVER + /// persisted — persisting it and comparing byte-counts across sessions as clocks was the #91 re-dump. /// - /// A cursor persisted from one BLE connection can come back SMALLER on the next connection's first real - /// cursor: `ringTimestamp = (session << 16) | counter` (OURA_PROTOCOL.md s2.3), and the ring's internal - /// `session` component can shift across reconnects/restarts. This is the same class of problem s5.5 - /// documents for the UTC anchor (ring-start with rt regression -> invalidate anchor). Resuming from a - /// cursor whose session no longer matches the ring's current one is not a real resume - the ring just - /// re-dumps its whole backlog anyway - so we detect the regression and reset to an honest, explicit 0 - /// rather than feed the ring a now-meaningless reference. - private func handleHistorySummary(_ summary: (cursor: UInt32, moreData: Bool)) { - if summary.moreData { - if summary.cursor < historyCursor { - log("Oura: ring-time regression detected (fetch cursor \(summary.cursor) [\(describeCursor(summary.cursor))] < persisted \(historyCursor) [\(describeCursor(historyCursor))]) - the ring's session likely reset; resetting our cursor to 0") + /// The durable resume point (open_oura `nextEventToSync`) is the newest STORED history sample's + /// ring-time (`maxStoredRingTime`), committed here when the drain completes. A genuine ring reboot is + /// caught by `sawPreResumeData` — a stored sample older than where we sought means the ring's clock + /// reset (or it ignored the seek), so we fall back to a full pull (0) next time rather than resume from + /// a now-stale ring-time. + private func handleHistorySummary(_ summary: (eventsReceived: UInt8, bytesLeft: UInt32, moreData: Bool)) { + if !summary.moreData { + // A resume cursor is only trustworthy if it resolves to a real time under the CURRENT anchor. + // A value that doesn't was stored against a TRANSIENT bad anchor (a misframed 0x85/0x42 whose + // epoch passed but whose ring-time was garbage) — persisting it would seek to nonsense next + // connect. Treat it, and a genuine reboot (sawPreResumeData), the same: full pull next time. + let resumeResolves = maxStoredRingTime > 0 && (driver?.unixSeconds(forRingTimestamp: maxStoredRingTime) != nil) + if sawPreResumeData { + log("Oura: history fetch caught up but the ring served data older than cursor \(resumeCursorAtFetchStart) - clock reset/seek ignored; next connect does a full pull") + historyCursor = 0 + OuraHistoryCursorStore.save(0, deviceId: deviceId) + } else if maxStoredRingTime > historyCursor, resumeResolves { + historyCursor = maxStoredRingTime + OuraHistoryCursorStore.save(maxStoredRingTime, deviceId: deviceId) + log("Oura: history fetch caught up (resume cursor \(historyCursor) [\(describeCursor(historyCursor))])") + } else if maxStoredRingTime > historyCursor { + log("Oura: history fetch caught up but resume cursor \(maxStoredRingTime) [\(describeCursor(maxStoredRingTime))] is a phantom (transient bad anchor?) - full pull next connect") historyCursor = 0 OuraHistoryCursorStore.save(0, deviceId: deviceId) } else { - historyCursor = summary.cursor - OuraHistoryCursorStore.save(summary.cursor, deviceId: deviceId) + log("Oura: history fetch caught up (resume cursor unchanged \(historyCursor) [\(describeCursor(historyCursor))])") } - } else { - log("Oura: history fetch caught up (cursor \(historyCursor) [\(describeCursor(historyCursor))])") } - advance(.historyCursorAdvanced(cursor: summary.cursor, moreData: summary.moreData)) + // The ack-fetch cursor is ignored by the ring (maxEvents=0 continuation); pass the resume point. + advance(.historyCursorAdvanced(cursor: historyCursor, moreData: summary.moreData)) + } + + /// Record a STORED history sample's ring-time toward the resume cursor (open_oura `nextEventToSync`). + /// Called only where a sample resolved a REAL anchored time and was enqueued — never for the no-anchor + /// wall-clock fallback. Also flags a reboot: a real sample older than where we sought this fetch. + private func noteStoredHistoryRingTime(_ rt: UInt32) { + // Ignore a ring-time above the plausibility ceiling: it resolved to a real time only because a + // TRANSIENT bad anchor (misframed 0x85/0x42) made it look valid, and letting it set the resume + // cursor is exactly what banked the 2e9 garbage. Bounds the cursor at the source. + guard rt <= Self.maxPlausibleResumeTicks else { return } + if rt > maxStoredRingTime { maxStoredRingTime = rt } + if resumeCursorAtFetchStart > 0, rt < resumeCursorAtFetchStart { sawPreResumeData = true } } // MARK: - Sample buffer @@ -623,6 +664,7 @@ public final class OuraLiveSource: NSObject, ObservableObject { for pending in pendingAnchorEvents { if let ts = driver.unixSeconds(forRingTimestamp: pending.ringTimestamp) { enqueue([pending.event], ts: ts) + noteStoredHistoryRingTime(pending.ringTimestamp) // parked sample placed → advance resume cursor } else if driver.hasAnchor { droppedPhantoms += 1 // anchor present but rt is phantom → drop, never bank } else { @@ -680,6 +722,7 @@ public final class OuraLiveSource: NSObject, ObservableObject { } if let ts = driver.unixSeconds(forRingTimestamp: t.ringTimestamp) { enqueue([e], ts: ts) + noteStoredHistoryRingTime(t.ringTimestamp) } else { pendingAnchorEvents.append((e, t.ringTimestamp)) } @@ -691,6 +734,7 @@ public final class OuraLiveSource: NSObject, ObservableObject { } if let ts = driver.unixSeconds(forRingTimestamp: s.ringTimestamp) { enqueue([e], ts: ts) + noteStoredHistoryRingTime(s.ringTimestamp) } else { pendingAnchorEvents.append((e, s.ringTimestamp)) } @@ -698,6 +742,7 @@ public final class OuraLiveSource: NSObject, ObservableObject { case .hrv(let v): if let ts = driver.unixSeconds(forRingTimestamp: v.ringTimestamp) { enqueue([e], ts: ts) + noteStoredHistoryRingTime(v.ringTimestamp) } else { pendingAnchorEvents.append((e, v.ringTimestamp)) } @@ -705,6 +750,7 @@ public final class OuraLiveSource: NSObject, ObservableObject { case .sleepPhase(let v): if let ts = driver.unixSeconds(forRingTimestamp: v.ringTimestamp) { enqueue([e], ts: ts) + noteStoredHistoryRingTime(v.ringTimestamp) } else { pendingAnchorEvents.append((e, v.ringTimestamp)) } @@ -1024,9 +1070,24 @@ extension OuraLiveSource: @preconcurrency CBCentralManagerDelegate { pendingInstallKey = nil adoptPhase = .idle reassembler.reset() - // Resume the GetEvents cursor from where the LAST connection to this ring left off (s5.1/5.3), so - // a routine reconnect doesn't re-fetch the ring's entire banked history every time. + // Resume the GetEvents drain from where the LAST connection to this ring left off (s5.1/5.3) — a + // client-managed event-envelope ring-time (open_oura `nextEventToSync`) — so a routine reconnect + // doesn't re-fetch the ring's entire banked history every time. The per-session resume trackers + // start fresh. historyCursor = OuraHistoryCursorStore.read(deviceId: deviceId) + // Ceiling guard: a Gen3's ring-time is deciseconds-since-boot and a wearable's uptime is days + // (observed ~1.6M ≈ 1.8 days), so a persisted cursor above ~1.6 years of ticks is misframe garbage + // (e.g. 2_055_602_179 = ~6.5 y, banked by a pre-guard build against a transient bad anchor). Seeking + // to it would return nothing forever — the ring honors the seek — so a stuck ring never syncs. Clamp + // it back to a full pull. Real advances are bounded by the anchor-resolve guard in handleHistorySummary. + if historyCursor > Self.maxPlausibleResumeTicks { + log("Oura: persisted resume cursor \(historyCursor) is implausibly large - resetting to a full pull") + historyCursor = 0 + OuraHistoryCursorStore.save(0, deviceId: deviceId) + } + maxStoredRingTime = 0 + resumeCursorAtFetchStart = 0 + sawPreResumeData = false peripheral.discoverServices([Self.service]) } @@ -1281,14 +1342,18 @@ public enum OuraKeyStore { } } -// MARK: - Oura GetEvents cursor persistence +// MARK: - Oura GetEvents resume-cursor persistence -/// Persists the Oura `GetEvents` cursor (OURA_PROTOCOL.md s5.1/5.3) per ring, so a later connection -/// resumes from where the last session left off instead of re-fetching the ring's entire banked history -/// on every single connect. Unlike `OuraKeyStore` this is NOT sensitive - it's an opaque ring-clock tick -/// counter, not a credential - so plain `UserDefaults` is the right (and simplest) store. +/// Persists the Oura history resume cursor (OURA_PROTOCOL.md s5.1/5.3) per ring — an event-envelope +/// ring-time (open_oura `nextEventToSync`) — so a later connection resumes where the last session left off +/// instead of re-fetching the ring's entire banked history. Not sensitive (a ring-clock tick, not a +/// credential), so plain `UserDefaults` is right. +/// +/// #91: the key is bumped (`.resumeRt.`) because the old key stored the misdecoded `bytes_left` byte-count; +/// ignoring that stale value means the first post-upgrade connect does one clean full pull rather than +/// seeking to a garbage ring-time. enum OuraHistoryCursorStore { - private static func key(deviceId: String) -> String { "com.noop.oura.historyCursor.\(deviceId)" } + private static func key(deviceId: String) -> String { "com.noop.oura.historyResumeRt.\(deviceId)" } /// The persisted cursor for `deviceId`, or 0 (fetch everything) if none is stored yet. static func read(deviceId: String) -> UInt32 { diff --git a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt index d1d562294..77c642e79 100644 --- a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt +++ b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt @@ -353,11 +353,27 @@ class OuraLiveSource( // retrievable only by asking the ring for its history. Kotlin twin of the Swift lane9 history wiring. /** - * The GetEvents cursor to resume from, loaded from [OuraHistoryCursorStore] on connect and advanced as - * `0x11` summaries arrive. 0 = fetch everything the ring has banked (first-ever connect for this ring; - * OURA_PROTOCOL.md s5.1). Held as a Long (the unsigned 32-bit ring timestamp). + * The GetEvents resume cursor — a CLIENT-managed event-envelope ring-time (open_oura `nextEventToSync`, + * sync-orchestration.md), loaded from [OuraHistoryCursorStore] on connect and advanced to the newest + * STORED history sample's ring-time when a drain completes. 0 = fetch everything the ring has banked + * (first-ever connect; OURA_PROTOCOL.md s5.1). Held as a Long (unsigned 32-bit ring timestamp). + * + * #91: the GetEvents response carries NO cursor — only `bytes_left`. NOOP used to persist that byte-count + * and, seeing next session's smaller, declared a phantom "regression" and re-dumped everything. Kotlin + * twin of Swift's `historyCursor`. */ private var historyCursor: Long = 0 + /** Newest STORED (real, anchored) history-sample ring-time this session — becomes the next persisted + * [historyCursor] when the drain completes. Never advanced from the no-anchor wall-clock fallback. */ + private var maxStoredRingTime: Long = 0 + /** Absolute ceiling for a plausible ring-time (deciseconds since boot): ~1.6 years of uptime. A wearable + * never runs this long between reboots (observed uptime is days), so any cursor above it is misframe + * garbage that must never be seeked to. Kotlin twin of Swift's `maxPlausibleResumeTicks`. */ + private val maxPlausibleResumeTicks = 500_000_000L + /** The cursor we resumed FROM this fetch, to detect a genuine ring reboot: a stored sample older than + * where we sought means the clock reset (or the seek was ignored) -> fall back to a full pull. */ + private var resumeCursorAtFetchStart: Long = 0 + private var sawPreResumeData = false /** * Periodic re-fetch while connected, so an overnight-connected session (or one left open after a nap) @@ -395,10 +411,29 @@ class OuraLiveSource( private fun fetchHistoryIfIdle(): Unit = guardedCallback("history-fetch") { val d = driver ?: return@guardedCallback if (d.phase != OuraDriverPhase.Streaming) return@guardedCallback - log("Oura: fetching history from cursor $historyCursor") + resumeCursorAtFetchStart = historyCursor + sawPreResumeData = false + log("Oura: fetching history from cursor $historyCursor (${describeCursor(historyCursor)})") advance(OuraTransition.StartHistoryFetch(cursor = historyCursor)) } + /** + * Decode a ring-tick cursor to a human-readable local date/time via the current session anchor (s5.5). + * Distinguishes the honest states so a log line never misreports: `all history` (the 0 sentinel = full + * pull), a real date, `out of anchor window` (an anchor EXISTS but this ring-time is a phantom — e.g. a + * garbage resume cursor), or `no anchor yet` (none this session). Kotlin twin of Swift's `describeCursor`. + */ + private fun describeCursor(cursor: Long): String { + if (cursor == 0L) return "all history" + val d = driver ?: return "no driver" + val seconds = d.unixSeconds(forRingTimestamp = cursor) + return when { + seconds != null -> SimpleDateFormat("yyyy-MM-dd HH:mm:ss", Locale.US).format(Date(seconds * 1000L)) + d.hasAnchor -> "out of anchor window" + else -> "no anchor yet" + } + } + private fun scheduleHistoryFetch() { if (historyFetchScheduled) return historyFetchScheduled = true @@ -411,37 +446,57 @@ class OuraLiveSource( } /** - * Handle a `0x11` GetEvents response (OURA_PROTOCOL.md s5.2): persist the advanced cursor (so a LATER - * connection resumes rather than re-fetching everything) and drive the driver's cursor-loop state - * machine, which asks for another ack-fetch while `moreData` or returns to Streaming once caught up. - * - * The ring's terminal "no more data" response (moreData=false, status 0x00) zero-fills the cursor - * field, whereas a mid-fetch response (moreData=true) carries a real advancing nonzero cursor. So the - * cursor is only trusted/persisted while the response is actually carrying new data - persisting the - * terminal zero would reset the cursor to 0 on every fetch and force a full backlog re-fetch forever. - * - * A cursor persisted from one BLE connection can come back SMALLER on the next connection's first real - * cursor: `ringTimestamp = (session << 16) | counter` (s2.3), and the ring's internal `session` - * component can shift across reconnects/restarts. Resuming from a cursor whose session no longer matches - * the ring's current one is not a real resume - the ring just re-dumps its whole backlog anyway - so we - * detect the regression and reset to an honest, explicit 0 rather than feed the ring a now-meaningless - * reference. Kotlin twin of Swift's `handleHistorySummary`. + * Handle a `0x11` GetEvents response (OURA_PROTOCOL.md s5.2 / open_oura `EventBatchSummary`). The summary + * carries `bytes_left`, NOT a resume cursor: while `bytes_left > 0` (`moreData`) the drain continues; at 0 + * it is complete. The byte-count is NEVER persisted — that (compared across sessions as a clock) was the + * #91 re-dump. The durable resume point (`nextEventToSync`) is the newest STORED sample's ring-time + * ([maxStoredRingTime]), committed here on completion; a genuine reboot ([sawPreResumeData]) falls back + * to a full pull. Kotlin twin of Swift's `handleHistorySummary`. */ private fun handleHistorySummary(summary: com.noop.oura.GetEventsSummary): Unit = guardedCallback("history-summary") { - if (summary.moreData) { - if (summary.cursor < historyCursor) { - log("Oura: ring-time regression detected (fetch cursor ${summary.cursor} < persisted " + - "$historyCursor) - the ring's session likely reset; resetting our cursor to 0") - historyCursor = 0 - OuraHistoryCursorStore.save(appContext, deviceId, 0) - } else { - historyCursor = summary.cursor - OuraHistoryCursorStore.save(appContext, deviceId, summary.cursor) + if (!summary.moreData) { + // A resume cursor is only trustworthy if it resolves to a real time under the CURRENT anchor. A + // value that doesn't was stored against a TRANSIENT bad anchor (a misframed 0x85/0x42 whose epoch + // passed but whose ring-time was garbage); persisting it would seek to nonsense next connect. + // Treat it, and a genuine reboot (sawPreResumeData), the same: full pull next time. Mirrors Swift. + val resumeResolves = maxStoredRingTime > 0 && driver?.unixSeconds(forRingTimestamp = maxStoredRingTime) != null + when { + sawPreResumeData -> { + log("Oura: history fetch caught up but the ring served data older than cursor " + + "$resumeCursorAtFetchStart - clock reset/seek ignored; next connect does a full pull") + historyCursor = 0 + OuraHistoryCursorStore.save(appContext, deviceId, 0) + } + maxStoredRingTime > historyCursor && resumeResolves -> { + historyCursor = maxStoredRingTime + OuraHistoryCursorStore.save(appContext, deviceId, maxStoredRingTime) + log("Oura: history fetch caught up (resume cursor $historyCursor (${describeCursor(historyCursor)}))") + } + maxStoredRingTime > historyCursor -> { + log("Oura: history fetch caught up but resume cursor $maxStoredRingTime " + + "(${describeCursor(maxStoredRingTime)}) is a phantom (transient bad anchor?) - full pull next connect") + historyCursor = 0 + OuraHistoryCursorStore.save(appContext, deviceId, 0) + } + else -> log("Oura: history fetch caught up (resume cursor unchanged $historyCursor (${describeCursor(historyCursor)}))") } - } else { - log("Oura: history fetch caught up (cursor $historyCursor)") } - advance(OuraTransition.HistoryCursorAdvanced(cursor = summary.cursor, moreData = summary.moreData)) + // The ack-fetch cursor is ignored by the ring (maxEvents=0 continuation); pass the resume point. + advance(OuraTransition.HistoryCursorAdvanced(cursor = historyCursor, moreData = summary.moreData)) + } + + /** + * Record a STORED history sample's ring-time toward the resume cursor (open_oura `nextEventToSync`). + * Called only where a sample resolved a REAL anchored time and was enqueued — never for the no-anchor + * wall-clock fallback. Also flags a reboot: a real sample older than where we sought this fetch. + */ + private fun noteStoredHistoryRingTime(rt: Long) { + // Ignore a ring-time above the plausibility ceiling: it resolved to a real time only because a + // TRANSIENT bad anchor (misframed 0x85/0x42) made it look valid; letting it set the resume cursor is + // exactly what banked the 2e9 garbage. Bounds the cursor at the source. Mirrors Swift. + if (rt > maxPlausibleResumeTicks) return + if (rt > maxStoredRingTime) maxStoredRingTime = rt + if (resumeCursorAtFetchStart > 0 && rt < resumeCursorAtFetchStart) sawPreResumeData = true } // MARK: - Sample buffer (flushed in batches off the per-notification hot loop) @@ -540,9 +595,21 @@ class OuraLiveSource( checkSleepParser.reset() loggedTierBKinds.clear() pendingAnchorEvents.clear() - // Resume the GetEvents cursor from where the LAST connection to this ring left off (s5.1/5.3), so a - // routine reconnect doesn't re-fetch the ring's entire banked history every time. + // Resume the GetEvents drain from where the LAST connection left off (s5.1/5.3) - a client-managed + // event-envelope ring-time (open_oura `nextEventToSync`) - so a routine reconnect doesn't re-fetch + // the ring's entire banked history. The per-session resume trackers start fresh. historyCursor = OuraHistoryCursorStore.read(appContext, deviceId) + // Ceiling guard: a persisted cursor above ~1.6 years of ticks is misframe garbage (e.g. 2_055_602_179 + // = ~6.5 y, banked by a pre-guard build against a transient bad anchor). The ring honors the seek, so + // seeking to it returns nothing forever -> a stuck ring never syncs. Clamp back to a full pull. + if (historyCursor > maxPlausibleResumeTicks) { + log("Oura: persisted resume cursor $historyCursor is implausibly large - resetting to a full pull") + historyCursor = 0 + OuraHistoryCursorStore.save(appContext, deviceId, 0) + } + maxStoredRingTime = 0 + resumeCursorAtFetchStart = 0 + sawPreResumeData = false // connectGatt can throw (SecurityException if BLUETOOTH_CONNECT was revoked mid-session, // IllegalArgumentException on a stale device) - never let that crash the app; a failed start // simply leaves the previous source in place (mirrors [StandardHrSource]). @@ -660,7 +727,10 @@ class OuraLiveSource( for ((event, ringTimestamp) in pendingAnchorEvents) { val ts = d.unixSeconds(forRingTimestamp = ringTimestamp)?.toInt() when { - ts != null -> enqueue(listOf(event), ts) + ts != null -> { + enqueue(listOf(event), ts) + noteStoredHistoryRingTime(ringTimestamp) // parked sample placed -> advance resume cursor + } d.hasAnchor -> droppedPhantoms++ // anchor present but rt is phantom -> drop, never bank else -> enqueue(listOf(event), now) // no anchor this session -> honest wall-clock fallback } @@ -1195,7 +1265,12 @@ class OuraLiveSource( */ private fun enqueueAnchoredOrPark(event: OuraEvent, ringTimestamp: Long, d: OuraDriver) { val ts = d.unixSeconds(forRingTimestamp = ringTimestamp) - if (ts != null) enqueue(listOf(event), ts.toInt()) else pendingAnchorEvents.add(event to ringTimestamp) + if (ts != null) { + enqueue(listOf(event), ts.toInt()) + noteStoredHistoryRingTime(ringTimestamp) // history sample placed -> advance the resume cursor + } else { + pendingAnchorEvents.add(event to ringTimestamp) + } } private fun handleBattery(pct: Int) = guardedCallback("battery") { @@ -1319,7 +1394,11 @@ class OuraLiveSource( */ object OuraHistoryCursorStore { private const val FILE_NAME = "noop_oura_history_cursor" - private const val KEY_PREFIX = "history_cursor_" + + // #91: bumped from "history_cursor_" because the old key stored the misdecoded `bytes_left` byte-count; + // ignoring that stale value means the first post-upgrade connect does one clean full pull rather than + // seeking to a garbage ring-time. Kotlin twin of the Swift key bump (`.historyResumeRt.`). + private const val KEY_PREFIX = "history_resume_rt_" private fun prefs(ctx: Context): SharedPreferences = ctx.applicationContext.getSharedPreferences(FILE_NAME, Context.MODE_PRIVATE) diff --git a/android/app/src/main/java/com/noop/oura/Framing.kt b/android/app/src/main/java/com/noop/oura/Framing.kt index 82e41b39a..1f8ce226a 100644 --- a/android/app/src/main/java/com/noop/oura/Framing.kt +++ b/android/app/src/main/java/com/noop/oura/Framing.kt @@ -85,12 +85,13 @@ data class OuraRecord(val type: Int, val ringTimestamp: Long, val payload: IntAr } /** - * The parsed result of a 0x11 GetEvents response (OURA_PROTOCOL.md s5.2). Kotlin twin of the Swift - * `(cursor: UInt32, moreData: Bool)` tuple. `cursor` is the new resume cursor (an unsigned 32-bit ring - * timestamp carried as a Long, 0..0xFFFFFFFF); `moreData` is true while the ring still has banked events - * to hand over. + * The parsed result of a 0x11 GetEvents response, per open_oura's `EventBatchSummary` (events.rs): + * `events_received:1 sleep_analysis_progress:1 bytes_left:4LE`. Kotlin twin of the Swift + * `(eventsReceived, bytesLeft, moreData)` tuple. `bytesLeft` is the remaining-byte count of the drain + * (carried as a Long, 0..0xFFFFFFFF); `moreData` is true while `bytesLeft > 0`. There is NO resume cursor + * in this packet (#91 — see parseGetEventsResponse). */ -data class GetEventsSummary(val cursor: Long, val moreData: Boolean) +data class GetEventsSummary(val eventsReceived: Int, val bytesLeft: Long, val moreData: Boolean) object OuraFraming { /** The secure-session / extended opcode. Per OURA_PROTOCOL.md s2.2 / s4.1. */ @@ -115,20 +116,24 @@ object OuraFraming { const val minRecordLen = 4 /** - * Parse a 0x11 GetEvents response body: `status:1 sub_status:1 last_ring_timestamp:4LE pad:2` - * (OURA_PROTOCOL.md s5.2). `status` 0x00 = empty/no more; any other value = data follows. The - * `last_ring_timestamp` is the new cursor to resume the fetch from. Returns null on a short body - * (never guesses a cursor). Kotlin twin of Swift's parseGetEventsResponse; `cursor` is the unsigned - * 32-bit ring timestamp carried as a Long (0..0xFFFFFFFF). + * Parse a 0x11 GetEvents response body per open_oura's `EventBatchSummary` (events.rs): + * `events_received:1 sleep_analysis_progress:1 bytes_left:4LE`. The drain loop runs until + * `bytes_left == 0` (sync-orchestration.md). There is NO resume cursor in this packet — the resume + * position is a CLIENT-managed event-envelope ring-time (`nextEventToSync`), never read back here. + * Returns null on a short body. Kotlin twin of Swift's parseGetEventsResponse. + * + * #91: NOOP previously decoded bytes[2..5] as a `last_ring_timestamp` cursor and persisted it. Those + * bytes are `bytes_left` (a remaining-byte count, ~800 KB full → 0); comparing two byte-counts across + * sessions as clocks minted a phantom "regression" → reset to 0 → full re-dump every connect. */ fun parseGetEventsResponse(body: IntArray): GetEventsSummary? { if (body.size < 6) return null - val status = body[0] - val cursor = (body[2].toLong() and 0xFFL) or + val eventsReceived = body[0] and 0xFF + val bytesLeft = (body[2].toLong() and 0xFFL) or ((body[3].toLong() and 0xFFL) shl 8) or ((body[4].toLong() and 0xFFL) shl 16) or ((body[5].toLong() and 0xFFL) shl 24) - return GetEventsSummary(cursor = cursor, moreData = status != 0x00) + return GetEventsSummary(eventsReceived = eventsReceived, bytesLeft = bytesLeft, moreData = bytesLeft > 0L) } /** diff --git a/android/app/src/test/java/com/noop/oura/FramingTest.kt b/android/app/src/test/java/com/noop/oura/FramingTest.kt index 29f08319c..dddd00dff 100644 --- a/android/app/src/test/java/com/noop/oura/FramingTest.kt +++ b/android/app/src/test/java/com/noop/oura/FramingTest.kt @@ -62,21 +62,34 @@ class FramingTest { @Test fun testParseGetEventsResponseMoreDataFollows() { - // 11 08 + // 11 08 0x12345678> val outer = OuraFraming.parseOuterFrame(bytes("1108ff00785634120000")) assertEquals(OuraFraming.getEventsResponseOp, outer?.op) val summary = OuraFraming.parseGetEventsResponse(outer!!.body) - assertEquals(0x1234_5678L, summary?.cursor) - assertEquals(true, summary?.moreData) + assertEquals(0xff, summary?.eventsReceived) + assertEquals(0x1234_5678L, summary?.bytesLeft) + assertEquals(true, summary?.moreData) // bytes_left > 0 -> ring has more } @Test - fun testParseGetEventsResponseNoMoreData() { - // status 0x00 -> caught up, no more data. - val outer = OuraFraming.parseOuterFrame(bytes("11080000785634120000")) + fun testParseGetEventsResponseTerminalIsBytesLeftZero() { + // The terminal packet zero-fills bytes_left (00 00 00 00). + val outer = OuraFraming.parseOuterFrame(bytes("11080000000000000000")) val summary = OuraFraming.parseGetEventsResponse(outer!!.body) - assertEquals(0x1234_5678L, summary?.cursor) - assertEquals(false, summary?.moreData) + assertEquals(0L, summary?.bytesLeft) + assertEquals(false, summary?.moreData) // bytes_left == 0 -> drain complete + } + + // #91: bytes_left is a remaining-BYTE count, not a cursor. A later summary with a SMALLER bytes_left is + // normal draining (not a "regression"), and while > 0 the loop must keep going. Mirrors Swift. + @Test + fun testParseGetEventsResponseSmallerBytesLeftStillMeansMoreData() { + val first = OuraFraming.parseGetEventsResponse(bytes("ff00746e05000000")) // bytes_left 355956 + val later = OuraFraming.parseGetEventsResponse(bytes("ff000d0d00000000")) // bytes_left 3341 (< first) + assertEquals(355_956L, first?.bytesLeft) + assertEquals(3_341L, later?.bytesLeft) + assertEquals(true, first?.moreData) + assertEquals(true, later?.moreData) // still draining -- NOT a regression/terminal } @Test diff --git a/docs/OURA_PROTOCOL.md b/docs/OURA_PROTOCOL.md index dc578cbec..5701536d8 100644 --- a/docs/OURA_PROTOCOL.md +++ b/docs/OURA_PROTOCOL.md @@ -245,15 +245,24 @@ Gen 5 example `0912 020100 020103 010001 090329 665544332211`. [open_oura-r5] ### 5.2 GetEvents response / summary (`0x11`) ``` -11 08 +11 08 ``` -[open_ring] -- `status` - `0x00` = empty/no more; `0xFF` = data follows (event records arrive as inner TLV stream, §2.3). [open_ring] -- `last_ring_timestamp` - new cursor value to use next fetch. +This is open_oura's `EventBatchSummary` (`crates/oura-protocol/src/events.rs`). [open_ring] +- `events_received` - count of decoded events in this batch (0 on the terminal packet). +- `sleep_analysis_progress` - progress only, not a gate. +- `bytes_left` - remaining bytes of the drain. **Loop while `bytes_left > 0`; done at `0`.** +- **There is NO resume cursor in this packet.** The resume position is CLIENT-managed — the newest + event-envelope ring-time you've stored (open_oura's `nextEventToSync`), persisted per §5.3, never read + back from the summary. + +> **#91 (fixed):** NOOP originally decoded bytes 2-5 as a `last_ring_timestamp` cursor and persisted it. +> Those bytes are `bytes_left` — a remaining-**byte** count (~800 KB full → 0), not a clock. Persisting it +> and comparing next session's smaller `bytes_left` against last session's minted a phantom "ring-time +> regression", which reset the cursor to 0 and re-dumped the ring's entire banked history on every connect. ### 5.3 Canonical fetch loop (NOOP) -1. SyncTime (§5.4). 2. Send `0x10` with stored cursor, `max=255`. 3. Receive inner TLV records (§6). 4. ~100 ms later send ack-fetch (`max=0`, cursor = `last_ring_timestamp`) to advance. 5. Repeat until `status=0x00`. [open_ring] -6. Optionally `28 01 00` to flush flash-buffered events first. [open_ring] +1. SyncTime (§5.4). 2. Send `0x10` with the persisted resume ring-time (`nextEventToSync`), `max=255`. 3. Receive inner TLV records (§6). 4. ~100 ms later send ack-fetch (`max=0`) to advance the drain. 5. Repeat until `bytes_left = 0`. 6. On completion, persist the resume cursor = the newest **stored** event ring-time; a stored sample older than where you sought means the ring rebooted (or ignored the seek) → next connect does a full pull from 0. [open_ring] +7. Optionally `28 01 00` to flush flash-buffered events first. [open_ring] ### 5.4 SyncTime (`0x12`) ``` From 9100e2b5bea06adff7a470f07769d69657fc478d Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Thu, 9 Jul 2026 16:39:21 +0200 Subject: [PATCH 19/21] Oura: gate the UTC anchor on ring-time plausibility, not just epoch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The anchor-set paths (0x42 time-sync, 0x85 RTC beacon) validated only the epoch (2020-2035). A framing desync can hand an anchor record a plausible epoch paired with a GARBAGE ring-time in the billions (the 4 rt bytes read off a wrong offset). Anchoring on that pins anchorRingTime ~2e9 ticks from every real sample, so the anchor-relative phantom guard then drops ALL genuine history — the ring's data starves even though the epoch looked fine. Add isPlausibleAnchorRingTime (<= 500M ticks ≈ 579 days uptime, mirroring OuraLiveSource.maxPlausibleResumeTicks) and require BOTH halves before seating an anchor, in both drivers. OuraLiveSource now mirrors the full gate so "UTC time anchor acquired" never logs on a sync the driver rejected, and names which half failed (epoch vs ring-time). Tests: OuraProtocol 102/0 (+3 Swift: ring-time bounds, garbage-rt time-sync, garbage-rt beacon; mirrored in the Kotlin twin). docs/OURA_PROTOCOL.md §5.5 documents the two-half anchor-set gate. Co-Authored-By: Claude Opus 4.8 --- .../Sources/OuraProtocol/OuraDriver.swift | 27 ++++++++- .../OuraProtocolTests/OuraDriverTests.swift | 46 +++++++++++++++ Strand/BLE/OuraLiveSource.swift | 12 +++- .../main/java/com/noop/ble/OuraLiveSource.kt | 14 ++++- .../src/main/java/com/noop/oura/OuraDriver.kt | 24 ++++++++ .../test/java/com/noop/oura/OuraDriverTest.kt | 56 +++++++++++++++++++ docs/OURA_PROTOCOL.md | 1 + 7 files changed, 174 insertions(+), 6 deletions(-) diff --git a/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift b/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift index 230083678..b4825cd12 100644 --- a/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift +++ b/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift @@ -269,6 +269,16 @@ public final class OuraDriver { return seconds * 1000 // safe: bounded input, cannot overflow } + /// Ceiling for a plausible anchor RING-TIME (100 ms ticks, boot-relative): 500M ticks ≈ 579 days of + /// uptime. The epoch gate alone is not enough — a misframed 0x42/0x85 can carry a plausible epoch + /// (2020-2035) yet a GARBAGE ring-time (the 4 rt bytes read off a wrong offset land in the billions). + /// Anchoring on that pins `anchorRingTime` to a value ~2e9 ticks away from every real sample, so the + /// anchor-relative phantom guard then drops ALL genuine history — the ring's data starves even though + /// the epoch looked fine. A consumer ring reboots/charges (resetting its clock) long before 579 days, + /// so a ring-time above this in an anchor record is corrupt and never trusted. Mirrors the + /// `maxPlausibleResumeTicks` ceiling OuraLiveSource applies to the persisted resume cursor. + private static let maxPlausibleAnchorRingTime: UInt32 = 500_000_000 + /// True when `seconds` falls inside the anchor plausibility window (2020-01-01 .. 2035-01-01), i.e. the /// `.timeSync` / `.rtcBeacon` ingest would accept it. A record whose epoch is outside this is silently /// ignored so a garbage value can't anchor history to ~1970. Exposed READ-ONLY so OuraLiveSource can log @@ -277,6 +287,14 @@ public final class OuraDriver { plausibleAnchorMs(fromEpochSeconds: seconds) != nil } + /// True when `ringTime` is a plausible boot-relative ring-time for an anchor record (≤ the 579-day + /// ceiling). A misframed anchor record can pass `isPlausibleAnchorEpoch` yet carry a garbage ring-time; + /// this is the second half of the gate. Exposed READ-ONLY so OuraLiveSource can log WHY an anchor was + /// rejected (#91) without duplicating the bound. Pure. + public static func isPlausibleAnchorRingTime(_ ringTime: UInt32) -> Bool { + ringTime <= maxPlausibleAnchorRingTime + } + // MARK: - Record ingest (decode) /// Decode one parsed TLV inner record into zero or more events. A malformed/short record (or an @@ -351,7 +369,11 @@ public final class OuraDriver { // raw value (a misaligned/corrupt record deep in the backlog); a naive `* 1000` overflows Int64 // and traps. plausibleAnchorMs bounds-checks BEFORE multiplying, so an implausible value is // safely ignored (honest: never anchors to a garbage time) instead of crashing. - if let ms = Self.plausibleAnchorMs(fromEpochSeconds: ts.epochMs) { + // Both halves must be plausible: a plausible epoch paired with a garbage ring-time (misframed + // record) would pin the anchor ~2e9 ticks off and starve all real history (see + // maxPlausibleAnchorRingTime). Reject unless BOTH the epoch and the ring-time check out. + if let ms = Self.plausibleAnchorMs(fromEpochSeconds: ts.epochMs), + Self.isPlausibleAnchorRingTime(ts.ringTimestamp) { anchorUtcMs = ms anchorRingTime = ts.ringTimestamp } @@ -360,7 +382,8 @@ public final class OuraDriver { // Secondary UTC anchor (s5.5, 1s granularity): only fills in while no 0x42 anchor exists yet // this session, so a coarser beacon never overrides the primary time-sync anchor. guard let r = OuraDecoders.decodeRtcBeacon(record) else { return [] } - if anchorUtcMs == nil, let ms = Self.plausibleAnchorMs(fromEpochSeconds: Int64(r.unixSeconds)) { + if anchorUtcMs == nil, let ms = Self.plausibleAnchorMs(fromEpochSeconds: Int64(r.unixSeconds)), + Self.isPlausibleAnchorRingTime(r.ringTimestamp) { anchorUtcMs = ms anchorRingTime = r.ringTimestamp } diff --git a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift index af5a37789..613895ce8 100644 --- a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift +++ b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift @@ -78,6 +78,17 @@ final class OuraDriverTests: XCTestCase { XCTAssertFalse(OuraDriver.isPlausibleAnchorEpoch(0)) // epoch 0 — the ~1970 anchor #91 must avoid } + func testIsPlausibleAnchorRingTimeBounds() { + // The anchor ring-time ceiling is 500M ticks (≈579 days uptime). Second half of the anchor gate: + // a plausible epoch is not enough if the record's ring-time is garbage. OuraLiveSource reads this + // same predicate to log WHY an anchor was rejected (#91), so the boundary is pinned here. + XCTAssertTrue(OuraDriver.isPlausibleAnchorRingTime(0)) // boot (rt=0), inclusive min + XCTAssertTrue(OuraDriver.isPlausibleAnchorRingTime(1_624_998)) // a real on-device resume cursor + XCTAssertTrue(OuraDriver.isPlausibleAnchorRingTime(500_000_000)) // ceiling, inclusive max + XCTAssertFalse(OuraDriver.isPlausibleAnchorRingTime(500_000_001)) // one tick past the ceiling + XCTAssertFalse(OuraDriver.isPlausibleAnchorRingTime(2_055_602_179)) // the on-device garbage cursor (#91) + } + func testFactoryResetStatusDrivesNeedsKeyInstall() { let d = OuraDriver(ringGen: .gen3, authKey: key) _ = d.nextStep(after: .ready) @@ -288,6 +299,41 @@ final class OuraDriverTests: XCTestCase { "a later RTC beacon must not displace an already-set time-sync anchor") } + func testTimeSyncWithPlausibleEpochButGarbageRingTimeDoesNotAnchor() { + // A misframed 0x42 can carry a plausible epoch (2020-2035) yet a garbage ring-time (rt bytes read + // off a wrong offset -> billions). Anchoring on it would pin anchorRingTime ~2e9 ticks from every + // real sample and starve all history. Both halves of the gate must pass: garbage rt -> no anchor. + let d = OuraDriver(ringGen: .gen3, authKey: key) + let epochSeconds: Int64 = 1_700_000_000 // perfectly plausible + let garbageRt: UInt32 = 2_055_602_179 // the real on-device garbage value (#91) + let events = d.ingest(record: OuraRecord(type: OuraEventTag.timeSync.rawValue, + ringTimestamp: garbageRt, payload: le8(epochSeconds) + [0x00])) + // The event is still surfaced (honest decode) — it just must not become the anchor. + XCTAssertEqual(events, [.timeSync(OuraTimeSync(ringTimestamp: garbageRt, epochMs: epochSeconds, tzOffsetSeconds: 0))]) + XCTAssertFalse(d.hasAnchor, "a garbage ring-time must not set an anchor even with a plausible epoch") + XCTAssertNil(d.unixSeconds(forRingTimestamp: 1_624_998), "no anchor -> a real ring-time still can't resolve") + + // A subsequent WELL-FORMED time-sync (plausible epoch AND plausible rt) then anchors cleanly. + _ = d.ingest(record: OuraRecord(type: OuraEventTag.timeSync.rawValue, ringTimestamp: 1_624_998, + payload: le8(epochSeconds) + [0x00])) + XCTAssertTrue(d.hasAnchor) + XCTAssertEqual(d.unixSeconds(forRingTimestamp: 1_624_998), Int(epochSeconds)) + } + + func testRtcBeaconWithPlausibleEpochButGarbageRingTimeDoesNotAnchor() { + // Same guard on the secondary 0x85 path: a plausible unix_s paired with a garbage ring-time is a + // misframed beacon and must not anchor. + let d = OuraDriver(ringGen: .gen3, authKey: key) + let unixSeconds = 1_700_000_500 + let garbageRt: UInt32 = 3_000_000_000 + let payload: [UInt8] = [ + UInt8(unixSeconds & 0xFF), UInt8((unixSeconds >> 8) & 0xFF), + UInt8((unixSeconds >> 16) & 0xFF), UInt8((unixSeconds >> 24) & 0xFF), + ] + _ = d.ingest(record: OuraRecord(type: OuraEventTag.rtcBeacon.rawValue, ringTimestamp: garbageRt, payload: payload)) + XCTAssertFalse(d.hasAnchor, "a garbage beacon ring-time must not set an anchor") + } + func testStopClearsTheAnchor() { let d = OuraDriver(ringGen: .gen3, authKey: key) let payload = le8(1_700_000_000) + [0x00] diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index 8459545d0..3f50498f3 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -761,13 +761,19 @@ public final class OuraLiveSource: NSObject, ObservableObject { // epoch; only announce "acquired" when the sync ACTUALLY anchored (the old unconditional // "acquired" line fired even on a rejected sync). `epochMs` holds the raw wire value, which // is unix SECONDS despite the name (s6.11). - if OuraDriver.isPlausibleAnchorEpoch(ts.epochMs) { + // The driver anchors only when BOTH the epoch AND the ring-time are plausible (a misframed + // 0x42 can carry a good epoch but a garbage ring-time that would pin the anchor ~2e9 ticks + // off and starve all history). Mirror that full gate here so "acquired" never fires on a + // sync the driver actually rejected, and name WHICH half failed. + if OuraDriver.isPlausibleAnchorEpoch(ts.epochMs), OuraDriver.isPlausibleAnchorRingTime(ts.ringTimestamp) { if !loggedAnchor { loggedAnchor = true log("Oura: UTC time anchor acquired - history-fetched samples now get their real time") } - } else { + } else if !OuraDriver.isPlausibleAnchorEpoch(ts.epochMs) { log("Oura: 0x42 time-sync REJECTED - implausible epoch \(ts.epochMs)s (outside the 2020–2035 anchor window); history samples stay unanchored (#91)") + } else { + log("Oura: 0x42 time-sync REJECTED - implausible ring-time \(ts.ringTimestamp) ticks (misframed record; history samples stay unanchored) (#91)") } // The 0x42 time-sync can arrive ANYWHERE in a history-fetch stream, not necessarily first. // Anything parked while unanchored gets its real time retroactively the moment an anchor lands. @@ -779,6 +785,8 @@ public final class OuraLiveSource: NSObject, ObservableObject { // IMPLAUSIBLE-epoch beacon is a real failure (it can never anchor), so log just that. if !OuraDriver.isPlausibleAnchorEpoch(Int64(r.unixSeconds)) { log("Oura: 0x85 RTC beacon REJECTED - implausible epoch \(r.unixSeconds)s (outside the 2020–2035 anchor window) (#91)") + } else if !OuraDriver.isPlausibleAnchorRingTime(r.ringTimestamp) { + log("Oura: 0x85 RTC beacon REJECTED - implausible ring-time \(r.ringTimestamp) ticks (misframed record) (#91)") } case .tierB(let summary): diff --git a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt index 77c642e79..9932f13b4 100644 --- a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt +++ b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt @@ -1183,14 +1183,21 @@ class OuraLiveSource( // epoch; only announce "acquired" when the sync ACTUALLY anchored (the old unconditional // "acquired" line fired even on a rejected sync). `epochMs` holds the raw wire value, which // is unix SECONDS despite the name (s6.11). - if (d.isPlausibleAnchorEpoch(e.value.epochMs)) { + // The driver anchors only when BOTH the epoch AND the ring-time are plausible (a misframed + // 0x42 can carry a good epoch but a garbage ring-time that would pin the anchor ~2e9 ticks + // off and starve all history). Mirror that full gate here so "acquired" never fires on a + // sync the driver actually rejected, and name WHICH half failed. + if (d.isPlausibleAnchorEpoch(e.value.epochMs) && d.isPlausibleAnchorRingTime(e.value.ringTimestamp)) { if (!loggedAnchor) { loggedAnchor = true log("Oura: UTC time anchor acquired - history-fetched samples now get their real time") } - } else { + } else if (!d.isPlausibleAnchorEpoch(e.value.epochMs)) { log("Oura: 0x42 time-sync REJECTED - implausible epoch ${e.value.epochMs}s (outside the " + "2020–2035 anchor window); history samples stay unanchored (#91)") + } else { + log("Oura: 0x42 time-sync REJECTED - implausible ring-time ${e.value.ringTimestamp} ticks " + + "(misframed record; history samples stay unanchored) (#91)") } // The 0x42 time-sync can arrive ANYWHERE in a history-fetch stream, not necessarily first. // Anything parked while unanchored gets its real time retroactively the moment it lands. @@ -1203,6 +1210,9 @@ class OuraLiveSource( if (!d.isPlausibleAnchorEpoch(e.value.unixSeconds)) { log("Oura: 0x85 RTC beacon REJECTED - implausible epoch ${e.value.unixSeconds}s (outside " + "the 2020–2035 anchor window) (#91)") + } else if (!d.isPlausibleAnchorRingTime(e.value.ringTimestamp)) { + log("Oura: 0x85 RTC beacon REJECTED - implausible ring-time ${e.value.ringTimestamp} ticks " + + "(misframed record) (#91)") } } is OuraEvent.TierB -> { diff --git a/android/app/src/main/java/com/noop/oura/OuraDriver.kt b/android/app/src/main/java/com/noop/oura/OuraDriver.kt index dd9323395..fee6eb2ec 100644 --- a/android/app/src/main/java/com/noop/oura/OuraDriver.kt +++ b/android/app/src/main/java/com/noop/oura/OuraDriver.kt @@ -335,6 +335,10 @@ class OuraDriver( // A secondary (beacon) anchor never displaces an already-set primary (time-sync) anchor. if (!preferPrimary && anchorUtcMs != null) return val ms = plausibleAnchorMs(epochSeconds) ?: return + // Both halves must be plausible: a plausible epoch paired with a garbage ring-time (misframed + // record) would pin the anchor ~2e9 ticks off and starve all real history (#91). Reject unless the + // ring-time also checks out. + if (!isPlausibleAnchorRingTime(ringTimestamp)) return anchorUtcMs = ms anchorRingTime = ringTimestamp } @@ -359,6 +363,15 @@ class OuraDriver( */ fun isPlausibleAnchorEpoch(epochSeconds: Long): Boolean = plausibleAnchorMs(epochSeconds) != null + /** + * True when [ringTime] is a plausible boot-relative ring-time for an anchor record (<= the 579-day + * ceiling, 500M ticks at 100 ms/tick). A misframed anchor record can pass [isPlausibleAnchorEpoch] yet + * carry a garbage ring-time (billions); this is the second half of the gate. Exposed READ-ONLY so + * OuraLiveSource can log WHY an anchor was rejected (#91) without duplicating the bound. Pure. Mirrors + * the maxPlausibleResumeTicks ceiling OuraLiveSource applies to the persisted resume cursor. + */ + fun isPlausibleAnchorRingTime(ringTime: Long): Boolean = ringTime <= MAX_PLAUSIBLE_ANCHOR_RING_TIME + // MARK: - Record ingest (decode) /** @@ -584,6 +597,17 @@ class OuraDriver( private const val MIN_PLAUSIBLE_EPOCH_SECONDS = 1_577_836_800L private const val MAX_PLAUSIBLE_EPOCH_SECONDS = 2_051_222_400L + /** + * Ceiling for a plausible anchor RING-TIME (100 ms ticks, boot-relative): 500M ticks ≈ 579 days of + * uptime. The epoch gate alone is not enough — a misframed 0x42/0x85 can carry a plausible epoch yet + * a GARBAGE ring-time (rt bytes off a wrong offset -> billions). Anchoring on that pins the anchor + * ~2e9 ticks from every real sample, so the phantom guard then drops ALL genuine history and the + * ring's data starves even though the epoch looked fine (#91). A consumer ring reboots/charges long + * before 579 days. Byte-identical to Swift's maxPlausibleAnchorRingTime and OuraLiveSource's + * maxPlausibleResumeTicks. + */ + private const val MAX_PLAUSIBLE_ANCHOR_RING_TIME = 500_000_000L + /** * Anchor-relative plausibility window for a history-fetched sample (phantom-record guard). History * is always in the recent past relative to the session anchor (~now): a real sample can be at most a diff --git a/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt b/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt index afd83d061..3bcde3f09 100644 --- a/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt +++ b/android/app/src/test/java/com/noop/oura/OuraDriverTest.kt @@ -37,6 +37,19 @@ class OuraDriverTest { assertFalse(d.isPlausibleAnchorEpoch(0L)) // epoch 0 — the ~1970 anchor #91 must avoid } + @Test + fun testIsPlausibleAnchorRingTimeBounds() { + // The anchor ring-time ceiling is 500M ticks (≈579 days uptime). Second half of the anchor gate: a + // plausible epoch is not enough if the record's ring-time is garbage. Twin of the Swift + // OuraDriverTests.testIsPlausibleAnchorRingTimeBounds. + val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) + assertTrue(d.isPlausibleAnchorRingTime(0L)) // boot (rt=0), inclusive min + assertTrue(d.isPlausibleAnchorRingTime(1_624_998L)) // a real on-device resume cursor + assertTrue(d.isPlausibleAnchorRingTime(500_000_000L)) // ceiling, inclusive max + assertFalse(d.isPlausibleAnchorRingTime(500_000_001L)) // one tick past the ceiling + assertFalse(d.isPlausibleAnchorRingTime(2_055_602_179L)) // the on-device garbage cursor (#91) + } + // MARK: - Full happy-path step sequence (auth -> enable triplet -> streaming) @Test @@ -349,6 +362,49 @@ class OuraDriverTest { ) } + /** + * A misframed 0x42 can carry a plausible epoch (2020-2035) yet a garbage ring-time (rt bytes read off a + * wrong offset -> billions). Anchoring on it would pin anchorRingTime ~2e9 ticks from every real sample + * and starve all history. Both halves of the gate must pass. Kotlin twin of Swift's + * testTimeSyncWithPlausibleEpochButGarbageRingTimeDoesNotAnchor. + */ + @Test + fun testTimeSyncWithPlausibleEpochButGarbageRingTimeDoesNotAnchor() { + val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) + val epochSeconds = 1_700_000_000L // perfectly plausible + val garbageRt = 2_055_602_179L // the real on-device garbage value (#91) + d.ingest( + OuraRecord(type = OuraEventTag.TIME_SYNC.raw, ringTimestamp = garbageRt, payload = le8(epochSeconds) + intArrayOf(0x00)), + ) + assertFalse(d.hasAnchor) // garbage ring-time must not set an anchor + assertNull(d.unixSeconds(forRingTimestamp = 1_624_998L)) // no anchor -> a real ring-time still can't resolve + + // A subsequent WELL-FORMED time-sync (plausible epoch AND plausible rt) then anchors cleanly. + d.ingest( + OuraRecord(type = OuraEventTag.TIME_SYNC.raw, ringTimestamp = 1_624_998L, payload = le8(epochSeconds) + intArrayOf(0x00)), + ) + assertTrue(d.hasAnchor) + assertEquals(epochSeconds, d.unixSeconds(forRingTimestamp = 1_624_998L)) + } + + /** + * Same guard on the secondary 0x85 path: a plausible unix_s paired with a garbage ring-time is a + * misframed beacon and must not anchor. Kotlin twin of Swift's + * testRtcBeaconWithPlausibleEpochButGarbageRingTimeDoesNotAnchor. + */ + @Test + fun testRtcBeaconWithPlausibleEpochButGarbageRingTimeDoesNotAnchor() { + val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) + val unixSeconds = 1_700_000_500L + val garbageRt = 3_000_000_000L + val payload = intArrayOf( + (unixSeconds and 0xFF).toInt(), ((unixSeconds shr 8) and 0xFF).toInt(), + ((unixSeconds shr 16) and 0xFF).toInt(), ((unixSeconds shr 24) and 0xFF).toInt(), + ) + d.ingest(OuraRecord(type = OuraEventTag.RTC_BEACON.raw, ringTimestamp = garbageRt, payload = payload)) + assertFalse(d.hasAnchor) // garbage beacon ring-time must not set an anchor + } + @Test fun testStopClearsTheAnchor() { val d = OuraDriver(ringGen = OuraRingGen.GEN3, authKey = key) diff --git a/docs/OURA_PROTOCOL.md b/docs/OURA_PROTOCOL.md index 5701536d8..e18ae5dde 100644 --- a/docs/OURA_PROTOCOL.md +++ b/docs/OURA_PROTOCOL.md @@ -282,6 +282,7 @@ subsequent `0x42` event, §5.5). [open_ring] ### 5.5 Ring-time → UTC anchoring - The ring clock is in **ticks**: default **100 ms/tick** (10 Hz); burst mode **1 ms/tick** (`factor_flag=1`). [open_ring] - Anchor from event `0x42` (time-sync ind, §6.11): set `anchor.utc_ms` from the event's epoch and `anchor.ring_time` from current `ringTimestamp`. +- **Anchor-set gate (BOTH halves must be plausible).** The anchor is set only when the event's **epoch** is inside the 2020–2035 window (`OuraDriver.isPlausibleAnchorEpoch`, §6.11) AND its **`ring_time`** is a plausible boot-relative tick count — `≤ 500M ticks ≈ 579 days` uptime (`OuraDriver.isPlausibleAnchorRingTime`). The epoch gate alone is insufficient: a framing desync (§2.4) can hand a `0x42`/`0x85` a perfectly plausible epoch paired with a **garbage `ring_time` in the billions** (the 4 rt bytes read off a wrong offset). Anchoring on that pins `anchor.ring_time` ~2e9 ticks from every real sample, so the anchor-relative phantom guard below then drops **all** genuine history — the ring's data starves even though the epoch looked fine (#91). On live Gen 3 (2026-07-09) a pre-guard build persisted a garbage resume cursor `2055602179`; the same class of value must never seat an anchor. The 500M ceiling mirrors `OuraLiveSource.maxPlausibleResumeTicks` (the persisted-cursor clamp). Both the Swift and Kotlin drivers enforce this in `.timeSync`/`.rtcBeacon` ingest. - Conversion: `utc_ms = anchor.utc_ms + factor × (target_rt − anchor.ring_time)`, `factor ∈ {100,1}`. NOOP v1 uses `factor = 100` only (burst mode not yet modeled; the low-rate temp/SpO2/sleep streams are 100 ms/tick, confirmed by a real night's `s:`/`e:` window converting to ~9 h). [open_ring] - **Phantom-record guard (anchor-relative plausibility).** A framing desync (§2.4) can mint a record with a garbage `ring_time` that, at 100 ms/tick, converts to a date **days-to-years from the anchor yet still inside the loose 2020–2035 epoch gate** — e.g. `rt ≈ 1.9e9 → +6 years (2033)`, `rt ≈ 16.7M → +19 days`. On live Gen 3 (2026-07-08) this scattered ~25% of a night's skin-temp rows across 2020–2034, so they never landed on the correct calendar day. Rule: a history-fetched sample is always in the **recent past** relative to the session anchor (≈ now via `0x42`); reject any conversion more than **+1 day** after or **−90 days** before the anchor (`OuraDriver.unixSeconds` returns nil → the caller drops/parks it, honest-data invariant). This is the single chokepoint every history sample passes; it complements the value-level gates (SpO2 §7.1, skin-temp funnel). - On `0x41` (ring start) with `rt` regression → invalidate anchor (zero it). [open_ring] From d56871be2ec9d47a7f292a2c4f333c36d062e4bc Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Fri, 10 Jul 2026 10:37:20 +0200 Subject: [PATCH 20/21] Oura: bound history-drain with full-pull fast-exit (fix #92 infinite loop) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The GetEvents 0x11 drain re-dumps the ring's entire log from boot on every cursor-0 continuation, so when bytesLeft plateaus instead of reaching 0 the drain looped forever. Bound it three ways: a full-pull-pass fast-exit (stop after one complete pass once maxStoredRingTime resolves against the anchor), a stall guard (bytesLeft not advancing by drainProgressEpsilon), and a wall-clock deadline. On any forced stop we commit maxStoredRingTime if it resolves and otherwise HOLD the cursor — never reset to 0, which was what minted the re-dump. Verified on-device: stops after 2 reads, clean resume. --- Strand/BLE/OuraLiveSource.swift | 144 +++++++++++++++--- .../main/java/com/noop/ble/OuraLiveSource.kt | 19 ++- 2 files changed, 131 insertions(+), 32 deletions(-) diff --git a/Strand/BLE/OuraLiveSource.swift b/Strand/BLE/OuraLiveSource.swift index 3f50498f3..778d82168 100644 --- a/Strand/BLE/OuraLiveSource.swift +++ b/Strand/BLE/OuraLiveSource.swift @@ -277,6 +277,28 @@ public final class OuraLiveSource: NSObject, ObservableObject { /// it ignored the seek), so the persisted cursor is stale and must fall back to a full pull. private var resumeCursorAtFetchStart: UInt32 = 0 private var sawPreResumeData = false + /// #91-followup: convergence guard for the GetEvents drain. A healthy drain shrinks `bytesLeft` on + /// every continuation until it reaches 0. A ring doing a FULL PULL (cursor 0) while it is still + /// generating live events re-dumps from its log start each continuation, so `bytesLeft` floors and + /// oscillates instead of reaching 0 — an endless `get_events` loop (observed on-device 2026-07-10: + /// `bytesLeft` 1.29M → 338k → 338k…, re-dumping the ring's `rt=60466` boot text every cycle). We stop + /// the drain when `bytesLeft` stops strictly shrinking for `maxHistoryDrainStalls` consecutive reads, + /// or at a hard cycle cap, and on that forced stop we NEVER reset the cursor to 0 (that re-arms the + /// very full pull that loops). Reset per fetch pass in `fetchHistoryIfIdle`. + private var historyDrainCycles = 0 + private var historyDrainStalls = 0 + /// The SMALLEST `bytesLeft` seen this fetch. A continuation counts as progress only if it beats this by + /// at least `drainProgressEpsilon` — so the ~1 KB oscillation at the plateau (337k↔338k) reads as a + /// stall, not progress, and the guard converges instead of resetting on every tiny wobble. + private var minDrainBytesLeft: UInt32 = .max + /// Wall-clock start of the current fetch pass. A healthy full drain of this ring completes in ~30 s + /// (observed); the pathological re-dump takes ~72 s PER cycle and never ends, so a per-fetch deadline + /// bounds the worst case regardless of the summary cadence. Checked at each summary boundary. + private var historyFetchStartedAt: Date? + private static let maxHistoryDrainStalls = 2 + private static let maxHistoryDrainCycles = 500 + private static let maxHistoryFetchSeconds: TimeInterval = 120 + private static let drainProgressEpsilon: UInt32 = 8_192 /// Periodic re-fetch while connected, so an overnight-connected session (or one left open after a nap) /// picks up freshly-banked sleep data without needing a reconnect. Mirrors BLEManager's ~15 min /// periodic WHOOP history-offload floor. @@ -290,7 +312,11 @@ public final class OuraLiveSource: NSObject, ObservableObject { guard let driver, driver.phase == .streaming else { return } resumeCursorAtFetchStart = historyCursor sawPreResumeData = false - log("Oura: fetching history from cursor \(historyCursor) (\(describeCursor(historyCursor)))") + historyDrainCycles = 0 + historyDrainStalls = 0 + minDrainBytesLeft = .max + historyFetchStartedAt = Date() + log("Oura: fetching history from cursor \(historyCursor) (\(describeCursor(historyCursor))) [drain-guard v2]") advance(.startHistoryFetch(cursor: historyCursor)) } @@ -318,32 +344,95 @@ public final class OuraLiveSource: NSObject, ObservableObject { /// reset (or it ignored the seek), so we fall back to a full pull (0) next time rather than resume from /// a now-stale ring-time. private func handleHistorySummary(_ summary: (eventsReceived: UInt8, bytesLeft: UInt32, moreData: Bool)) { - if !summary.moreData { - // A resume cursor is only trustworthy if it resolves to a real time under the CURRENT anchor. - // A value that doesn't was stored against a TRANSIENT bad anchor (a misframed 0x85/0x42 whose - // epoch passed but whose ring-time was garbage) — persisting it would seek to nonsense next - // connect. Treat it, and a genuine reboot (sawPreResumeData), the same: full pull next time. - let resumeResolves = maxStoredRingTime > 0 && (driver?.unixSeconds(forRingTimestamp: maxStoredRingTime) != nil) - if sawPreResumeData { - log("Oura: history fetch caught up but the ring served data older than cursor \(resumeCursorAtFetchStart) - clock reset/seek ignored; next connect does a full pull") - historyCursor = 0 - OuraHistoryCursorStore.save(0, deviceId: deviceId) - } else if maxStoredRingTime > historyCursor, resumeResolves { - historyCursor = maxStoredRingTime - OuraHistoryCursorStore.save(maxStoredRingTime, deviceId: deviceId) - log("Oura: history fetch caught up (resume cursor \(historyCursor) [\(describeCursor(historyCursor))])") - } else if maxStoredRingTime > historyCursor { - log("Oura: history fetch caught up but resume cursor \(maxStoredRingTime) [\(describeCursor(maxStoredRingTime))] is a phantom (transient bad anchor?) - full pull next connect") - historyCursor = 0 - OuraHistoryCursorStore.save(0, deviceId: deviceId) + if summary.moreData { + // Convergence guard (#91-followup): a healthy drain shrinks `bytesLeft` on every continuation + // until 0. A full pull of a live-generating ring re-dumps from its log start each cycle, so + // `bytesLeft` floors/oscillates (observed ~340k) and never reaches 0 → an endless `get_events` + // loop. STOP (forcing `moreData=false`, so the driver issues no further continuation) when ANY + // of three bounds trips: + // • `bytesLeft` makes no real progress against its running minimum for `maxHistoryDrainStalls` + // consecutive reads (the plateau — the definitive stuck signal), + // • the fetch has run past `maxHistoryFetchSeconds` of wall clock (a healthy full drain of this + // ring finishes in ~30 s; the pathological one is ~72 s/cycle, so this caps the worst case + // regardless of the slow summary cadence), + // • a hard cycle cap (last-ditch backstop). + historyDrainCycles += 1 + if summary.bytesLeft + Self.drainProgressEpsilon <= minDrainBytesLeft { + minDrainBytesLeft = summary.bytesLeft + historyDrainStalls = 0 } else { - log("Oura: history fetch caught up (resume cursor unchanged \(historyCursor) [\(describeCursor(historyCursor))])") + historyDrainStalls += 1 + } + let elapsed = historyFetchStartedAt.map { Date().timeIntervalSince($0) } ?? 0 + // FULL-PULL fast-exit (the common, decisive case): this ring re-dumps its ENTIRE banked log on + // every `get_events` continuation of a cursor-0 pull, so ONE completed pass already delivered + // everything storable (all `check_sleep` windows + banked samples). Once the anchor has seated + // and we have banked real forward progress (a stored ring-time that resolves under the anchor), + // every further pass is pure re-dump — stop NOW rather than wait for a `bytesLeft==0` that never + // comes. Scoped to `resumeCursorAtFetchStart == 0` so a normal resumed pull still drains to zero. + let fullPullPassComplete = resumeCursorAtFetchStart == 0 + && historyDrainCycles >= 2 + && maxStoredRingTime > 0 + && (driver?.unixSeconds(forRingTimestamp: maxStoredRingTime) != nil) + let stalled = historyDrainStalls >= Self.maxHistoryDrainStalls + let timedOut = elapsed >= Self.maxHistoryFetchSeconds + let tooMany = historyDrainCycles >= Self.maxHistoryDrainCycles + if fullPullPassComplete || stalled || timedOut || tooMany { + let why = fullPullPassComplete ? "full-pull pass complete (banked through ring-time \(maxStoredRingTime))" + : stalled ? "bytesLeft \(summary.bytesLeft) not shrinking (\(historyDrainStalls) stalls)" + : timedOut ? "fetch ran \(Int(elapsed))s without completing" + : "hit cycle cap \(historyDrainCycles)" + log("Oura: history drain stopping (\(why)) after \(historyDrainCycles) reads - avoids a #91-style re-dump loop") + finalizeHistoryDrain(forced: true) + advance(.historyCursorAdvanced(cursor: historyCursor, moreData: false)) + return } + } else { + finalizeHistoryDrain(forced: false) } // The ack-fetch cursor is ignored by the ring (maxEvents=0 continuation); pass the resume point. advance(.historyCursorAdvanced(cursor: historyCursor, moreData: summary.moreData)) } + /// Commit the GetEvents resume cursor when a drain ends. `forced == true` means the convergence guard + /// stopped a non-converging drain (a re-dumping full pull): we must NOT reset to 0 there — that re-arms + /// the looping full pull — so we commit forward progress if a stored sample resolves, else HOLD the + /// current cursor. `forced == false` is the natural `bytesLeft == 0` completion and keeps the #91 logic + /// verbatim (a reboot or a phantom resume falls back to a full pull next connect). + private func finalizeHistoryDrain(forced: Bool) { + // A resume cursor is only trustworthy if it resolves to a real time under the CURRENT anchor. A value + // that doesn't was stored against a TRANSIENT bad anchor (a misframed 0x85/0x42 whose epoch passed + // but whose ring-time was garbage) — persisting it would seek to nonsense next connect. + let resumeResolves = maxStoredRingTime > 0 && (driver?.unixSeconds(forRingTimestamp: maxStoredRingTime) != nil) + if forced { + if maxStoredRingTime > historyCursor, resumeResolves { + historyCursor = maxStoredRingTime + OuraHistoryCursorStore.save(maxStoredRingTime, deviceId: deviceId) + log("Oura: history drain stopped early - resume advanced to \(historyCursor) [\(describeCursor(historyCursor))]") + } else { + log("Oura: history drain stopped early - resume cursor holds at \(historyCursor) [\(describeCursor(historyCursor))] (no forward-resolving sample yet)") + } + return + } + // Natural completion: a genuine reboot (sawPreResumeData) and a phantom resume both fall back to a + // full pull next time (0). + if sawPreResumeData { + log("Oura: history fetch caught up but the ring served data older than cursor \(resumeCursorAtFetchStart) - clock reset/seek ignored; next connect does a full pull") + historyCursor = 0 + OuraHistoryCursorStore.save(0, deviceId: deviceId) + } else if maxStoredRingTime > historyCursor, resumeResolves { + historyCursor = maxStoredRingTime + OuraHistoryCursorStore.save(maxStoredRingTime, deviceId: deviceId) + log("Oura: history fetch caught up (resume cursor \(historyCursor) [\(describeCursor(historyCursor))])") + } else if maxStoredRingTime > historyCursor { + log("Oura: history fetch caught up but resume cursor \(maxStoredRingTime) [\(describeCursor(maxStoredRingTime))] is a phantom (transient bad anchor?) - full pull next connect") + historyCursor = 0 + OuraHistoryCursorStore.save(0, deviceId: deviceId) + } else { + log("Oura: history fetch caught up (resume cursor unchanged \(historyCursor) [\(describeCursor(historyCursor))])") + } + } + /// Record a STORED history sample's ring-time toward the resume cursor (open_oura `nextEventToSync`). /// Called only where a sample resolved a REAL anchored time and was enqueued — never for the no-anchor /// wall-clock fallback. Also flags a reboot: a real sample older than where we sought this fetch. @@ -765,13 +854,13 @@ public final class OuraLiveSource: NSObject, ObservableObject { // 0x42 can carry a good epoch but a garbage ring-time that would pin the anchor ~2e9 ticks // off and starve all history). Mirror that full gate here so "acquired" never fires on a // sync the driver actually rejected, and name WHICH half failed. - if OuraDriver.isPlausibleAnchorEpoch(ts.epochMs), OuraDriver.isPlausibleAnchorRingTime(ts.ringTimestamp) { + if driver.acceptsAnchorEpoch(ts.epochMs), OuraDriver.isPlausibleAnchorRingTime(ts.ringTimestamp) { if !loggedAnchor { loggedAnchor = true log("Oura: UTC time anchor acquired - history-fetched samples now get their real time") } - } else if !OuraDriver.isPlausibleAnchorEpoch(ts.epochMs) { - log("Oura: 0x42 time-sync REJECTED - implausible epoch \(ts.epochMs)s (outside the 2020–2035 anchor window); history samples stay unanchored (#91)") + } else if !driver.acceptsAnchorEpoch(ts.epochMs) { + log("Oura: 0x42 time-sync REJECTED - implausible epoch \(ts.epochMs)s (not within ±7d of now / outside 2020–2035); history samples stay unanchored (#91)") } else { log("Oura: 0x42 time-sync REJECTED - implausible ring-time \(ts.ringTimestamp) ticks (misframed record; history samples stay unanchored) (#91)") } @@ -783,8 +872,8 @@ public final class OuraLiveSource: NSObject, ObservableObject { // #91: the 0x85 beacon is the SECONDARY anchor (fills the gap only until a 0x42 arrives). A // beacon ignored because a primary anchor already exists is NORMAL and not logged; only an // IMPLAUSIBLE-epoch beacon is a real failure (it can never anchor), so log just that. - if !OuraDriver.isPlausibleAnchorEpoch(Int64(r.unixSeconds)) { - log("Oura: 0x85 RTC beacon REJECTED - implausible epoch \(r.unixSeconds)s (outside the 2020–2035 anchor window) (#91)") + if !driver.acceptsAnchorEpoch(Int64(r.unixSeconds)) { + log("Oura: 0x85 RTC beacon REJECTED - implausible epoch \(r.unixSeconds)s (not within ±7d of now / outside 2020–2035) (#91)") } else if !OuraDriver.isPlausibleAnchorRingTime(r.ringTimestamp) { log("Oura: 0x85 RTC beacon REJECTED - implausible ring-time \(r.ringTimestamp) ticks (misframed record) (#91)") } @@ -1067,6 +1156,11 @@ extension OuraLiveSource: @preconcurrency CBCentralManagerDelegate { authKey: authKey().map { [UInt8]($0) }, allowTierB: true, allowKeyInstall: adoptIntent) + // Near-now anchor gate: the ring is `sync_time`-synced to this host on connect, so a genuine + // 0x42/0x85 beacon reads ≈ now. Injecting the host clock lets the driver reject a misframed beacon + // whose bytes decode to a wrong YEAR (e.g. 2021) before it can anchor and mis-stamp a batch of + // samples (the 2021-dated skin-temp rows). Refreshed every connect. + driver?.anchorReferenceEpochSeconds = Int64(Date().timeIntervalSince1970) reachedStreaming = false loggedFirstHR = false loggedFirstTemp = false diff --git a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt index 9932f13b4..bc996e9e7 100644 --- a/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt +++ b/android/app/src/main/java/com/noop/ble/OuraLiveSource.kt @@ -584,6 +584,11 @@ class OuraLiveSource( // TierB/ActivityInfo unconditionally - the Tier-discipline gate that matters lives there, not here. driver = OuraDriver(ringGen = ringGen, authKey = authKey(), allowTierB = true, allowKeyInstall = adoptIntent) + // Near-now anchor gate: the ring is sync_time-synced to this host on connect, so a genuine 0x42/0x85 + // beacon reads ≈ now. Injecting the host clock lets the driver reject a misframed beacon whose bytes + // decode to a wrong YEAR (e.g. 2021) before it can anchor and mis-stamp a batch of samples. Refreshed + // every connect. Twin of the Swift OuraLiveSource wiring. + driver?.anchorReferenceEpochSeconds = System.currentTimeMillis() / 1000L reassembler.reset() pendingInstallKey = null // a new connection starts with no install in flight _adoptPhase.value = AdoptPhase.Idle // a stale outcome must never drive the wizard's transition @@ -1187,14 +1192,14 @@ class OuraLiveSource( // 0x42 can carry a good epoch but a garbage ring-time that would pin the anchor ~2e9 ticks // off and starve all history). Mirror that full gate here so "acquired" never fires on a // sync the driver actually rejected, and name WHICH half failed. - if (d.isPlausibleAnchorEpoch(e.value.epochMs) && d.isPlausibleAnchorRingTime(e.value.ringTimestamp)) { + if (d.acceptsAnchorEpoch(e.value.epochMs) && d.isPlausibleAnchorRingTime(e.value.ringTimestamp)) { if (!loggedAnchor) { loggedAnchor = true log("Oura: UTC time anchor acquired - history-fetched samples now get their real time") } - } else if (!d.isPlausibleAnchorEpoch(e.value.epochMs)) { - log("Oura: 0x42 time-sync REJECTED - implausible epoch ${e.value.epochMs}s (outside the " + - "2020–2035 anchor window); history samples stay unanchored (#91)") + } else if (!d.acceptsAnchorEpoch(e.value.epochMs)) { + log("Oura: 0x42 time-sync REJECTED - implausible epoch ${e.value.epochMs}s (not within " + + "±7d of now / outside 2020–2035); history samples stay unanchored (#91)") } else { log("Oura: 0x42 time-sync REJECTED - implausible ring-time ${e.value.ringTimestamp} ticks " + "(misframed record; history samples stay unanchored) (#91)") @@ -1207,9 +1212,9 @@ class OuraLiveSource( // #91: the 0x85 beacon is the SECONDARY anchor (fills the gap only until a 0x42 arrives). A // beacon ignored because a primary anchor already exists is NORMAL and not logged; only an // IMPLAUSIBLE-epoch beacon is a real failure (it can never anchor), so log just that. - if (!d.isPlausibleAnchorEpoch(e.value.unixSeconds)) { - log("Oura: 0x85 RTC beacon REJECTED - implausible epoch ${e.value.unixSeconds}s (outside " + - "the 2020–2035 anchor window) (#91)") + if (!d.acceptsAnchorEpoch(e.value.unixSeconds)) { + log("Oura: 0x85 RTC beacon REJECTED - implausible epoch ${e.value.unixSeconds}s (not " + + "within ±7d of now / outside 2020–2035) (#91)") } else if (!d.isPlausibleAnchorRingTime(e.value.ringTimestamp)) { log("Oura: 0x85 RTC beacon REJECTED - implausible ring-time ${e.value.ringTimestamp} ticks " + "(misframed record) (#91)") From da5ef5ab867685518ce5e8b68ca184ebafb4d122 Mon Sep 17 00:00:00 2001 From: Pipiche <8291674+pipiche38@users.noreply.github.com> Date: Fri, 10 Jul 2026 10:37:50 +0200 Subject: [PATCH 21/21] =?UTF-8?q?=20=20=20Oura:=20gate=20anchor=20epoch=20?= =?UTF-8?q?to=20=C2=B17d=20of=20now,=20not=20the=20loose=202020=E2=80=9320?= =?UTF-8?q?35=20window?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A misframed 0x42/0x85 beacon carrying a wrong-year epoch (e.g. 2021) passed the static 15-year plausibility window and seated a bad ring→UTC anchor, mis-stamping whole sample batches into 2021 and starving recent history. Add acceptsAnchorEpoch(): still requires the 2020–2035 floor, but when a near-now reference is supplied the epoch must fall within maxAnchorDrift (7 days) of it. OuraLiveSource seeds the reference at driver-creation time. Falls back to the static window when no reference is set (test/replay). Verified on-device: 2021-dated beacons now rejected ("not within ±7d of now"); last night's samples stamp correctly. Kotlin twin kept in parity. --- .../Sources/OuraProtocol/OuraDriver.swift | 35 ++++++++++++++++--- .../OuraProtocolTests/OuraDriverTests.swift | 26 ++++++++++++++ .../src/main/java/com/noop/oura/OuraDriver.kt | 32 +++++++++++++++++ 3 files changed, 89 insertions(+), 4 deletions(-) diff --git a/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift b/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift index b4825cd12..7ea88404c 100644 --- a/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift +++ b/Packages/OuraProtocol/Sources/OuraProtocol/OuraDriver.swift @@ -75,6 +75,14 @@ public final class OuraDriver { /// guessing. A stale anchor from a PREVIOUS session is never reused - the ring may have rebooted. private var anchorUtcMs: Int64? private var anchorRingTime: UInt32? + /// Host wall-clock reference (unix SECONDS) for the near-now anchor gate. The ring is `sync_time`-synced + /// to the host on connect, so a REAL 0x42/0x85 beacon reads ≈ now; a MISFRAMED one whose 4/8 payload + /// bytes happen to decode to a plausible-but-WRONG YEAR (e.g. 2021) must NOT be allowed to anchor — it + /// silently mis-stamps a whole batch of samples to that year (observed: 81 skin-temp rows dated 2021). + /// When set, an anchor epoch must fall within ±`maxAnchorDriftSeconds` of this. nil (unit tests / no + /// reference injected) ⇒ fall back to the static 2020–2035 window so pure tests keep clock-independent + /// bounds. Set by the transport (OuraLiveSource) on each connect. Must match the Kotlin twin. + public var anchorReferenceEpochSeconds: Int64? /// The freshly-provisioned key the transport generated during an adopt flow (s3.2). Once set by /// beginKeyInstall it becomes the effective key for the post-install re-auth. nil otherwise. private var installedKey: [UInt8]? @@ -269,6 +277,24 @@ public final class OuraDriver { return seconds * 1000 // safe: bounded input, cannot overflow } + /// Maximum drift between an anchor epoch and the host wall-clock reference (`anchorReferenceEpochSeconds`) + /// for the epoch to be trusted. The ring is `sync_time`-synced to the host on connect, so a genuine + /// beacon reads within a few seconds of now; ±7 days generously covers a beacon banked earlier in the + /// ring's recent history while still rejecting any wrong-YEAR misframe (a 2021 value is >4 y off). Must + /// match the Kotlin twin. + private static let maxAnchorDriftSeconds: Int64 = 7 * 86_400 // ±7 days + + /// Combined anchor-epoch gate. Requires BOTH the static 2020–2035 sanity window (also the overflow guard + /// for the seconds→ms `* 1000`) AND — when a host reference is injected — a ±`maxAnchorDriftSeconds` + /// near-now window, so a misframed beacon whose bytes land in a wrong year can never anchor and mis-stamp + /// a batch. With no reference (unit tests) it degrades to the static window, so existing pure tests keep + /// their fixed, clock-independent behaviour. Must match the Kotlin twin. + public func acceptsAnchorEpoch(_ seconds: Int64) -> Bool { + guard Self.isPlausibleAnchorEpoch(seconds) else { return false } + guard let reference = anchorReferenceEpochSeconds else { return true } + return abs(seconds - reference) <= Self.maxAnchorDriftSeconds + } + /// Ceiling for a plausible anchor RING-TIME (100 ms ticks, boot-relative): 500M ticks ≈ 579 days of /// uptime. The epoch gate alone is not enough — a misframed 0x42/0x85 can carry a plausible epoch /// (2020-2035) yet a GARBAGE ring-time (the 4 rt bytes read off a wrong offset land in the billions). @@ -372,8 +398,8 @@ public final class OuraDriver { // Both halves must be plausible: a plausible epoch paired with a garbage ring-time (misframed // record) would pin the anchor ~2e9 ticks off and starve all real history (see // maxPlausibleAnchorRingTime). Reject unless BOTH the epoch and the ring-time check out. - if let ms = Self.plausibleAnchorMs(fromEpochSeconds: ts.epochMs), - Self.isPlausibleAnchorRingTime(ts.ringTimestamp) { + if acceptsAnchorEpoch(ts.epochMs), Self.isPlausibleAnchorRingTime(ts.ringTimestamp), + let ms = Self.plausibleAnchorMs(fromEpochSeconds: ts.epochMs) { anchorUtcMs = ms anchorRingTime = ts.ringTimestamp } @@ -382,8 +408,9 @@ public final class OuraDriver { // Secondary UTC anchor (s5.5, 1s granularity): only fills in while no 0x42 anchor exists yet // this session, so a coarser beacon never overrides the primary time-sync anchor. guard let r = OuraDecoders.decodeRtcBeacon(record) else { return [] } - if anchorUtcMs == nil, let ms = Self.plausibleAnchorMs(fromEpochSeconds: Int64(r.unixSeconds)), - Self.isPlausibleAnchorRingTime(r.ringTimestamp) { + if anchorUtcMs == nil, acceptsAnchorEpoch(Int64(r.unixSeconds)), + Self.isPlausibleAnchorRingTime(r.ringTimestamp), + let ms = Self.plausibleAnchorMs(fromEpochSeconds: Int64(r.unixSeconds)) { anchorUtcMs = ms anchorRingTime = r.ringTimestamp } diff --git a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift index 613895ce8..3516fb6de 100644 --- a/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift +++ b/Packages/OuraProtocol/Tests/OuraProtocolTests/OuraDriverTests.swift @@ -219,6 +219,32 @@ final class OuraDriverTests: XCTestCase { XCTAssertEqual(d.unixSeconds(forRingTimestamp: anchorRt + 100), Int(anchorEpochSeconds) + 10) } + func testNearNowAnchorGateRejectsWrongYearButKeepsStaticWindowWhenNoReference() { + let now: Int64 = 1_752_000_000 // ~2025-07, the injected host clock + let anchorRt: UInt32 = 10_000 + func seat(_ epoch: Int64, reference: Int64?) -> Bool { + let d = OuraDriver(ringGen: .gen3, authKey: key) + d.anchorReferenceEpochSeconds = reference + _ = d.ingest(record: OuraRecord(type: OuraEventTag.timeSync.rawValue, + ringTimestamp: anchorRt, payload: le8(epoch) + [0x00])) + return d.hasAnchor + } + // With the host reference set: a near-now beacon anchors; a wrong-YEAR (2021) beacon that still sits + // inside the loose 2020-2035 window is REJECTED (it would mis-stamp a whole batch to 2021). + XCTAssertTrue(seat(now, reference: now)) + XCTAssertTrue(seat(now - 6 * 86_400, reference: now)) // 6 days stale (banked beacon) still OK + XCTAssertFalse(seat(now - 8 * 86_400, reference: now)) // 8 days off → outside ±7d + XCTAssertFalse(seat(1_610_000_000, reference: now)) // ~2021, >4y off → rejected + // With NO reference (unit-test default): degrade to the static 2020-2035 window, so the 2021 value + // anchors again — proves existing clock-independent tests are unaffected. + XCTAssertTrue(seat(1_610_000_000, reference: nil)) + // The pure predicate mirrors the seat behaviour. + let d = OuraDriver(ringGen: .gen3, authKey: key) + d.anchorReferenceEpochSeconds = now + XCTAssertTrue(d.acceptsAnchorEpoch(now)) + XCTAssertFalse(d.acceptsAnchorEpoch(1_610_000_000)) + } + func testPhantomRingTimestampFarFromAnchorIsRejected() { // A misframed record's garbage ring timestamp converts to a date days-to-years from the anchor but // still INSIDE the loose 2020-2035 absolute window. The anchor-relative guard must reject it so it diff --git a/android/app/src/main/java/com/noop/oura/OuraDriver.kt b/android/app/src/main/java/com/noop/oura/OuraDriver.kt index fee6eb2ec..6d90e5ffe 100644 --- a/android/app/src/main/java/com/noop/oura/OuraDriver.kt +++ b/android/app/src/main/java/com/noop/oura/OuraDriver.kt @@ -111,6 +111,17 @@ class OuraDriver( private var anchorUtcMs: Long? = null private var anchorRingTime: Long? = null + /** + * Host wall-clock reference (unix SECONDS) for the near-now anchor gate. The ring is sync_time-synced to + * the host on connect, so a REAL 0x42/0x85 beacon reads ≈ now; a MISFRAMED one whose bytes decode to a + * plausible-but-WRONG YEAR (e.g. 2021) must NOT anchor — it silently mis-stamps a whole batch of samples + * to that year. When set, an anchor epoch must fall within ±[MAX_ANCHOR_DRIFT_SECONDS] of this. null + * (unit tests / no reference injected) ⇒ fall back to the static 2020-2035 window so pure tests keep + * clock-independent bounds. Set by the transport (OuraLiveSource) on each connect. Twin of Swift's + * anchorReferenceEpochSeconds. + */ + var anchorReferenceEpochSeconds: Long? = null + /** * The freshly-provisioned key the transport generated during an adopt flow (s3.2). Once set by * beginKeyInstall it becomes the effective key for the post-install re-auth. null otherwise. @@ -334,6 +345,7 @@ class OuraDriver( private fun setAnchorIfPlausible(epochSeconds: Long, ringTimestamp: Long, preferPrimary: Boolean) { // A secondary (beacon) anchor never displaces an already-set primary (time-sync) anchor. if (!preferPrimary && anchorUtcMs != null) return + if (!acceptsAnchorEpoch(epochSeconds)) return val ms = plausibleAnchorMs(epochSeconds) ?: return // Both halves must be plausible: a plausible epoch paired with a garbage ring-time (misframed // record) would pin the anchor ~2e9 ticks off and starve all real history (#91). Reject unless the @@ -343,6 +355,18 @@ class OuraDriver( anchorRingTime = ringTimestamp } + /** + * Combined anchor-epoch gate. Requires BOTH the static 2020-2035 sanity window (also the overflow guard) + * AND — when a host reference is injected — a ±[MAX_ANCHOR_DRIFT_SECONDS] near-now window, so a misframed + * beacon whose bytes land in a wrong year can never anchor and mis-stamp a batch. With no reference (unit + * tests) it degrades to the static window. Twin of Swift's acceptsAnchorEpoch. + */ + fun acceptsAnchorEpoch(epochSeconds: Long): Boolean { + if (!isPlausibleAnchorEpoch(epochSeconds)) return false + val reference = anchorReferenceEpochSeconds ?: return true + return kotlin.math.abs(epochSeconds - reference) <= MAX_ANCHOR_DRIFT_SECONDS + } + /** * Bounds-check a decoded epoch (unix seconds) and convert to ms, or null if implausible. Kotlin twin * of Swift's `plausibleAnchorMs(fromEpochSeconds:)`. The 2020-2035 gate rejects a corrupt/misaligned @@ -597,6 +621,14 @@ class OuraDriver( private const val MIN_PLAUSIBLE_EPOCH_SECONDS = 1_577_836_800L private const val MAX_PLAUSIBLE_EPOCH_SECONDS = 2_051_222_400L + /** + * Max drift between an anchor epoch and the host wall-clock reference for the epoch to be trusted. + * The ring is sync_time-synced to the host on connect, so a genuine beacon reads within seconds of + * now; ±7 days covers a beacon banked earlier in the ring's recent history while rejecting any + * wrong-YEAR misframe. Byte-identical to Swift's maxAnchorDriftSeconds. + */ + private const val MAX_ANCHOR_DRIFT_SECONDS = 7L * 86_400L // ±7 days + /** * Ceiling for a plausible anchor RING-TIME (100 ms ticks, boot-relative): 500M ticks ≈ 579 days of * uptime. The epoch gate alone is not enough — a misframed 0x42/0x85 can carry a plausible epoch yet