Skip to content

Design session: persistent per-session log storage (flight-recorder) #27

@mibrahimdev

Description

@mibrahimdev

Type: design discussion (needs a proper session before any code). Captures the persistence idea raised 2026-06-14. The roadmap entry is in docs/ROADMAP.md.

Goal

Move Sharingan from a live in-memory tail to a flight recorder: persist captured events to a local on-device DB so logs survive process death/crash, are grouped per session, and a consumer can dump previous sessions' logs anytime.

Value

  • Crash survival — the logs right before a crash are exactly what an in-memory ring buffer loses.
  • Per-session history + dump-anytime — "send me the logs from your last run".
  • History beyond the current 300-event in-memory ceiling without holding it all in RAM.

Design constraints / open questions to settle in the session

  1. Noop parity is sacred. Persistence must live entirely behind the existing SharinganStore surface (record(), events: StateFlow, clear()). :sharingan-noop stays empty/in-memory with a byte-identical public API and zero DB dependency (it ships in release builds). No DB type may leak into the public API.
  2. DB engine — decide: SQLDelight (KMP-first, lean, mature on Native — my lean) vs Room KMP (2.7+, Android-familiar, heavier on iOS).
  3. Hot path. record() is currently a lock-free CAS on an in-memory list. Don't write to disk per event — use a write-behind buffer: keep the in-memory StateFlow as the live UI window and flush batches to the DB on a background dispatcher. DB augments the ring buffer, doesn't replace it.
  4. Schema. session table (id, started-at, app/build/device meta) + events table (FK session, timestamp, type, indexed cols + a kotlinx.serialization JSON blob for the event payload). Composes with the event ABI frozen in Freeze event-type ABI: make HttpEvent/MqttEvent/BleEvent constructors internal #15.
  5. Session model. One session per process launch? Explicit start/stop? "Dump previous logs" = query by session id.
  6. Retention policy. max sessions / max age / max rows-per-session. This is what configure(capacity) (Expose Sharingan.configure(capacity) for store capacity (HITL — public API decision) #18) becomes — a retention config, added additively later. (Expose Sharingan.configure(capacity) for store capacity (HITL — public API decision) #18 was closed in favour of this.)
  7. Security — must design in, not bolt on. A SQLite file is extractable; in-memory bodies are ephemeral. Persisting request/response bodies can write tokens/PII to disk. Need: bodies-off-by-default for persistence, header/body redaction, retention/auto-purge, clear-on-new-session. Debug-only mitigates but isn't sufficient alone. (Encryption/SQLCipher = probably overkill for debug-only — discuss.)
  8. New public API (mirror in noop): list sessions, dump/export a session (file/JSON). Keep minimal.
  9. iOS. native SQLite driver; XCFramework size impact (debug-only, acceptable).

Out of scope until decided

No implementation until this is brainstormed and broken into tracer-bullet issues. Sequencing: ship 1.0 on the in-memory architecture first (close #12), then take this on as a v2 epic.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestquestionFurther information is requested

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions