feat(search): commercial-parity features, privacy hardening, and cross-layer fixes

[ErikChevalier]

What & why

This change closes much of the everyday-experience gap to major commercial engines while tightening the privacy guarantees, and fixes a broad set of verified bugs found in a full four-layer audit (engine, server/web UI, app UI, data/crypto).

New search features — all computed on-device, no new third parties:

• Instant answers: calculator (2+2, sqrt(9)*3), unit conversions (10 km to miles, 72 f to c), number-base conversions (0xff in decimal), and percentages (15% of 80), shown as an answer card on the served page and in-app. Date/phone-shaped input (2020-2021, 555-1234) is guarded so the card never mis-fires.
• !bangs (!w, !gh, !yt, !osm, …): jump straight to a site's own search, resolved from a curated on-device table. The terms never enter the metasearch fan-out; unknown tags (!important css) are never hijacked.
• Search-as-you-type on the served pages: keyboard-navigable ARIA-listbox dropdown over the existing /suggest endpoint (debounced, aborts stale fetches, same-origin only). / focuses the search box.
• Results meta line shows result count + elapsed time; a favicon is served; the topbar / sort selector / did-you-mean link now carry the active vertical and sort instead of silently resetting them; the engine-status diagnostic now renders on empty result pages (exactly when the owner needs it).

Privacy hardening:

• The Wikipedia summary thumbnail is re-served through a new loopback /img proxy (scoped to upload.wikimedia.org, image-only, size-capped) — previously the browser fetched it from Wikimedia directly, leaking the user's IP plus the searched entity via the image filename. CSP tightens from img-src https: to img-src 'self' data: (plus connect-src 'self' for the suggestions dropdown).
• Engine adapters pin one User-Agent per logical search (a 429 retry no longer switches identities seconds apart from the same IP), the UA pool is refreshed, and per-host politeness spacing is now process-wide and thread-safe (it was rebuilt per search and racy, i.e. never effective).
• WikiSummaryProvider body reads are size-capped like every other fetch; the OpenSearch descriptor uses the request Host for network-mode visitors (without embedding the access token) so browser integration works off-device.

Correctness / data-safety fixes (highlights):

• Bootstrap metadata (the only copy of the wrapped DEK) is written atomically (temp + fsync + rename), and a present-but-corrupt file now fails closed instead of silently re-keying — which previously destroyed all encrypted data (prefs, API keys, history) unrecoverably.
• Whole-call HTTP timeout (callTimeout) so a byte-trickling upstream can no longer hold a search open for minutes.
• Vertical scoping no longer poisons engines without site:/OR syntax (Wikipedia/Marginalia/Mwmbl get the clean query; constraints stay locally enforced); upstream spelling corrections no longer leak or double the vertical's site: clause.
• Ranking-quality fixes: http/https and mobile-host variants dedup into one result, blank titles backfill from other engines, domain rules resolve most-specific-first, day-first slash dates parse, after:2024-3-1 works, DuckDuckGo's direct-href A/B variant parses instead of yielding zero results, and the spell-corrector's auto-search threshold is raised so brand names ("spotify") are never silently rewritten ("spotty").
• App-layer fixes: fresh ViewModels per create() (Settings previously went permanently dead after one back-out, silently stopping history recording), cancelled searches no longer flash a raw error, onboarding survives rotation + handles system back + routes "Open privacy settings" to Settings, battery-exemption cards refresh on resume, widget deeplink no longer re-fires on rotation, notification prompt defers to onboarding, engine toggles are transactional (no lost writes), update-banner dismissal survives rotation, APK staging is off the main thread, all startActivity calls are guarded, result cards show the source host, and the search field auto-focuses when opened idle.

Known items deliberately deferred (documented for follow-up): true upstream pagination, streamed/early page render, vault-lock races around DataStore caching, and loopback app-to-app access control.

Relates to OpenSpec change: add-relevance-ranking (plus touches on add-result-paging, add-engine-status-visibility)

Checklist

• Built on its own feat//fix/ branch off main
• ./gradlew ktlintCheck lint test assembleDebug is green locally — no Android SDK in this environment; the pure-JVM engine/data suites (241 + 19 tests) were compiled and run green against Kotlin 2.4.0 locally, CI validates the rest
• Added/updated tests (unit and/or instrumentation)
• No telemetry, trackers, or device identifiers added
• No Google scraping introduced (the !g bang is an explicit user-chosen redirect, not scraping)
• Battery: no wake-lock held while idle
• OpenSpec change validates (openspec validate <name> --strict) and tasks are checked off — no new OpenSpec change authored for this cross-cutting fix/parity sweep

🤖 Generated with Claude Code

https://claude.ai/code/session_01Qsx1eDT79eSgUwfxUaYS34

---

Generated by Claude Code