feat: connector prerequisites system + Xiaohongshu connector

[graydawnc]

Summary

Adds a package-level prerequisites declaration to the Spool connector plugin system, with the Xiaohongshu (XHS) connector as the first real consumer.

A connector package can now declare external dependencies (CLI tools, browser extensions, site logins) in its package.json. The app detects them at runtime, renders a Setup checklist in the UI, offers one-click primary actions (supervised install, open Chrome Store, open login page), and re-checks automatically when the window regains focus or the user clicks Install.

Status: draft. XHS package isn't on npm yet; tested locally via the dev-mode workspace symlink path.

Design

Two concepts, two names:

• prerequisites (manifest, static): what the package needs
• setup (runtime, dynamic): current per-step status

Three prerequisite kinds, schema open for future ones:

kind	detection	install action
cli	exec --version + optional minVersion (semver)	supervised install via user's login shell, 120s + SIGKILL escalation, cancel; copy-to-clipboard fallback
browser-extension	exec doctor + matchStdout regex	Chrome Web Store URL, or manual-install modal with Download + Copy URL buttons
site-session	exec doctor + connectivity regex	open login URL in default browser

Dependency ordering via requires: ["otherId"] — downstream marked pending if upstream isn't ok, skipping its detect call.

Detection in main process (PrerequisiteChecker) using the existing exec capability. In-memory cache, in-flight dedup prevents redundant spawns on rapid focus events. No DB migration — prerequisite state is purely runtime.

Auto-recheck triggers: window focus (always, even when previously all-ok — extensions / CLIs / login sessions can degrade silently outside the app), post-install success, manual Re-check. Diff-before-broadcast prevents spurious renders when nothing changed.

Default checkAuth: SDK helper checkAuthViaPrerequisites(caps) that connectors can call when their auth is fully covered by prerequisites. XHS uses this — zero custom checkAuth logic in the connector code.

UI: one Setup card per package. SourcesPanel hides it when everything is ok (overview stays quiet); the connector detail view in SettingsPanel keeps it visible in collapsed form so users can inspect prereq state any time. The card pre-populates with status='pending' rows from the manifest the moment the package is opened, then per-step status flips when the actual check completes — no flash of empty state.

IPC error shapes are discriminated unions everywhere ({ok: true, ...} | {ok: false, reason: '...'}), no string sentinels.

Manifest is the single source of truth

The loader used to require the connector class fields (id, label, ephemeral, ...) to exactly match the package manifest, throwing if they drifted. That made multi-connector packages fragile: tweak class.ephemeral without updating package.json and the entire sub-connector silently fails to load while its siblings still appear, looking like a UI bug rather than a metadata bug. The loader now uses defineProperty to overwrite class fields with manifest values after instantiation — manifest wins, drift only logs a warning.

Exec capability inherits user shell env

GUI-launched apps on macOS don't inherit shell env (no http_proxy, no nvm-managed PATH, etc.), which surfaced as the GitHub connector hanging on gh api calls (TCP read reset) for users behind a proxy. The exec capability now wraps spawn in the user's shell so rc files get sourced:

• zsh: -ilc (sources .zprofile + .zshrc)
• bash: -lc (sources .bash_profile, which standardly chains to .bashrc; -i avoided because bash emits TTY warnings in CI)

Subprocesses spawn detached so timeouts can SIGKILL the whole process group rather than orphaning the inner command.

XHS Connector

Currently ships one sub-connector: xiaohongshu-notes (creator notes, persistent, single-shot fetch). The package keeps its multi-connector array structure so feed and notifications can return cleanly when their upstream issues resolve:

• xiaohongshu-feed — opencli reads the page Pinia store snapshot without scrolling, so item count fluctuates (~20) regardless of --limit
• xiaohongshu-notifications — opencli returns only [{rank: N}] placeholders without real content fields

Notes uses opencli's creator-notes subcommand (single-shot, --limit 100, no pagination since opencli has no cursor flag). Empty results (which opencli reports as exit 1 "No notes found") are treated as a successful empty page rather than a sync error.

Safety against infinite crawl

Two layers:

1. Connector-level: nextCursor: null always for opencli-backed connectors (no pagination supported upstream)
2. Sync-engine maxPages option: new optional field on SyncOptions (default 100). Both ephemeral and persistent sync loops check the cap and exit with stopReason: 'max_pages'. Defense in depth for any connector

Dev-mode conveniences

Three non-production-affecting paths added for local testing:

• Local registry: in !app.isPackaged, connector:fetch-registry reads packages/landing/public/registry.json from the workspace. SPOOL_REGISTRY_URL env var overrides explicitly. Production still fetches from raw.githubusercontent.com
• Workspace install: in dev, the Install button symlinks the package from packages/connectors/<dir> if present; otherwise falls through to npm. Lets us test unpublished connectors via the real Install UX
• XHS is NOT bundled: registered as a regular installable package. Tested via the dev-mode symlink path until published

Tests

• packages/core/src/connectors/prerequisites.test.ts — PrerequisiteChecker detect paths and validatePrerequisites manifest validator
• packages/core/src/connectors/registry.test.ts — multi-connector package merge in ConnectorRegistry
• packages/core/src/connectors/loader.test.ts — manifest-as-truth (drift between class field and manifest is logged but doesn't fail loading)
• packages/core/src/connectors/capabilities/exec-impl.test.ts — login-shell semantics, arg quoting, missing-binary returns exit 127, timeout kills process group
• packages/connectors/xiaohongshu/src/index.test.ts — single-shot fetch, never passes unsupported flags, treats opencli "no X found" exit as empty result, surfaces real errors

Total: 140 core tests + 5 XHS tests pass. CI green.

Files of note

• packages/connector-sdk/src/connector.ts — SDK types (Prerequisite, Detect, Install, ManualInstall, SetupStep, AuthStatus.setup), checkAuthViaPrerequisites helper
• packages/core/src/connectors/prerequisites.ts — PrerequisiteChecker (semver, in-flight dedup, dependency-aware ordering)
• packages/core/src/connectors/loader.ts — manifest as single source of truth via applyManifestMetadata
• packages/core/src/connectors/capabilities/exec-impl.ts — login-shell wrap with shell-specific flag (zsh -ilc, bash -lc), detached process group for clean kill
• packages/core/src/connectors/registry.ts — registerPackage merges on same id (for multi-connector packages)
• packages/core/src/connectors/sync-engine.ts — maxPages safety cap in both loops
• packages/app/src/main/index.ts — 6 IPC handlers, supervised install (SIGKILL escalation, login-shell, 120s timeout, cancel), focus-driven recheck (always, with diff-before-broadcast), pending-steps seeding so the prereq card renders before the first check completes
• packages/app/src/renderer/components/PackageSetupCard.tsx — status icons, per-kind primary actions, inline install progress, hide-when-all-ok by default, alwaysShow mode for connector detail (collapsed pill that expands on click), ghost-style accent install button
• packages/app/src/renderer/components/ManualInstallModal.tsx — data-driven step list, Download / Copy URL / Check now actions
• packages/connectors/xiaohongshu/ — package with xiaohongshu-notes sub-connector + prereqs

Test plan

• Core: 140 tests passing across PrerequisiteChecker, validatePrerequisites, registry, loader (manifest truth), exec-impl (login shell), sync-engine
• XHS connector: 5 tests covering single-shot fetch, unsupported-flag avoidance, empty-state handling, error propagation
• App tsc --noEmit clean
• CI green (unit + e2e mac + e2e linux)
• Manual: fresh install → prereq card visible immediately with pending rows → check completes within ~1s and rows flip to ✓/✗ → Install OpenCLI triggers supervised shell → extension manual-install modal → login flow → card collapses to pill on detail view when all ok, expandable on click → focus event triggers recheck → removing extension flips card back to ✗
• Manual: GitHub connector sync works through user's .zshrc proxy via the new login-shell wrap (verified: no more read tcp reset)
• Regression: HN, Twitter Bookmarks, Typeless render and sync unchanged (no prerequisites declared)

Known caveats

• XHS package not yet published to npm. Registry URL points to a package that cannot be npm installed outside this workspace. Before merging, either publish the package or gate the registry entry behind a flag
• XHS feed and notifications sub-connectors are intentionally absent — see the refactor(connector-xiaohongshu): scope to notes-only commit for rationale and re-add path

Follow-ups

• Prod-style smoke test before releases (npm pack → install into fresh app) — dev/prod install paths diverge enough that pure dev testing has missed regressions before
• Re-add xiaohongshu-feed once opencli supports scroll-before-snapshot, and xiaohongshu-notifications once its JSON output includes real content fields

🤖 Generated with Claude Code