From 5a2ed66500807159f0c4dfc14035a77166221626 Mon Sep 17 00:00:00 2001 From: David Vandenbogaerde Date: Thu, 11 Jun 2026 16:51:16 +0200 Subject: [PATCH 01/21] added implementation plan for multi-image support (1 frame per image) --- per_frame_pipeline_plan.md | 579 +++++++++++++++++++++++++++++++++++++ 1 file changed, 579 insertions(+) create mode 100644 per_frame_pipeline_plan.md diff --git a/per_frame_pipeline_plan.md b/per_frame_pipeline_plan.md new file mode 100644 index 0000000..7cf4ec1 --- /dev/null +++ b/per_frame_pipeline_plan.md @@ -0,0 +1,579 @@ +# Per-Frame Alignment Pipeline — Implementation Plan + +This document plans the addition of a third `alignmentPipeline` mode — `"per-frame"` — +that lets the user upload N images, one per animation frame, instead of a single +frame-sheet photo. Each image is page-rectified independently and the rectified +results are stacked into a synthetic `baseRectifiedMat`, after which the existing +stabilization / ordering / appearance / export path runs unchanged. + +This is an engineering plan, not user docs. User-facing copy belongs in +`documentation.md`; durable invariants belong in `AGENTS.md`. + +## Locked Decisions + +These were chosen up front so individual sessions do not need to relitigate them. + +1. **Frame count = image count (Option 1A).** The number of uploaded images is the + number of animation frames. Internally treated as a flat `1 × N` strip. The + `Frame Rows` / `Frame Columns` Layout controls become display-only / disabled + in per-frame mode. +2. **Rescale to common cell size (Option 2B).** After per-image page rectification, + every cell is resized to a single common cell size (default: median rectified + width × median rectified height). All cells in the synthetic `baseRectifiedMat` + are the same size; stabilization and extraction work on uniform cells. +3. **Persist per-image overrides (Option 3A).** Page-corner overrides and + post-rotation are saved per image using indexed settings keys + (`page_corner_override_tl_0`, `..._tl_1`, …, `per_frame_post_rotation_deg_0`, + …). Settings files round-trip cleanly. +4. **Single image in per-frame mode is allowed (Option 4A).** Loading a single + image in per-frame mode produces a 1-frame animation. The user can add more + images via the strip afterwards. + +## Architectural Summary + +- Add `"per-frame"` as a third value for `config.alignmentPipeline`. +- New per-image source structure under `state.source.images[]`; the existing + `state.source.image` / `manualPageContour` / `dragUrl` / etc. become *views into* + the active image (`state.source.images[state.source.activeImageIndex]`) so most + legacy callers keep working. +- Factor out the page-detection + page-rectification block from + `pipeline.js → runPipeline` into a reusable `rectifySinglePage(...)`. Markers and + markerless modes continue to call it as before; per-frame mode calls it once per + uploaded image. +- New `pipeline.js → runPerFramePipeline(images, config, …)`: + 1. Run `rectifySinglePage` per image (respecting per-image page-corner and + post-rotation overrides). + 2. Pick a common cell size (median of rectified sizes; clamped so the composite + fits within the existing large-image limits). + 3. Resize each rectified Mat to the common cell size. + 4. Concatenate horizontally into a single composite `rectifiedMat` + (`cellWidth * N × cellHeight × 1` row × N column grid). + 5. Synthesize an `alignmentInfo` whose marker lookup contains regular corner + intersections at the known cell boundaries. + 6. Return the same result shape that `runPipeline` returns. +- `runPipeline` becomes a dispatcher: per-frame → `runPerFramePipeline`; otherwise + the existing markers/markerless path. +- Downstream (frame extraction, stabilization, ordering, appearance, export) does + not branch on per-frame mode. The synthetic sheet looks like a 1×N markerless + sheet to those subsystems. + +## Phases + +Each phase below is self-contained: it lists scope, files, acceptance criteria, +and explicit out-of-scope items so individual implementation sessions do not +overreach. + +--- + +### Phase 0 — Spike (½ day, throwaway) + +**Goal:** prove that a tiled composite Mat plus existing stabilization can produce +a working animation end-to-end, before doing any real refactoring. + +**Scope:** +- Hardcode two demo images (or load two arbitrary images via the existing file + input twice in sequence). +- Add a dev-only code path that builds a `2 × cellW × cellH` composite Mat from + the two latest rectified Mats and stuffs it into `state.geometry.baseRectifiedMat` + alongside a fake `alignmentInfo` with two cell corners. +- Confirm: preview animates, stabilization measurement runs, GIF export produces + the expected 2-frame GIF. + +**Files touched (throwaway code, do not merge as-is):** +- `js/app.js` — small scratch block, clearly marked `// SPIKE: per-frame, remove`. + +**Acceptance:** +- Two-frame GIF exports correctly. +- Stabilization runs without errors and offsets are non-zero when the two source + images are deliberately misregistered. +- Nothing about the marker/markerless code paths regresses for a normal source + image. + +**Out of scope:** +- Real UI, real `state.source.images[]`, settings, i18n, mobile, memory trimming. +- Any code that should survive into Phase 2+. + +**Rollback:** delete the spike block; no other files were touched. + +--- + +### Phase 1 — Refactor: extract `rectifySinglePage` + +**Goal:** isolate the page-detection + page-rectification stage of `runPipeline` +into a reusable helper without behavioral change to markers or markerless mode. + +**Scope:** +- In `js/pipeline.js`, factor lines roughly 250–385 of `runPipeline` (page + detection through `applyPostRectificationRotation`) into a new function: + ``` + rectifySinglePage(sourceCanvas, perPageConfig, requestId, throwIfAborted) + -> { rectifiedWarp, pageQuad, pageQuadSource, pageWarpPreviewCanvas, + pageWarpPreviewWidth, pageWarpPreviewHeight, + useNearIdentityRectification } + ``` + Caller is responsible for Mat lifetime of the returned `rectifiedWarp`. +- `perPageConfig` is a strict subset of the full config: `paperAspect`, + `manualPageQuadPoints`, `fallbackPageQuadPoints`, `thresholdMethod`, + `thresholdOffset`, `postRotationDeg`, `lightOnDarkDesign`, `alignmentPipeline` + (still needed because near-identity fast-path branches on markerless). +- `runPipeline` is unchanged behaviorally; it just calls `rectifySinglePage` and + then proceeds with the existing grid-rectification + alignment + extraction + path. + +**Files touched:** +- `js/pipeline.js` + +**Acceptance:** +- Demo round-trip on every demo image listed in `index.html#loadDemoSelect` + produces visually identical previews to `main`. +- Settings save/load round-trip is byte-identical. +- No new console errors. Memory profile during a large-image reprocess does not + regress (peak Mat count unchanged). +- `runPipeline` is meaningfully shorter; `rectifySinglePage` has a clear contract + documented in JSDoc. + +**Out of scope:** +- Any per-frame mode code. +- Any change to `runPipeline`'s output shape. +- Any change to the marker/markerless detector functions. + +--- + +### Phase 2 — State: `state.source.images[]` and active-image accessor + +**Goal:** introduce per-image source state without yet adding any UI or pipeline +support. Existing markers/markerless flows continue to use the active image +transparently. + +**Scope:** +- In `js/dom-state.js`, extend `state.source`: + ``` + images: [], // [{ image, filename, mimeType, ownedObjectUrl, dragUrl, + // canvas, manualPageContour, postRotationDeg, + // rectifiedMatCache: null, rectifiedDirty: true }] + activeImageIndex: 0, + ``` +- Add accessor helpers (likely in a new tiny module `js/source-images.js`, or + inline in `dom-state.js`): + - `getActiveSourceImage(state)` returns the active entry, or `null`. + - `setActiveSourceImage(state, index)`. + - `releaseAllSourceImages(state)` revokes object URLs and clears Mats. +- Mutate `load-controller.js → loadImageSource` to push the loaded image into + `state.source.images` as a single-entry array and set `activeImageIndex = 0`, + while *also* keeping the existing `state.source.image / filename / dragUrl / + manualPageContour` fields populated. The legacy fields become projections of + the active entry for the duration of the migration. +- No mode logic yet — `state.source.images[]` always has exactly 0 or 1 entries + during this phase. + +**Files touched:** +- `js/dom-state.js` +- `js/load-controller.js` +- new `js/source-images.js` (optional) + +**Acceptance:** +- Markers and markerless modes still load the same demos and round-trip + identically. +- `releaseOwnedSourceUrl` is replaced by / wraps `releaseAllSourceImages`; no + blob URLs leak across reloads. +- Loading a new image clears prior `images[]` entries. + +**Out of scope:** +- Multi-file `handleFile`. (That arrives in Phase 4.) +- Per-image overrides. (Phase 5.) +- Any UI for the image strip. + +--- + +### Phase 3 — Pipeline: `runPerFramePipeline` + dispatcher + +**Goal:** wire the new per-frame pipeline behind the existing `runPipeline` +contract. No UI yet — exercised through manual config hacking or unit-level +testing. + +**Scope:** +- In `js/pipeline.js`: + - Add `runPerFramePipeline(images, config, requestId, throwIfAborted)`: + 1. For each `image` in `images`, build a per-image `sourceCanvas` (already + in `state.source.images[i].canvas`). + 2. Call `rectifySinglePage(sourceCanvas, perImagePageConfig, …)`. The + `perImagePageConfig` is built per image, using that image's + `manualPageContour` and `postRotationDeg`. + 3. Collect rectified `styledMat`s. + 4. Decide common cell size: + - `cellW = clamp(median(rectifiedWidths), MIN_CELL_PX, MAX_CELL_PX)` + - `cellH = clamp(median(rectifiedHeights), MIN_CELL_PX, MAX_CELL_PX)` + - Limits chosen to keep `cellW × cellH × N` inside existing + `RECTIFIED_PREVIEW_LONG_EDGE_PX` / large-image-memory budgets. + 5. Resize each rectified `styledMat` to `(cellW, cellH)` using the + interpolation flag from `config.exportOptions.resampling`. + 6. Allocate composite `rectifiedMat` of size `(cellW * N, cellH)` and copy + each resized cell into its column. + 7. Synthesize `alignmentInfo` whose corner intersection points are at + `(i * cellW, 0)`, `((i+1) * cellW, 0)`, `(i * cellW, cellH)`, + `((i+1) * cellW, cellH)` for `i = 0 … N`. The structure should match what + `buildUnrefinedCrossRegionInfo` produces for a 1×N markerless sheet. + 8. Build a `rectifiedCanvas` preview from the composite Mat using the + existing `matToPreviewCanvas`. + 9. Extract frame canvases via the existing `sliceRectifiedToCanvases` so the + returned `frames` array matches what marker/markerless modes produce. + 10. Return the same result shape as `runPipeline`: `{ frames, rectifiedCanvas, + rectifiedMat, pagePreviewCanvas: null, pagePreviewGridQuad: null, + pagePreviewGridBounds: null, alignmentInfo, statusText, pageQuadPoints: + null, pageQuadSource: "per-frame", rectifiedDownloadUsesRawSource: false }`. + (`pagePreview*` is null because there is no single "page" in per-frame mode; + Phase 7 will hide the corresponding UI.) + - Modify `runPipeline` so its first lines dispatch: + ``` + if (config.alignmentPipeline === "per-frame") { + return runPerFramePipeline(state.source.images, config, requestId, throwIfAborted); + } + ``` + `runPipeline` should not import `state` directly; pass `images` from the + caller. So update the caller in `app.js → processCurrentImage` to pass + `state.source.images` alongside `state.source.canvas`. +- `app.js → readConfig`: when the per-frame radio is checked, set + `alignmentPipeline: "per-frame"`. Also force `useCrossAlignment: false` for + this mode (the synthetic grid does not need refinement). + +**Files touched:** +- `js/pipeline.js` +- `js/app.js` (readConfig + the `runPipeline` invocation site only) + +**Acceptance:** +- With per-frame mode forced via dev console (`dom.alignmentPipelineMarkerless.checked = false; dom.alignmentPipelineMarkers.checked = false; /* + manual flag */`) + and two images loaded via dev hack, a 2-frame animation appears in + `gifPreviewCanvas` and exports correctly. +- Stabilization measurement runs on the synthetic sheet without errors. +- Markers/markerless flows continue to work unchanged. +- Memory: `MAX_CELL_PX` and median selection keep peak Mat size bounded. + +**Out of scope:** +- The "per-frame" radio button itself (Phase 6 adds it). +- Multi-file upload (Phase 4). +- Per-image overrides (Phase 5). +- UI strip (Phase 7). +- Settings persistence (Phase 8). + +--- + +### Phase 4 — Multi-file upload (`handleFile`, drop zone) + +**Goal:** support dragging or selecting multiple image files at once. Always +populate `state.source.images[]`; never silently drop additional images. + +**Scope:** +- `js/load-controller.js → handleFile`: when multiple image files are present in + the drag payload (or file input), build per-image entries for each. Sibling + `_settings.txt` is still matched against the first image's filename and applied + once after all images are loaded. +- `index.html`: `` and drop-zone copy update. +- `js/i18n.js`: drop-zone copy strings for per-frame mode (and a generic copy + that mentions multi-file support; localize across all locale tables). +- After multi-file load, if the currently selected pipeline is **not** + `"per-frame"`, the app should auto-switch to `"per-frame"` (this is the most + forgiving UX). Otherwise the user just dropped 12 files into a markerless app + and would silently lose 11 of them. +- Single-image drops continue to behave exactly as before. + +**Files touched:** +- `index.html` +- `js/load-controller.js` +- `js/i18n.js` + +**Acceptance:** +- Drag-and-drop with 1 image: identical to current behavior. +- Drag-and-drop with N images: app switches to per-frame mode, loads all N, + active index is 0, animation appears. +- File picker with multi-select: same behavior as drag. +- Mixed drag (N images + 1 `_settings.txt`): settings file is applied against + the first image's name; per-frame mode is selected if N > 1. + +**Out of scope:** +- Strip UI (Phase 7). +- Reorder/delete buttons (Phase 7). +- Settings-driven per-image overrides on reload (Phase 8 plus Phase 5). + +--- + +### Phase 5 — Per-image page-corner and post-rotation overrides + +**Goal:** make the existing Page Corners editor and Post-Rotation control operate +on the *active* image in per-frame mode. + +**Scope:** +- Route `state.source.manualPageContour` reads/writes through the active image + in per-frame mode. Concretely, where `app.js` currently writes + `state.source.manualPageContour = …`, replace with a helper + `setActiveManualPageContour(state, contour)` that: + - in per-frame mode, writes to `state.source.images[activeIndex].manualPageContour` + and also mirrors to the legacy field (so other read sites keep working); + - in markers/markerless modes, writes the legacy field directly (no-op for + `images[]`). +- Same treatment for `postRotationDeg` (per-image storage; the slider reads/writes + the active image's value in per-frame mode). +- Switching active image redraws `rawCanvas`, Page Corners overlay, and + Post-Rotation slider position to match the newly active image. The rectified + preview / animation are *not* rebuilt on active-image switch — that is a UI + navigation, not a config change. +- Page Detection Threshold remains a global control (acts on all per-frame + images when reprocessing). Per-image threshold is out of scope for v1. + +**Files touched:** +- `js/app.js` (Page Corners drag handlers, Post-Rotation handlers, redraws) +- `js/dom-state.js` (per-image fields added in Phase 2) +- `js/ui-controls.js` (active-image switching invalidations) + +**Acceptance:** +- In per-frame mode with 3 images, the user can edit page corners on image #2 + without affecting images #1 or #3. +- Switching active image redraws the Page Corners overlay with that image's + saved corners. +- Reprocessing uses every per-image override correctly. +- Markers/markerless modes are completely unaffected. + +**Out of scope:** +- UI strip (Phase 7) — at this point active-image switching can be triggered by + dev console (`state.source.activeImageIndex = 1; renderRawPreview();`). +- Persisting overrides to disk (Phase 8). + +--- + +### Phase 6 — `alignmentPipeline` radio: add "per-frame" + +**Goal:** expose per-frame mode via the existing Alignment Pipeline radio group +and wire mode-gated visibility. + +**Scope:** +- `index.html`: add a third radio + `#alignmentPipelinePerFrame` to `#alignmentPipelineField`. +- `js/dom-state.js`: add the DOM ref. +- `js/settings-defaults.js`: leave default as `"markers"`; add a sync line for + the new radio. +- `js/ui-controls.js`: add to the change listeners; on switch into per-frame + mode with `state.source.images.length === 0`, do not process; just wait for + upload. +- `js/app.js → readConfig`: emit `"per-frame"` when the new radio is checked + (already stubbed in Phase 3; this phase makes the radio real). +- Mode-gated visibility (audit and update): + - **Disabled in per-frame mode:** all marker controls, all markerless gutter + controls, Grid Edge Threshold, Grid Edge Run Length, marker editor, + Rectified Grid `Pre`/`Post` toggle, Page Detection Threshold's "live + preview" semantics (still functional but global). + - **Enabled in per-frame mode:** stabilization (Neighbor / Median both work), + Vertical Drift Compensation, Frame Corners overrides, ordering, appearance, + export options, Layout's Frame Rows/Cols (display-only). +- `js/i18n.js`: new strings for the radio label, tooltip, and any mode-gated UI + labels that diverge. +- Audit every `alignmentPipeline === "markerless"` site (≈30 in `app.js`) and + classify: + - **"markerless-only behavior"**: stays as `=== "markerless"`. Examples: + markerless gutter chart, markerless phase debug, autocorrelation. + - **"non-marker behavior"**: change to `!== "markers"` so per-frame inherits. + Examples: stabilization availability, drift comp UI. + - **"per-frame-disabled"**: add `&& alignmentPipeline !== "per-frame"` guard. + Examples: Rectified Grid Pre/Post toggle visibility. + + Produce a checklist file (or inline TODO list) during this phase listing each + classification decision; it makes the code review tractable. + +**Files touched:** +- `index.html` +- `js/dom-state.js` +- `js/settings-defaults.js` +- `js/ui-controls.js` +- `js/app.js` +- `js/i18n.js` + +**Acceptance:** +- Switching the radio between all three modes works without errors. +- Per-frame mode hides marker/markerless-specific UI cleanly (no orphan labels, + no flashed empty panels). +- Markers and markerless modes still pass demo round-trips. +- Mobile single-viewer mode still behaves correctly across all three pipelines. + +**Out of scope:** +- The image strip itself (Phase 7). +- Settings persistence (Phase 8). + +--- + +### Phase 7 — Image strip UI + +**Goal:** give the user a visible way to switch active image, see upload count, +reorder, and delete. + +**Scope:** +- New section in `index.html`, visible only in per-frame mode. Likely placed + inside the Photo control group below the drop zone, or as a new collapsible + `#perFrameStripPanel`. +- Renders a horizontal scrollable strip of thumbnails (one per + `state.source.images[i]`). Each thumbnail shows: + - the image (small preview, square-cropped or letterboxed) + - the frame number `1`, `2`, … + - a delete (×) button on hover + - active state highlighting +- Drag-to-reorder within the strip (HTML5 drag-and-drop API, scoped to the + strip). +- Click selects active image. +- A `+` tile at the end accepts additional dropped or chosen images. +- New file `js/per-frame-strip.js` keeps strip rendering / event handling out of + `app.js` and `ui-controls.js`. +- Mobile: the strip should still be usable in single-viewer mode. The strip can + appear above the active raw photo viewer. +- Reorder triggers reprocessing (frame order changes). +- Delete triggers reprocessing. + +**Files touched:** +- `index.html` +- `js/dom-state.js` (strip DOM refs) +- `js/ui-controls.js` (wire-up) +- new `js/per-frame-strip.js` +- `style.css` (strip + thumbnail styling) +- `js/i18n.js` + +**Acceptance:** +- Upload 5 images, reorder them, the animation reflects the new order on next + preview tick. +- Delete an image; the animation rebuilds with N-1 frames. +- Active-image highlighting matches `state.source.activeImageIndex`. +- Mobile: strip is reachable and usable in the Page viewer tab. + +**Out of scope:** +- Cross-strip drag (e.g. dragging a thumbnail out of the app). +- Bulk operations (multi-select delete). + +--- + +### Phase 8 — Settings persistence for per-image state + +**Goal:** make `_settings.txt` round-trip in per-frame mode. + +**Scope:** +- `js/settings-io.js` — save side: + - Emit `alignment_pipeline = per-frame`. + - For each image `i = 0 … N-1`: + - `page_corner_override_tl_i`, `_tr_i`, `_br_i`, `_bl_i` (only if that image + has overrides; otherwise omit). + - `per_frame_post_rotation_deg_i` (only if non-zero). + - Emit `per_frame_image_count = N` so reload knows how many to expect. +- `js/settings-io.js` — load side: + - Recognize `per_frame_image_count` to size the per-image override array + before images themselves load. + - On image load (single image first, additional images added later), apply the + matching indexed overrides to the corresponding `state.source.images[i]`. + - Backward-compat: legacy files without `per_frame_*` keys load with empty + per-image overrides. +- Loading a saved per-frame project requires the user to re-upload all N images + (we cannot save image data in the settings file). On load with no images yet, + the saved per-image overrides are buffered into a pending structure and + applied as images arrive (matching by upload order — index 0 → first uploaded + image). Document this clearly in `documentation.md`. + +**Files touched:** +- `js/settings-io.js` +- `js/dom-state.js` (pending overrides buffer) +- `js/load-controller.js` (apply pending overrides as images arrive) +- `documentation.md` + +**Acceptance:** +- Edit per-image corners + post-rotation across 3 images; save settings; reload + the page; re-upload the same 3 images; per-image overrides reappear correctly. +- Legacy `_settings.txt` files (markers / markerless) still load without + regression. +- The saved file remains a TSV that humans can inspect. + +**Out of scope:** +- Saving image data inside the settings file. +- Auto-matching by filename across reloads (matching is strictly by upload + order in v1). + +--- + +### Phase 9 — Memory and polish + +**Goal:** make per-frame mode safe on large inputs, finish documentation, and +write the AGENTS.md invariants. + +**Scope:** +- `js/app.js → trimCachesBeforeReprocess`: release any per-image rectified Mat + cache entries that are not the active image. The composite `baseRectifiedMat` + is the only large Mat that needs to stay live between reprocesses. +- `js/pipeline.js → runPerFramePipeline`: bound the cell size such that + `cellW × N × cellH` cannot exceed the same memory ceiling that the existing + large-image path uses for `pageSizeHigh`. If the median would exceed the + ceiling, scale the cell size down uniformly. +- Mobile single-viewer audit: confirm the Page tab works with the new strip and + active-image switching. +- `documentation.md`: add a "Per-Frame Pipeline" section explaining when to use + it, how to upload multiple images, per-image page-corner editing, and the + reload story for saved settings. +- `AGENTS.md`: add invariants: + - Per-image page-corner overrides are post-load, pre-rectification. They feed + into `rectifySinglePage` for that image only. + - Active-image switching is a UI navigation and does NOT trigger reprocessing. + - Per-frame mode disables marker and markerless-specific controls. + - Per-frame `_settings.txt` round-trip requires re-uploading images in the + same order. +- `llm_readme.md`: add a "Per-Frame Pipeline" section that mirrors the marker / + markerless sections and points back to this plan. + +**Files touched:** +- `js/app.js` +- `js/pipeline.js` +- `documentation.md` +- `AGENTS.md` +- `llm_readme.md` + +**Acceptance:** +- Loading 20 images at 4000×3000 each does not crash the tab (cell size scaled + down). +- Switching active image is instant (no reprocessing). +- Mobile UX is acceptable. +- Documentation is current and the AGENTS.md invariants are stated. + +**Out of scope:** +- New features beyond what Phases 0–8 added. + +--- + +## Cross-Cutting Invariants (apply across every phase) + +- Markers and markerless modes are not allowed to regress at any phase boundary. + Run a demo round-trip for at least one markers demo and one markerless demo + before declaring a phase done. +- `_settings.txt` files saved by `main` must still load in every phase up to and + including Phase 8. Per-image keys are additive. +- Mat lifetime stays explicit. Every new Mat allocation in + `runPerFramePipeline` has a matching `.delete()` in a `finally` block. +- i18n strings are added to **all** locale tables, not just English. +- Mobile single-viewer mode is checked at Phase 6, Phase 7, and Phase 9. + +## Files Touched (cumulative reference) + +| File | Phases | +|------|--------| +| `index.html` | 4, 6, 7 | +| `style.css` | 7 | +| `js/dom-state.js` | 2, 6, 7, 8 | +| `js/load-controller.js` | 2, 4, 8 | +| `js/pipeline.js` | 1, 3, 9 | +| `js/app.js` | 3, 5, 6, 9 | +| `js/ui-controls.js` | 5, 6, 7 | +| `js/settings-io.js` | 8 | +| `js/settings-defaults.js` | 6 | +| `js/i18n.js` | 4, 6, 7 | +| `js/per-frame-strip.js` (new) | 7 | +| `js/source-images.js` (new, optional) | 2 | +| `documentation.md` | 8, 9 | +| `AGENTS.md` | 9 | +| `llm_readme.md` | 9 | + +## Open Items Deliberately Deferred + +- Per-image Page Detection Threshold (v1 keeps it global). +- Auto-matching saved per-image overrides by filename on reload (v1 matches by + upload order). +- Saving image data inside settings files (out of scope; settings stay text). +- Per-image grid mode (every image as its own 1×1; out of scope — frame count + equals image count). From df431a3247c2563804d88cd666dc58a66e76b5ca Mon Sep 17 00:00:00 2001 From: David Vandenbogaerde Date: Thu, 11 Jun 2026 17:10:53 +0200 Subject: [PATCH 02/21] phase 0 --- js/app.js | 145 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 145 insertions(+) diff --git a/js/app.js b/js/app.js index 398b519..3f6c19d 100644 --- a/js/app.js +++ b/js/app.js @@ -72,6 +72,9 @@ import { applyTranslations, getTooltipText, t } from "./i18n.js"; // Final output-size scaling can be done either with browser canvas drawImage() or with OpenCV. // Keep both code paths available for comparison while evaluating tiny-output quality. const bUseOpenCvOutputScaling = true; +// SPIKE: per-frame, remove — flip to true to activate the 2-frame composite spike. +const SPIKE_PER_FRAME = true; +const SPIKE_SHIFT_PX = 4; const MOBILE_VIEWER_BREAKPOINT_PX = 960; const LIVE_THRESHOLD_PREVIEW_MAX_LONG_EDGE_PX = 512; const RECTIFIED_FULL_RES_EAGER_LONG_EDGE_PX = 3000; @@ -4168,6 +4171,148 @@ async function processCurrentImage(requestId = state.processing.requestId) { return; } + // SPIKE: per-frame, remove — build a 2-frame composite from the single-image pipeline result. + if (SPIKE_PER_FRAME && result.rectifiedMat && !result.rectifiedMat.isDeleted()) { + timeProfiled("spikePerFrame", () => { + const srcMat = result.rectifiedMat; + const cellW = srcMat.cols; + const cellH = srcMat.rows; + const compositeW = cellW * 2; + + // Build a 2-cell horizontal composite: [original | shifted copy]. + const composite = new cv.Mat(cellH, compositeW, srcMat.type(), new cv.Scalar(0, 0, 0, 0)); + const roi0 = composite.roi(new cv.Rect(0, 0, cellW, cellH)); + srcMat.copyTo(roi0); + roi0.delete(); + + // Shift the second copy by a few pixels so stabilization produces non-zero offsets. + const roi1 = composite.roi(new cv.Rect(cellW, 0, cellW, cellH)); + const shifted = new cv.Mat(); + const translationMatrix = cv.matFromArray(2, 3, cv.CV_64FC1, [1, 0, SPIKE_SHIFT_PX, 0, 1, SPIKE_SHIFT_PX]); + try { + cv.warpAffine(srcMat, shifted, translationMatrix, new cv.Size(cellW, cellH), cv.INTER_LINEAR, cv.BORDER_REPLICATE, new cv.Scalar()); + shifted.copyTo(roi1); + } finally { + shifted.delete(); + translationMatrix.delete(); + } + roi1.delete(); + + // Build synthetic alignmentInfo for a 1-row × 2-column grid. + const gridBounds = { left: 0, top: 0, width: compositeW, height: cellH }; + const cols = 2; + const rows = 1; + const markerLookup = new Map(); + for (let r = 0; r <= rows; r++) { + for (let c = 0; c <= cols; c++) { + const x = gridBounds.left + gridBounds.width * (c / cols); + const y = gridBounds.top + gridBounds.height * (r / rows); + const key = getMarkerKey(c, r); + markerLookup.set(key, { + col: c, row: r, x, y, + kind: "synthetic", + detectedX: x, detectedY: y, + dx: 0, dy: 0, + confidence: 10, + accepted: true, + }); + } + } + const syntheticAlignmentInfo = { + ok: true, + reason: "spike-per-frame", + includeCornerCrosses: true, + requestedMarkerType: "crosses", + resolvedMarkerType: "crosses", + markerTypeMedianCircularity: null, + rectifiedWidth: compositeW, + rectifiedHeight: cellH, + gridBounds, + cols, + rows, + expectedCount: markerLookup.size, + detectedCount: markerLookup.size, + expectedCrosses: [...markerLookup.values()], + anchorDots: [], + detectedCrosses: [...markerLookup.values()], + rejectedCrosses: [], + markerLookup, + frameDebugQuads: [], + crossRoiTiles: [], + crossRoiTileMap: new Map(), + }; + + // Extract 2 frames from the composite. + const crop = config.crop; + const interpolation = getCvInterpolationFlag(config.exportOptions.resampling); + const spikeFrames = []; + for (let c = 0; c < cols; c++) { + spikeFrames.push(extractSingleFrameToCanvas(composite, syntheticAlignmentInfo, c, 0, crop, interpolation)); + } + + // Build a preview canvas from the composite (reuse the pipeline's approach). + const maxPreviewEdge = 2200; + const longEdge = Math.max(composite.cols, composite.rows); + let spikePreviewCanvas; + if (longEdge <= maxPreviewEdge) { + spikePreviewCanvas = document.createElement("canvas"); + const rgba = new cv.Mat(); + try { + if (composite.type() === cv.CV_8UC3) { + cv.cvtColor(composite, rgba, cv.COLOR_BGR2RGBA); + } else { + composite.copyTo(rgba); + } + spikePreviewCanvas.width = rgba.cols; + spikePreviewCanvas.height = rgba.rows; + const imgData = new ImageData(new Uint8ClampedArray(rgba.data), rgba.cols, rgba.rows); + spikePreviewCanvas.getContext("2d").putImageData(imgData, 0, 0); + } finally { + rgba.delete(); + } + } else { + const scale = maxPreviewEdge / longEdge; + const pw = Math.max(1, Math.round(composite.cols * scale)); + const ph = Math.max(1, Math.round(composite.rows * scale)); + const resized = new cv.Mat(); + const rgba = new cv.Mat(); + try { + cv.resize(composite, resized, new cv.Size(pw, ph), 0, 0, cv.INTER_AREA); + if (resized.type() === cv.CV_8UC3) { + cv.cvtColor(resized, rgba, cv.COLOR_BGR2RGBA); + } else { + resized.copyTo(rgba); + } + spikePreviewCanvas = document.createElement("canvas"); + spikePreviewCanvas.width = rgba.cols; + spikePreviewCanvas.height = rgba.rows; + const imgData = new ImageData(new Uint8ClampedArray(rgba.data), rgba.cols, rgba.rows); + spikePreviewCanvas.getContext("2d").putImageData(imgData, 0, 0); + } finally { + resized.delete(); + rgba.delete(); + } + } + + // Release the original single-image rectified Mat and swap in the composite. + srcMat.delete(); + result.rectifiedMat = composite; + result.rectifiedCanvas = spikePreviewCanvas; + result.frames = spikeFrames; + result.alignmentInfo = syntheticAlignmentInfo; + result.pagePreviewCanvas = null; + result.pagePreviewWidth = 0; + result.pagePreviewHeight = 0; + result.pagePreviewGridQuad = null; + result.pagePreviewGridBounds = null; + result.pageQuadSource = "spike-per-frame"; + result.rectifiedDownloadUsesRawSource = false; + result.statusText = `SPIKE: 2-frame composite (${compositeW}×${cellH}), shift=${SPIKE_SHIFT_PX}px`; + console.log("[SPIKE per-frame]", result.statusText); + }); + } + // SPIKE: per-frame, remove — end of spike block. + timeProfiled("applyPipelineResult", () => { // `runPipeline()` returns raw extracted frame canvases at rectified-sheet/crop resolution. // Those are useful for status text (`Animation size`) but they do not yet include output-size From 171773daa0e0485c6710eed41cee7d03db4663ab Mon Sep 17 00:00:00 2001 From: David Vandenbogaerde Date: Thu, 11 Jun 2026 17:34:28 +0200 Subject: [PATCH 03/21] phase 1 --- js/app.js | 145 ----------------- js/pipeline.js | 312 ++++++++++++++++++++++++------------- per_frame_pipeline_plan.md | 65 +++++++- 3 files changed, 262 insertions(+), 260 deletions(-) diff --git a/js/app.js b/js/app.js index 3f6c19d..398b519 100644 --- a/js/app.js +++ b/js/app.js @@ -72,9 +72,6 @@ import { applyTranslations, getTooltipText, t } from "./i18n.js"; // Final output-size scaling can be done either with browser canvas drawImage() or with OpenCV. // Keep both code paths available for comparison while evaluating tiny-output quality. const bUseOpenCvOutputScaling = true; -// SPIKE: per-frame, remove — flip to true to activate the 2-frame composite spike. -const SPIKE_PER_FRAME = true; -const SPIKE_SHIFT_PX = 4; const MOBILE_VIEWER_BREAKPOINT_PX = 960; const LIVE_THRESHOLD_PREVIEW_MAX_LONG_EDGE_PX = 512; const RECTIFIED_FULL_RES_EAGER_LONG_EDGE_PX = 3000; @@ -4171,148 +4168,6 @@ async function processCurrentImage(requestId = state.processing.requestId) { return; } - // SPIKE: per-frame, remove — build a 2-frame composite from the single-image pipeline result. - if (SPIKE_PER_FRAME && result.rectifiedMat && !result.rectifiedMat.isDeleted()) { - timeProfiled("spikePerFrame", () => { - const srcMat = result.rectifiedMat; - const cellW = srcMat.cols; - const cellH = srcMat.rows; - const compositeW = cellW * 2; - - // Build a 2-cell horizontal composite: [original | shifted copy]. - const composite = new cv.Mat(cellH, compositeW, srcMat.type(), new cv.Scalar(0, 0, 0, 0)); - const roi0 = composite.roi(new cv.Rect(0, 0, cellW, cellH)); - srcMat.copyTo(roi0); - roi0.delete(); - - // Shift the second copy by a few pixels so stabilization produces non-zero offsets. - const roi1 = composite.roi(new cv.Rect(cellW, 0, cellW, cellH)); - const shifted = new cv.Mat(); - const translationMatrix = cv.matFromArray(2, 3, cv.CV_64FC1, [1, 0, SPIKE_SHIFT_PX, 0, 1, SPIKE_SHIFT_PX]); - try { - cv.warpAffine(srcMat, shifted, translationMatrix, new cv.Size(cellW, cellH), cv.INTER_LINEAR, cv.BORDER_REPLICATE, new cv.Scalar()); - shifted.copyTo(roi1); - } finally { - shifted.delete(); - translationMatrix.delete(); - } - roi1.delete(); - - // Build synthetic alignmentInfo for a 1-row × 2-column grid. - const gridBounds = { left: 0, top: 0, width: compositeW, height: cellH }; - const cols = 2; - const rows = 1; - const markerLookup = new Map(); - for (let r = 0; r <= rows; r++) { - for (let c = 0; c <= cols; c++) { - const x = gridBounds.left + gridBounds.width * (c / cols); - const y = gridBounds.top + gridBounds.height * (r / rows); - const key = getMarkerKey(c, r); - markerLookup.set(key, { - col: c, row: r, x, y, - kind: "synthetic", - detectedX: x, detectedY: y, - dx: 0, dy: 0, - confidence: 10, - accepted: true, - }); - } - } - const syntheticAlignmentInfo = { - ok: true, - reason: "spike-per-frame", - includeCornerCrosses: true, - requestedMarkerType: "crosses", - resolvedMarkerType: "crosses", - markerTypeMedianCircularity: null, - rectifiedWidth: compositeW, - rectifiedHeight: cellH, - gridBounds, - cols, - rows, - expectedCount: markerLookup.size, - detectedCount: markerLookup.size, - expectedCrosses: [...markerLookup.values()], - anchorDots: [], - detectedCrosses: [...markerLookup.values()], - rejectedCrosses: [], - markerLookup, - frameDebugQuads: [], - crossRoiTiles: [], - crossRoiTileMap: new Map(), - }; - - // Extract 2 frames from the composite. - const crop = config.crop; - const interpolation = getCvInterpolationFlag(config.exportOptions.resampling); - const spikeFrames = []; - for (let c = 0; c < cols; c++) { - spikeFrames.push(extractSingleFrameToCanvas(composite, syntheticAlignmentInfo, c, 0, crop, interpolation)); - } - - // Build a preview canvas from the composite (reuse the pipeline's approach). - const maxPreviewEdge = 2200; - const longEdge = Math.max(composite.cols, composite.rows); - let spikePreviewCanvas; - if (longEdge <= maxPreviewEdge) { - spikePreviewCanvas = document.createElement("canvas"); - const rgba = new cv.Mat(); - try { - if (composite.type() === cv.CV_8UC3) { - cv.cvtColor(composite, rgba, cv.COLOR_BGR2RGBA); - } else { - composite.copyTo(rgba); - } - spikePreviewCanvas.width = rgba.cols; - spikePreviewCanvas.height = rgba.rows; - const imgData = new ImageData(new Uint8ClampedArray(rgba.data), rgba.cols, rgba.rows); - spikePreviewCanvas.getContext("2d").putImageData(imgData, 0, 0); - } finally { - rgba.delete(); - } - } else { - const scale = maxPreviewEdge / longEdge; - const pw = Math.max(1, Math.round(composite.cols * scale)); - const ph = Math.max(1, Math.round(composite.rows * scale)); - const resized = new cv.Mat(); - const rgba = new cv.Mat(); - try { - cv.resize(composite, resized, new cv.Size(pw, ph), 0, 0, cv.INTER_AREA); - if (resized.type() === cv.CV_8UC3) { - cv.cvtColor(resized, rgba, cv.COLOR_BGR2RGBA); - } else { - resized.copyTo(rgba); - } - spikePreviewCanvas = document.createElement("canvas"); - spikePreviewCanvas.width = rgba.cols; - spikePreviewCanvas.height = rgba.rows; - const imgData = new ImageData(new Uint8ClampedArray(rgba.data), rgba.cols, rgba.rows); - spikePreviewCanvas.getContext("2d").putImageData(imgData, 0, 0); - } finally { - resized.delete(); - rgba.delete(); - } - } - - // Release the original single-image rectified Mat and swap in the composite. - srcMat.delete(); - result.rectifiedMat = composite; - result.rectifiedCanvas = spikePreviewCanvas; - result.frames = spikeFrames; - result.alignmentInfo = syntheticAlignmentInfo; - result.pagePreviewCanvas = null; - result.pagePreviewWidth = 0; - result.pagePreviewHeight = 0; - result.pagePreviewGridQuad = null; - result.pagePreviewGridBounds = null; - result.pageQuadSource = "spike-per-frame"; - result.rectifiedDownloadUsesRawSource = false; - result.statusText = `SPIKE: 2-frame composite (${compositeW}×${cellH}), shift=${SPIKE_SHIFT_PX}px`; - console.log("[SPIKE per-frame]", result.statusText); - }); - } - // SPIKE: per-frame, remove — end of spike block. - timeProfiled("applyPipelineResult", () => { // `runPipeline()` returns raw extracted frame canvases at rectified-sheet/crop resolution. // Those are useful for status text (`Animation size`) but they do not yet include output-size diff --git a/js/pipeline.js b/js/pipeline.js index cd895a5..0140b5b 100644 --- a/js/pipeline.js +++ b/js/pipeline.js @@ -247,6 +247,189 @@ function clampPositiveConvolutionToUint8(conv32, target8) { * }} */ export function runPipeline(sourceCanvas, config, requestId, throwIfAborted) { + const useMarkerlessAlignment = config.alignmentPipeline === "markerless"; + let rectifiedWarp = null; + let pageRectification = null; + + try { + // Page detection + rectification + Post-Rotation now live in a reusable helper so the per-frame + // pipeline can run them once per uploaded image. The caller owns the returned `rectifiedWarp`. + pageRectification = rectifySinglePage(sourceCanvas, config, requestId, throwIfAborted); + rectifiedWarp = pageRectification.rectifiedWarp; + const { + pageQuad, + pageQuadSource, + pageWarpPreviewCanvas, + pageWarpPreviewWidth, + pageWarpPreviewHeight, + useNearIdentityRectification, + threshVal, + pageSizeLow, + pageSizeHigh, + } = pageRectification; + + const pagePreviewGridBounds = rectifiedWarp.previewGridQuad + ? { ...rectifiedWarp.gridBounds } + : null; + + // Resolve the marker lattice if enabled; otherwise keep the nominal grid and unrefined ROI views. + const alignmentInfo = config.useCrossAlignment + ? ( + useMarkerlessAlignment + ? buildMarkerlessAlignmentData( + rectifiedWarp.visionMat, + config.frameCols, + config.frameRows, + config.crossRoiScale, + rectifiedWarp.gridBounds, + config.paperMarginXPx, + config.paperMarginYPx, + { + useDarkness: config.markerlessUseDarkness, + useTexture: config.markerlessUseTexture, + useVariance: config.markerlessUseVariance, + lightOnDark: config.lightOnDarkDesign, + blurScale: config.markerlessAutocorrelationBlurScale, + }, + ) + : buildCrossAlignmentData( + rectifiedWarp.visionMat, + config.frameCols, + config.frameRows, + config.crossRoiScale, + rectifiedWarp.gridBounds, + { + markerType: config.alignmentMarkerType, + includeCornerCrosses: rectifiedWarp.includeCornerCrosses, + detectCrossesWithConvolution: config.detectCrossesWithConvolution, + } + ) + ) + : buildUnrefinedCrossRegionInfo( + rectifiedWarp.visionMat, + config.frameCols, + config.frameRows, + "disabled", + rectifiedWarp.gridBounds, + config.crossRoiScale, + { + markerType: config.alignmentMarkerType, + includeCornerCrosses: rectifiedWarp.includeCornerCrosses, + } + ); + // In the all-cross format, the coarse quad is only approximate; use the detected corner crosses + // to tighten the working grid bounds before frame extraction. + if (rectifiedWarp.includeCornerCrosses) { + refineAlignmentBoundsFromCornerCrosses(alignmentInfo); + } + throwIfAborted(requestId); + + // Extract each animation frame from the styled rectified sheet with the chosen interpolation mode. + const frames = sliceRectifiedToCanvases( + rectifiedWarp.styledMat, + alignmentInfo, + config.crop, + getCvInterpolationFlag(config.exportOptions.resampling), + requestId, + throwIfAborted + ); + const rectifiedCanvas = matToPreviewCanvas(rectifiedWarp.styledMat, RECTIFIED_PREVIEW_LONG_EDGE_PX); + const rectifiedMat = rectifiedWarp.styledMat; + rectifiedWarp.styledMat = null; + const statusText = buildStatusText({ + threshVal, + rawWidth: sourceCanvas.width, + rawHeight: sourceCanvas.height, + pageAreaPct: pageQuad.areaPct, + pageWarpWidth: pageSizeLow.width, + pageWarpHeight: pageSizeLow.height, + highPageWarpWidth: pageSizeHigh.width, + highPageWarpHeight: pageSizeHigh.height, + alignmentInfo, + frameCount: frames.length, + expectedFrameCount: config.frameCols * config.frameRows, + rectifiedWidth: rectifiedMat.cols, + rectifiedHeight: rectifiedMat.rows, + animationWidth: frames[0]?.width || 0, + animationHeight: frames[0]?.height || 0, + gridDetector: "cross-only", + }); + + return { + frames, + rectifiedCanvas, + rectifiedMat, + pagePreviewCanvas: pageWarpPreviewCanvas, + pagePreviewWidth: pageWarpPreviewWidth, + pagePreviewHeight: pageWarpPreviewHeight, + pagePreviewGridQuad: rectifiedWarp.previewGridQuad || null, + pagePreviewGridBounds, + alignmentInfo, + statusText, + pageQuadPoints: pageQuad.points, + pageQuadSource, + rectifiedDownloadUsesRawSource: + useNearIdentityRectification && + useMarkerlessAlignment && + Math.abs(Number(config.postRotationDeg) || 0) < 1e-6, + }; + } catch (error) { + // `rectifySinglePage` already attaches a partialResult for failures in the page-rectification + // stage. Only fill it in here for failures that happen later (alignment / extraction), where the + // page quad and warp preview are already available from the returned rectification info. + if (error?.name !== "ProcessAbortedError") { + if (error && typeof error === "object" && !error.partialResult) { + error.partialResult = { + pageQuadPoints: pageRectification?.pageQuad?.points || null, + pageQuadSource: pageRectification?.pageQuadSource || "pipeline-detection", + rectifiedCanvas: pageRectification?.pageWarpPreviewCanvas || null, + }; + } + } + throw error; + } finally { + // `rectifySinglePage` released every intermediate Mat itself; only the returned rectifiedWarp is + // still owned here. Its styledMat is set to null after being handed off as `rectifiedMat`. + rectifiedWarp?.visionMat?.delete(); + rectifiedWarp?.styledMat?.delete(); + } +} + +/** + * Detect, rectify, and Post-Rotate a single source page into a working sheet. + * + * This is the shared page-localization + rectification stage of the pipeline, extracted so it can be + * reused per uploaded image by the per-frame pipeline. Markers and markerless modes both call it via + * {@link runPipeline}; behaviour is identical to the previous inline implementation. + * + * Ownership: the caller takes ownership of the returned `rectifiedWarp` (both `visionMat` and + * `styledMat`) and must delete those Mats. Every other intermediate Mat is released internally, + * including on failure. + * + * Reads from `config`: `alignmentPipeline` (near-identity fast path branches on markerless), + * `lightOnDarkDesign`, `thresholdMethod`, `thresholdOffset`, `manualPageQuadPoints`, + * `fallbackPageQuadPoints`, `paperAspect`, `postRotationDeg`, plus the frame-grid fields consumed by + * the marker rectifier (`useRectifiedAsSource`, `frameCols`, `frameRows`, `crossRoiScale`, + * `paperMarginXPx`, `paperMarginYPx`, `boundarySensitivity`, `boundaryPersistencePx`). + * + * @param {HTMLCanvasElement} sourceCanvas + * @param {object} config + * @param {number} requestId + * @param {(requestId:number) => void} throwIfAborted + * @returns {{ + * rectifiedWarp: {visionMat:cv.Mat, styledMat:cv.Mat, gridBounds:{left:number, top:number, width:number, height:number}, includeCornerCrosses:boolean, previewGridQuad:object|null}, + * pageQuad: {points:{x:number,y:number}[], areaPct:number, quadAreaPx:number}, + * pageQuadSource: "pipeline-detection" | "manual-override" | "threshold-preview-fallback", + * pageWarpPreviewCanvas: HTMLCanvasElement | null, + * pageWarpPreviewWidth: number, + * pageWarpPreviewHeight: number, + * useNearIdentityRectification: boolean, + * threshVal: number, + * pageSizeLow: cv.Size, + * pageSizeHigh: cv.Size + * }} + */ +function rectifySinglePage(sourceCanvas, config, requestId, throwIfAborted) { const useMarkerlessAlignment = config.alignmentPipeline === "markerless"; const useInvertedMarkerVision = !useMarkerlessAlignment && config.lightOnDarkDesign; const styledSrcRgba = cv.imread(sourceCanvas); @@ -378,126 +561,33 @@ export function runPipeline(sourceCanvas, config, requestId, throwIfAborted) { pageWarpHigh = null; } applyPostRectificationRotation(rectifiedWarp, config.postRotationDeg); - const pagePreviewGridBounds = rectifiedWarp.previewGridQuad - ? { ...rectifiedWarp.gridBounds } - : null; throwIfAborted(requestId); - // Resolve the marker lattice if enabled; otherwise keep the nominal grid and unrefined ROI views. - const alignmentInfo = config.useCrossAlignment - ? ( - useMarkerlessAlignment - ? buildMarkerlessAlignmentData( - rectifiedWarp.visionMat, - config.frameCols, - config.frameRows, - config.crossRoiScale, - rectifiedWarp.gridBounds, - config.paperMarginXPx, - config.paperMarginYPx, - { - useDarkness: config.markerlessUseDarkness, - useTexture: config.markerlessUseTexture, - useVariance: config.markerlessUseVariance, - lightOnDark: config.lightOnDarkDesign, - blurScale: config.markerlessAutocorrelationBlurScale, - }, - ) - : buildCrossAlignmentData( - rectifiedWarp.visionMat, - config.frameCols, - config.frameRows, - config.crossRoiScale, - rectifiedWarp.gridBounds, - { - markerType: config.alignmentMarkerType, - includeCornerCrosses: rectifiedWarp.includeCornerCrosses, - detectCrossesWithConvolution: config.detectCrossesWithConvolution, - } - ) - ) - : buildUnrefinedCrossRegionInfo( - rectifiedWarp.visionMat, - config.frameCols, - config.frameRows, - "disabled", - rectifiedWarp.gridBounds, - config.crossRoiScale, - { - markerType: config.alignmentMarkerType, - includeCornerCrosses: rectifiedWarp.includeCornerCrosses, - } - ); - // In the all-cross format, the coarse quad is only approximate; use the detected corner crosses - // to tighten the working grid bounds before frame extraction. - if (rectifiedWarp.includeCornerCrosses) { - refineAlignmentBoundsFromCornerCrosses(alignmentInfo); - } - throwIfAborted(requestId); - - // Extract each animation frame from the styled rectified sheet with the chosen interpolation mode. - const frames = sliceRectifiedToCanvases( - rectifiedWarp.styledMat, - alignmentInfo, - config.crop, - getCvInterpolationFlag(config.exportOptions.resampling), - requestId, - throwIfAborted - ); - const rectifiedCanvas = matToPreviewCanvas(rectifiedWarp.styledMat, RECTIFIED_PREVIEW_LONG_EDGE_PX); - const rectifiedMat = rectifiedWarp.styledMat; - rectifiedWarp.styledMat = null; - const statusText = buildStatusText({ - threshVal, - rawWidth: sourceCanvas.width, - rawHeight: sourceCanvas.height, - pageAreaPct: pageQuad.areaPct, - pageWarpWidth: pageSizeLow.width, - pageWarpHeight: pageSizeLow.height, - highPageWarpWidth: pageSizeHigh.width, - highPageWarpHeight: pageSizeHigh.height, - alignmentInfo, - frameCount: frames.length, - expectedFrameCount: config.frameCols * config.frameRows, - rectifiedWidth: rectifiedMat.cols, - rectifiedHeight: rectifiedMat.rows, - animationWidth: frames[0]?.width || 0, - animationHeight: frames[0]?.height || 0, - gridDetector: "cross-only", - }); - return { - frames, - rectifiedCanvas, - rectifiedMat, - pagePreviewCanvas: pageWarpPreviewCanvas, - pagePreviewWidth: pageWarpPreviewWidth, - pagePreviewHeight: pageWarpPreviewHeight, - pagePreviewGridQuad: rectifiedWarp.previewGridQuad || null, - pagePreviewGridBounds, - alignmentInfo, - statusText, - pageQuadPoints: pageQuad.points, + rectifiedWarp, + pageQuad, pageQuadSource, - rectifiedDownloadUsesRawSource: - useNearIdentityRectification && - useMarkerlessAlignment && - Math.abs(Number(config.postRotationDeg) || 0) < 1e-6, + pageWarpPreviewCanvas, + pageWarpPreviewWidth, + pageWarpPreviewHeight, + useNearIdentityRectification, + threshVal, + pageSizeLow, + pageSizeHigh, }; } catch (error) { - if (error?.name !== "ProcessAbortedError") { - if (error && typeof error === "object") { - error.partialResult = { - pageQuadPoints: pageQuad?.points || null, - pageQuadSource, - rectifiedCanvas: pageWarpPreviewCanvas, - }; - } + if (error?.name !== "ProcessAbortedError" && error && typeof error === "object") { + error.partialResult = { + pageQuadPoints: pageQuad?.points || null, + pageQuadSource, + rectifiedCanvas: pageWarpPreviewCanvas, + }; } - throw error; - } finally { + // On failure the caller never receives `rectifiedWarp`, so release it here to avoid a Mat leak. rectifiedWarp?.visionMat?.delete(); rectifiedWarp?.styledMat?.delete(); + throw error; + } finally { pageWarpLow?.visionMat?.delete(); pageWarpLow?.styledMat?.delete(); if (pageWarpHigh && pageWarpHigh.visionMat !== visionSrc) pageWarpHigh.visionMat?.delete(); diff --git a/per_frame_pipeline_plan.md b/per_frame_pipeline_plan.md index 7cf4ec1..7099ee8 100644 --- a/per_frame_pipeline_plan.md +++ b/per_frame_pipeline_plan.md @@ -137,6 +137,37 @@ into a reusable helper without behavioral change to markers or markerless mode. - Any change to `runPipeline`'s output shape. - Any change to the marker/markerless detector functions. +**As built (Phase 1 — COMPLETE):** +- `rectifySinglePage` was implemented in `js/pipeline.js`, taking the + **full `config` object** rather than the trimmed `perPageConfig` subset + proposed above. The block it owns (`buildFrameGridRectification_fromCrosses`, + via the marker path) needs the frame-grid fields (`useRectifiedAsSource`, + `frameCols`, `frameRows`, `crossRoiScale`, `paperMarginXPx`, `paperMarginYPx`, + `boundarySensitivity`, `boundaryPersistencePx`) in addition to the page-stage + fields, so passing a strict subset would have meant re-threading the entire + grid config anyway. The JSDoc on `rectifySinglePage` enumerates exactly which + `config` keys it reads. Phase 3's per-image call therefore passes a shallow + copy of `config` with per-image overrides (see Phase 3, step 2). +- Actual signature/return: + ``` + rectifySinglePage(sourceCanvas, config, requestId, throwIfAborted) + -> { rectifiedWarp, pageQuad, pageQuadSource, pageWarpPreviewCanvas, + pageWarpPreviewWidth, pageWarpPreviewHeight, + useNearIdentityRectification, threshVal, pageSizeLow, pageSizeHigh } + ``` + `threshVal`, `pageSizeLow`, and `pageSizeHigh` were added beyond the proposed + shape so `runPipeline` can keep building byte-identical status text. +- Mat lifetime: `rectifySinglePage` releases every intermediate Mat in its own + `finally` and releases `rectifiedWarp` in its `catch` on failure, so the + caller only ever owns a successfully returned `rectifiedWarp`. `runPipeline`'s + `finally` shrank to just deleting `rectifiedWarp.visionMat` / `styledMat`. +- Error handling: `rectifySinglePage` attaches `error.partialResult` for + page-stage failures; `runPipeline` only fills it in for later + (alignment/extraction) failures, guarded by `!error.partialResult`. +- Phase 0 spike code in `js/app.js` (the `SPIKE_PER_FRAME` constants and the + 2-frame composite block in `processCurrentImage`) was removed as part of this + work. Verified via demo round-trip; markers and markerless modes unchanged. + --- ### Phase 2 — State: `state.source.images[]` and active-image accessor @@ -196,10 +227,36 @@ testing. - Add `runPerFramePipeline(images, config, requestId, throwIfAborted)`: 1. For each `image` in `images`, build a per-image `sourceCanvas` (already in `state.source.images[i].canvas`). - 2. Call `rectifySinglePage(sourceCanvas, perImagePageConfig, …)`. The - `perImagePageConfig` is built per image, using that image's - `manualPageContour` and `postRotationDeg`. - 3. Collect rectified `styledMat`s. + 2. Call `rectifySinglePage(sourceCanvas, perImageConfig, …)`. As implemented + in Phase 1, `rectifySinglePage` takes the **full `config` object** (not a + trimmed `perPageConfig` subset) and reads the page-relevant fields plus the + frame-grid fields needed by the marker rectifier; its JSDoc lists exactly + which keys it consumes. So `perImageConfig` is a shallow copy of the base + `config` with the per-image fields overridden: + ``` + const perImageConfig = { + ...config, + // Treat each rectified page as a whole working sheet (no cross sweep, + // no frame-grid crop). rectifySinglePage branches its near-identity + // fast path and grid rectification on this value, so per-frame mode + // must alias to "markerless" here rather than passing "per-frame". + alignmentPipeline: "markerless", + // Per-image page-corner override (source-space quad) for this image. + manualPageQuadPoints: images[i].manualPageContour ?? null, + // No live threshold-preview fallback when rectifying per image. + fallbackPageQuadPoints: null, + // Per-image Post-Rotation (Phase 5 stores this on the image entry). + postRotationDeg: images[i].postRotationDeg ?? 0, + }; + ``` + The base `config.alignmentPipeline` stays `"per-frame"` for the dispatcher + and for downstream readConfig/UI gating; only the per-image copy handed to + `rectifySinglePage` is aliased to `"markerless"`. + 3. From each call, take `result.rectifiedWarp`; the caller owns it (per the + Phase 1 contract). Use `rectifiedWarp.styledMat` as the cell source and + delete `rectifiedWarp.visionMat` immediately (per-frame mode does not run + per-image alignment on it). Each `styledMat` is deleted after it has been + resized/copied into the composite in step 6. 4. Decide common cell size: - `cellW = clamp(median(rectifiedWidths), MIN_CELL_PX, MAX_CELL_PX)` - `cellH = clamp(median(rectifiedHeights), MIN_CELL_PX, MAX_CELL_PX)` From 929f95e70abb83af5ca3a2b51b184b5dc82701c2 Mon Sep 17 00:00:00 2001 From: David Vandenbogaerde Date: Thu, 11 Jun 2026 17:45:01 +0200 Subject: [PATCH 04/21] phase 2 --- js/dom-state.js | 4 ++ js/load-controller.js | 21 +++++- js/source-images.js | 142 +++++++++++++++++++++++++++++++++++++ per_frame_pipeline_plan.md | 45 ++++++++++++ 4 files changed, 211 insertions(+), 1 deletion(-) create mode 100644 js/source-images.js diff --git a/js/dom-state.js b/js/dom-state.js index 5c466d6..77e0d08 100644 --- a/js/dom-state.js +++ b/js/dom-state.js @@ -286,6 +286,10 @@ export const state = { dragUrl: "", ownedObjectUrl: "", canvas: document.createElement("canvas"), + // Per-image source entries for the per-frame pipeline. The legacy fields above project the + // active entry. During Phase 2 this array always holds 0 or 1 entries. See js/source-images.js. + images: [], + activeImageIndex: 0, rawPageContour: null, // Tracks where the currently drawn page quad came from; this is display/status metadata, not // an instruction to bypass full-resolution page detection. diff --git a/js/load-controller.js b/js/load-controller.js index 9dce1f9..4bdab68 100644 --- a/js/load-controller.js +++ b/js/load-controller.js @@ -5,6 +5,7 @@ * and the staged process of loading a new source image into the app. */ import { t } from "./i18n.js"; +import { createSourceImageEntry, releaseAllSourceImages } from "./source-images.js"; /** * Toggle the small busy spinners used during image loading and processing. * @@ -38,12 +39,15 @@ export async function waitForNextPaint() { } /** - * Release any blob URL that the app currently owns for raw-photo drag/download behavior. + * Release any blob URLs the app currently owns for raw-photo drag/download behavior, including the + * per-image source entries, so nothing leaks across image reloads. * * @param {import("./dom-state.js").state} state * @returns {void} */ export function releaseOwnedSourceUrl(state) { + // Revoke per-image blob URLs and free cached Mats; clears state.source.images back to empty. + releaseAllSourceImages(state); if (!state.source.ownedObjectUrl) return; URL.revokeObjectURL(state.source.ownedObjectUrl); state.source.ownedObjectUrl = ""; @@ -209,6 +213,18 @@ export async function loadImageSource({ syncRawPhotoCreditDisplay?.(); state.source.rawPageContour = null; drawImageToCanvas(image, state.source.canvas); + // Register this image as the single per-image source entry. The legacy state.source.* fields + // continue to project this active entry; later phases grow images[] to more than one entry. + const sourceEntry = createSourceImageEntry({ + image, + filename: state.source.filename, + mimeType: state.source.mimeType, + ownedObjectUrl: state.source.ownedObjectUrl, + dragUrl: state.source.dragUrl, + canvas: state.source.canvas, + }); + state.source.images = [sourceEntry]; + state.source.activeImageIndex = 0; syncPaperPresetUi?.(); renderRawPreview(); const hasSettingsText = !!settingsText.trim(); @@ -219,6 +235,9 @@ export async function loadImageSource({ applyLoadedSettingsText(settingsText); state.source.settingsLoaded = true; } + // Keep the active entry's page-corner override in sync with the legacy field after any + // settings load. Per-image override routing arrives in a later phase; this just mirrors state. + sourceEntry.manualPageContour = state.source.manualPageContour ?? null; invalidateAppearanceCache(); setStatus(`${loadedWhat}\n${t("status.analyzingPage")}`); await waitForNextPaint(); diff --git a/js/source-images.js b/js/source-images.js new file mode 100644 index 0000000..f445a57 --- /dev/null +++ b/js/source-images.js @@ -0,0 +1,142 @@ +/** + * Per-image source state helpers. + * + * The per-frame alignment pipeline lets the user upload one image per animation frame. To support + * that without rewriting every legacy caller at once, the canonical per-image data lives in + * `state.source.images[]` and the existing `state.source.image / canvas / dragUrl / ...` fields are + * kept as projections of the active entry. + * + * During Phase 2 of the per-frame rollout, `state.source.images[]` always holds 0 or 1 entries. + * Later phases add multi-image upload, per-image overrides, and the strip UI on top of this shape. + */ + +/** + * @typedef {Object} SourceImageEntry + * @property {HTMLImageElement | null} image Decoded source image element. + * @property {string} filename Original filename for status text / settings matching. + * @property {string} mimeType Source MIME type (e.g. `image/jpeg`). + * @property {string} ownedObjectUrl Blob URL this entry owns and must revoke on release ("" if none). + * @property {string} dragUrl URL used for raw-photo drag/download (may be a demo path or blob URL). + * @property {HTMLCanvasElement | null} canvas Source-resolution canvas the CV pipeline reads from. + * @property {Array<{x:number,y:number}> | null} manualPageContour Per-image source-space page quad override. + * @property {number} postRotationDeg Per-image Post-Rotation, applied after page rectification. + * @property {*} rectifiedMatCache Cached rectified Mat (or `{visionMat, styledMat}`); released on clear. + * @property {boolean} rectifiedDirty Whether the cached rectified Mat must be rebuilt. + */ + +/** + * Build a fresh per-image source entry with sensible defaults. + * + * @param {Partial} [fields={}] + * @returns {SourceImageEntry} + */ +export function createSourceImageEntry(fields = {}) { + return { + image: fields.image ?? null, + filename: fields.filename ?? "", + mimeType: fields.mimeType ?? "", + ownedObjectUrl: fields.ownedObjectUrl ?? "", + dragUrl: fields.dragUrl ?? "", + canvas: fields.canvas ?? null, + manualPageContour: fields.manualPageContour ?? null, + postRotationDeg: fields.postRotationDeg ?? 0, + rectifiedMatCache: fields.rectifiedMatCache ?? null, + rectifiedDirty: fields.rectifiedDirty ?? true, + }; +} + +/** + * Return the active per-image source entry, or `null` when no images are loaded. + * + * @param {import("./dom-state.js").state} state + * @returns {SourceImageEntry | null} + */ +export function getActiveSourceImage(state) { + const images = state.source.images; + if (!Array.isArray(images) || images.length === 0) return null; + const index = state.source.activeImageIndex; + if (!Number.isInteger(index) || index < 0 || index >= images.length) return null; + return images[index]; +} + +/** + * Set the active per-image index (clamped to the loaded range) and return the new active entry. + * + * @param {import("./dom-state.js").state} state + * @param {number} index + * @returns {SourceImageEntry | null} + */ +export function setActiveSourceImage(state, index) { + const images = state.source.images; + if (!Array.isArray(images) || images.length === 0) { + state.source.activeImageIndex = 0; + return null; + } + const clamped = Math.max(0, Math.min(Math.trunc(index) || 0, images.length - 1)); + state.source.activeImageIndex = clamped; + return images[clamped]; +} + +/** + * Release any cached rectified Mat held on a per-image entry. + * + * Handles both a bare OpenCV `Mat` and the `{ visionMat, styledMat }` rectified-warp shape produced + * by `rectifySinglePage`. + * + * @param {SourceImageEntry | null | undefined} entry + * @returns {void} + */ +export function releaseEntryRectifiedCache(entry) { + if (!entry) return; + const cache = entry.rectifiedMatCache; + if (cache) { + if (typeof cache.delete === "function") { + try { + cache.delete(); + } catch { + /* already freed */ + } + } else { + for (const mat of [cache.visionMat, cache.styledMat]) { + if (mat && typeof mat.delete === "function") { + try { + mat.delete(); + } catch { + /* already freed */ + } + } + } + } + } + entry.rectifiedMatCache = null; + entry.rectifiedDirty = true; +} + +/** + * Release every per-image source entry: revoke owned blob URLs, free cached Mats, and reset the + * per-image array back to empty. Safe to call when no images are loaded. + * + * @param {import("./dom-state.js").state} state + * @returns {void} + */ +export function releaseAllSourceImages(state) { + const images = state.source.images; + if (Array.isArray(images)) { + for (const entry of images) { + if (!entry) continue; + if (entry.ownedObjectUrl) { + try { + URL.revokeObjectURL(entry.ownedObjectUrl); + } catch { + /* already revoked */ + } + entry.ownedObjectUrl = ""; + } + releaseEntryRectifiedCache(entry); + entry.image = null; + entry.canvas = null; + } + } + state.source.images = []; + state.source.activeImageIndex = 0; +} diff --git a/per_frame_pipeline_plan.md b/per_frame_pipeline_plan.md index 7099ee8..ce39f75 100644 --- a/per_frame_pipeline_plan.md +++ b/per_frame_pipeline_plan.md @@ -214,6 +214,38 @@ transparently. - Per-image overrides. (Phase 5.) - Any UI for the image strip. +**As built (Phase 2 — COMPLETE):** +- `state.source` gained `images: []` and `activeImageIndex: 0` in `js/dom-state.js`. + The legacy `state.source.image / filename / mimeType / dragUrl / ownedObjectUrl / + canvas / manualPageContour` fields stay populated and are treated as projections + of the active entry. +- New module `js/source-images.js` exports: + - `createSourceImageEntry(fields)` — builds an entry with the documented shape + (`image, filename, mimeType, ownedObjectUrl, dragUrl, canvas, + manualPageContour, postRotationDeg, rectifiedMatCache: null, + rectifiedDirty: true`). + - `getActiveSourceImage(state)` — active entry or `null`. + - `setActiveSourceImage(state, index)` — clamps and returns the new active entry. + - `releaseAllSourceImages(state)` — revokes per-entry blob URLs, frees cached + Mats, resets `images` to `[]` and `activeImageIndex` to `0`. + - `releaseEntryRectifiedCache(entry)` — helper that frees either a bare `Mat` or + a `{ visionMat, styledMat }` rectified-warp cache (added beyond the proposed + list because Phase 9 / Phase 3 caches use that shape). +- `js/load-controller.js`: + - `loadImageSource`'s `image.onload` now registers the loaded image as the single + entry (`state.source.images = [entry]; activeImageIndex = 0`). The entry's + `canvas` aliases the shared `state.source.canvas` (only one image exists in this + phase); the entry's `manualPageContour` is mirrored from the legacy field after + any settings file is applied. **Note for Phase 4:** this canvas aliasing is only + safe while `images[]` holds at most one entry — Phase 4 must give each entry its + own dedicated canvas (see Phase 4 scope). + - `releaseOwnedSourceUrl(state)` now wraps `releaseAllSourceImages(state)` before + revoking the legacy `ownedObjectUrl`, so loading a new image clears prior + `images[]` entries and no blob URLs leak across reloads. +- No mode logic, multi-file handling, per-image overrides, or strip UI were added. + Markers/markerless flows are unaffected (changes are additive; the legacy fields + the pipeline reads remain authoritative). + --- ### Phase 3 — Pipeline: `runPerFramePipeline` + dispatcher @@ -324,6 +356,17 @@ populate `state.source.images[]`; never silently drop additional images. the drag payload (or file input), build per-image entries for each. Sibling `_settings.txt` is still matched against the first image's filename and applied once after all images are loaded. +- **Per-entry canvases (carried over from Phase 2):** Phase 2 left each entry's + `canvas` aliasing the shared `state.source.canvas`, which is only safe while + `images[]` holds at most one entry. Phase 4 introduces multiple entries, so each + entry **must** get its own dedicated source-resolution `canvas` drawn from its + own decoded image (do not reuse the shared `state.source.canvas` for more than + one entry, or every cell in `runPerFramePipeline` would rectify the same image). + The legacy `state.source.canvas` / `state.source.image` should then project the + **active** entry (e.g. point `state.source.canvas` at + `images[activeImageIndex].canvas`), so legacy single-image callers keep reading + the active image. Update the single-image load path accordingly so it produces a + per-entry canvas too, keeping one code path for the 1-image and N-image cases. - `index.html`: `` and drop-zone copy update. - `js/i18n.js`: drop-zone copy strings for per-frame mode (and a generic copy that mentions multi-file support; localize across all locale tables). @@ -345,6 +388,8 @@ populate `state.source.images[]`; never silently drop additional images. - File picker with multi-select: same behavior as drag. - Mixed drag (N images + 1 `_settings.txt`): settings file is applied against the first image's name; per-frame mode is selected if N > 1. +- Each loaded entry has its own distinct `canvas` (no two entries share a canvas + reference, and none alias the shared `state.source.canvas` once N > 1). **Out of scope:** - Strip UI (Phase 7). From 631a07f95cd87d2807b1c3808ee01b614d169743 Mon Sep 17 00:00:00 2001 From: David Vandenbogaerde Date: Thu, 11 Jun 2026 17:54:45 +0200 Subject: [PATCH 05/21] phase 3 --- js/app.js | 18 +++- js/dom-state.js | 3 + js/pipeline.js | 192 ++++++++++++++++++++++++++++++++++++- per_frame_pipeline_plan.md | 40 ++++++++ 4 files changed, 249 insertions(+), 4 deletions(-) diff --git a/js/app.js b/js/app.js index 398b519..7e7750e 100644 --- a/js/app.js +++ b/js/app.js @@ -2941,6 +2941,10 @@ function readConfig() { const frameCols = Math.max(1, Math.min(20, Math.round(Number(dom.frameCols.value) || SETTINGS_DEFAULTS.layout.frameCols))); const frameRows = Math.max(1, Math.min(20, Math.round(Number(dom.frameRows.value) || SETTINGS_DEFAULTS.layout.frameRows))); const sourceFrameCount = Math.max(1, frameCols * frameRows); + // Per-frame mode is selected by its own radio (added in Phase 6) or, until then, by a dev flag so + // the pipeline can be exercised before the UI exists. The radio ref is read with optional chaining + // so this stays correct both before and after Phase 6 wires `dom.alignmentPipelinePerFrame`. + const perFrameModeActive = !!dom.alignmentPipelinePerFrame?.checked || !!state.runtime.forcePerFrameMode; const encodingQuality = getEncodingQualityValue(); const readSearchInset = (input, fallback) => Math.max( 0, @@ -2978,7 +2982,11 @@ function readConfig() { : SETTINGS_DEFAULTS.detection.postRotationDeg ) ), - alignmentPipeline: dom.alignmentPipelineMarkerless.checked ? "markerless" : "markers", + alignmentPipeline: perFrameModeActive + ? "per-frame" + : dom.alignmentPipelineMarkerless.checked + ? "markerless" + : "markers", // The radio group exposes a temporary-friendly UI label, but the config keeps stable internal // ids so settings files and solver branching do not depend on user-facing wording. stabilizationMethod: dom.stabilizationMethodAverage?.checked ? "difference-from-average" : "pairwise-cyclic", @@ -3011,7 +3019,11 @@ function readConfig() { markerlessUseVariance: dom.markerlessUseVariance ? dom.markerlessUseVariance.checked : true, lightOnDarkDesign: dom.lightOnDarkDesign ? dom.lightOnDarkDesign.checked : false, detectCrossesWithConvolution: (dom.alignmentPipelineMarkers.checked && dom.alignmentMarkerType.value === "crosses") && dom.detectCrossesWithConvolution.checked, - useCrossAlignment: dom.alignmentPipelineMarkerless.checked ? true : dom.useCrossAlignment.checked, + useCrossAlignment: perFrameModeActive + ? false + : dom.alignmentPipelineMarkerless.checked + ? true + : dom.useCrossAlignment.checked, useRectifiedAsSource: false, crop: { left: Math.max(0, Math.round(Number(dom.cropLeft.value) || 0)), @@ -4162,7 +4174,7 @@ async function processCurrentImage(requestId = state.processing.requestId) { try { const config = timeProfiled("readConfig", () => readConfig()); - const result = timeProfiled("runPipeline", () => runPipeline(state.source.canvas, config, requestId, throwIfProcessAborted)); + const result = timeProfiled("runPipeline", () => runPipeline(state.source.canvas, config, requestId, throwIfProcessAborted, state.source.images)); if (requestId !== state.processing.requestId) { finishTimingProfile(timingProfile); return; diff --git a/js/dom-state.js b/js/dom-state.js index 77e0d08..c709ba9 100644 --- a/js/dom-state.js +++ b/js/dom-state.js @@ -276,6 +276,9 @@ export const state = { mobileSingleViewerMode: false, activeViewerTab: "raw", activeMobileControlTab: "layout", + // Phase 3 dev flag: forces readConfig() to emit alignmentPipeline "per-frame" before the + // real radio (Phase 6) exists, so the per-frame pipeline can be exercised from the console. + forcePerFrameMode: false, }, source: { image: null, diff --git a/js/pipeline.js b/js/pipeline.js index 0140b5b..a4c0b8d 100644 --- a/js/pipeline.js +++ b/js/pipeline.js @@ -26,6 +26,12 @@ const MARKERLESS_PHASE_BAND_WIDTH = 3; const MARKERLESS_DEFAULT_CORNER_TILE_SCALE = 0.52; const MARKERLESS_DEFAULT_CORNER_TILE_MAX_SIDE_PX = 127; const RECTIFIED_PREVIEW_LONG_EDGE_PX = 2200; +// Per-frame mode resizes every rectified page to one common cell size before stacking them into a +// synthetic 1×N composite sheet. The chosen median cell dimensions are clamped to this range so a +// single oversized page cannot blow up the composite Mat. Phase 9 tightens this into a strict +// composite-area ceiling; this is the per-dimension guard. +const PER_FRAME_MIN_CELL_PX = 16; +const PER_FRAME_MAX_CELL_PX = 1600; const NEAR_IDENTITY_PAGE_AREA_PCT = 0.998; const NEAR_IDENTITY_CORNER_TOLERANCE_PX = 2; const NEAR_IDENTITY_DIM_TOLERANCE_PX = 2; @@ -246,7 +252,14 @@ function clampPositiveConvolutionToUint8(conv32, target8) { * rectifiedDownloadUsesRawSource: boolean * }} */ -export function runPipeline(sourceCanvas, config, requestId, throwIfAborted) { +export function runPipeline(sourceCanvas, config, requestId, throwIfAborted, images = null) { + // Per-frame mode does not rectify a single frame-sheet; it rectifies one page per uploaded image + // and stacks them into a synthetic 1×N composite. Dispatch before touching the single-page path. + // `images` is passed in by the caller so this module never reaches into `state` directly. + if (config.alignmentPipeline === "per-frame") { + return runPerFramePipeline(images || [], config, requestId, throwIfAborted); + } + const useMarkerlessAlignment = config.alignmentPipeline === "markerless"; let rectifiedWarp = null; let pageRectification = null; @@ -395,6 +408,183 @@ export function runPipeline(sourceCanvas, config, requestId, throwIfAborted) { } } +/** + * Per-frame alignment pipeline. + * + * Instead of one frame-sheet photo, the user uploads one image per animation frame. Each image is + * page-rectified independently (reusing {@link rectifySinglePage}), every rectified page is resized + * to a single common cell size, and the cells are stacked horizontally into a synthetic `1 × N` + * composite sheet. From that point on the rest of the app treats the composite exactly like a + * markerless frame-sheet, so stabilization / ordering / appearance / export need no per-frame + * branching. + * + * Ownership: the caller takes ownership of the returned `rectifiedMat` (the composite) and must + * delete it. Every other Mat allocated here — per-image rectified pages and per-cell resize buffers — + * is released internally, including on abort or failure. + * + * Each `rectifySinglePage` call is handed a shallow copy of `config` with the per-image fields + * overridden. `alignmentPipeline` is aliased to `"markerless"` for that copy because + * `rectifySinglePage` branches its near-identity fast path and grid rectification on that value; the + * base `config.alignmentPipeline` stays `"per-frame"` for the dispatcher and downstream UI gating. + * + * @param {Array<{canvas:HTMLCanvasElement, manualPageContour?:object|null, postRotationDeg?:number}>} images + * @param {object} config + * @param {number} requestId + * @param {(requestId:number) => void} throwIfAborted + * @returns {{ + * frames: HTMLCanvasElement[], + * rectifiedCanvas: HTMLCanvasElement, + * rectifiedMat: cv.Mat, + * pagePreviewCanvas: null, + * pagePreviewWidth: number, + * pagePreviewHeight: number, + * pagePreviewGridQuad: null, + * pagePreviewGridBounds: null, + * alignmentInfo: object, + * statusText: string, + * pageQuadPoints: null, + * pageQuadSource: "per-frame", + * rectifiedDownloadUsesRawSource: false + * }} + */ +function runPerFramePipeline(images, config, requestId, throwIfAborted) { + const entries = Array.isArray(images) ? images.filter((entry) => entry && entry.canvas) : []; + if (entries.length === 0) { + throw new Error("Per-frame mode requires at least one uploaded image."); + } + const interpolation = getCvInterpolationFlag(config.exportOptions.resampling); + /** @type {(cv.Mat | null)[]} Per-image styled (BGR) rectified pages, freed as they are composited. */ + const cellStyledMats = []; + let composite = null; + + try { + // 1-3. Rectify each uploaded page independently. Per-frame mode does not run per-image alignment, + // so the grayscale vision Mat is released immediately and only the styled (BGR) page is kept as + // the cell source. + for (const entry of entries) { + throwIfAborted(requestId); + const perImageConfig = { + ...config, + // rectifySinglePage branches its fast path / grid rectification on this value, so per-frame + // mode aliases to "markerless" here rather than passing "per-frame". + alignmentPipeline: "markerless", + // Per-image page-corner override (source-space quad). Phase 5 will populate this from the UI. + manualPageQuadPoints: entry.manualPageContour ?? null, + // No live threshold-preview fallback when rectifying per image. + fallbackPageQuadPoints: null, + // Per-image Post-Rotation. Phase 5 stores this on the image entry. + postRotationDeg: entry.postRotationDeg ?? 0, + }; + const rectification = rectifySinglePage(entry.canvas, perImageConfig, requestId, throwIfAborted); + const rectifiedWarp = rectification.rectifiedWarp; + rectifiedWarp.visionMat?.delete(); + rectifiedWarp.visionMat = null; + cellStyledMats.push(rectifiedWarp.styledMat); + } + throwIfAborted(requestId); + + // 4. Pick a single common cell size from the median rectified dimensions, clamped so one + // oversized page cannot blow up the composite Mat. + const frameCount = cellStyledMats.length; + const widths = cellStyledMats.map((mat) => mat.cols); + const heights = cellStyledMats.map((mat) => mat.rows); + const cellW = Math.round( + Math.max(PER_FRAME_MIN_CELL_PX, Math.min(PER_FRAME_MAX_CELL_PX, computeMedian(widths))) + ); + const cellH = Math.round( + Math.max(PER_FRAME_MIN_CELL_PX, Math.min(PER_FRAME_MAX_CELL_PX, computeMedian(heights))) + ); + + // 5-6. Resize each rectified page to the common cell size and copy it into its composite column. + // Each per-image styled Mat is freed immediately after it is consumed. + composite = new cv.Mat(cellH, cellW * frameCount, cv.CV_8UC3, new cv.Scalar(0, 0, 0)); + const cellSize = new cv.Size(cellW, cellH); + for (let i = 0; i < frameCount; i++) { + throwIfAborted(requestId); + const styledMat = cellStyledMats[i]; + const resized = new cv.Mat(); + try { + if (styledMat.cols === cellW && styledMat.rows === cellH) { + styledMat.copyTo(resized); + } else { + cv.resize(styledMat, resized, cellSize, 0, 0, interpolation); + } + const roi = composite.roi(new cv.Rect(i * cellW, 0, cellW, cellH)); + try { + resized.copyTo(roi); + } finally { + roi.delete(); + } + } finally { + resized.delete(); + styledMat.delete(); + cellStyledMats[i] = null; + } + } + throwIfAborted(requestId); + + // 7. Synthesize the 1×N corner lattice the downstream subsystems expect. The cell boundaries are + // known exactly (no detection needed), so the unrefined-region builder used by markerless mode is + // reused directly with regular corner intersections at the column boundaries. + const gridBounds = { left: 0, top: 0, width: composite.cols, height: composite.rows }; + const alignmentInfo = buildUnrefinedCrossRegionInfo( + composite, + frameCount, + 1, + "per-frame", + gridBounds, + config.crossRoiScale, + { markerType: "crosses", includeCornerCrosses: true } + ); + throwIfAborted(requestId); + + // 9. Extract each frame canvas from the composite using the same slicer as the other pipelines. + const frames = sliceRectifiedToCanvases( + composite, + alignmentInfo, + config.crop, + interpolation, + requestId, + throwIfAborted + ); + + // 8. Bounded preview canvas of the composite sheet for the Rectified Grid panel. + const rectifiedCanvas = matToPreviewCanvas(composite, RECTIFIED_PREVIEW_LONG_EDGE_PX); + + const statusText = [ + t("status.framesExtracted", { count: frames.length, expected: frameCount }), + t("status.rectifiedSheet", { width: composite.cols, height: composite.rows }), + t("status.animationSize", { width: frames[0]?.width || 0, height: frames[0]?.height || 0 }), + ].join("\n"); + + // 10. Hand the composite to the caller; clear our reference so the finally block does not free it. + const rectifiedMat = composite; + composite = null; + return { + frames, + rectifiedCanvas, + rectifiedMat, + pagePreviewCanvas: null, + pagePreviewWidth: 0, + pagePreviewHeight: 0, + pagePreviewGridQuad: null, + pagePreviewGridBounds: null, + alignmentInfo, + statusText, + pageQuadPoints: null, + pageQuadSource: "per-frame", + rectifiedDownloadUsesRawSource: false, + }; + } finally { + // Release any per-image styled Mats not yet consumed (abort/failure before they were composited). + for (const mat of cellStyledMats) { + mat?.delete?.(); + } + // If we threw after allocating the composite but before handing it off, release it here. + composite?.delete?.(); + } +} + /** * Detect, rectify, and Post-Rotate a single source page into a working sheet. * diff --git a/per_frame_pipeline_plan.md b/per_frame_pipeline_plan.md index ce39f75..aaa6e9c 100644 --- a/per_frame_pipeline_plan.md +++ b/per_frame_pipeline_plan.md @@ -344,6 +344,46 @@ testing. - UI strip (Phase 7). - Settings persistence (Phase 8). +**As built (Phase 3 — COMPLETE):** +- `js/pipeline.js`: + - Added `runPerFramePipeline(images, config, requestId, throwIfAborted)`. It filters the entries + to those with a `canvas`, throws if none, then per image calls `rectifySinglePage` with a shallow + `config` copy overriding `alignmentPipeline: "markerless"`, `manualPageQuadPoints: + entry.manualPageContour ?? null`, `fallbackPageQuadPoints: null`, `postRotationDeg: + entry.postRotationDeg ?? 0`. The grayscale `visionMat` is deleted immediately; only the styled + (BGR) page is kept as the cell source. + - Common cell size is `Math.round(clamp(median(dim), PER_FRAME_MIN_CELL_PX, + PER_FRAME_MAX_CELL_PX))` per dimension (new constants `16` / `1600`). The strict composite-area + ceiling and uniform scale-down remain deferred to Phase 9; this is the per-dimension guard. + - Each styled page is resized (or `copyTo` when already cell-sized) into its column of a + `cv.CV_8UC3` composite via `composite.roi(...)`, then freed right after it is consumed. The + `finally` block releases any still-unconsumed per-image Mats and the composite itself if the + function throws before handing it off. + - `alignmentInfo` is synthesized with `buildUnrefinedCrossRegionInfo(composite, N, 1, "per-frame", + fullBounds, config.crossRoiScale, { markerType: "crosses", includeCornerCrosses: true })` — i.e. + a 1×N markerless-style lattice with regular corner intersections at the column boundaries. + - Frames come from the existing `sliceRectifiedToCanvases`; the preview canvas from + `matToPreviewCanvas`. `statusText` is built from existing `status.framesExtracted` / + `status.rectifiedSheet` / `status.animationSize` keys (no new i18n strings). The return shape + matches `runPipeline` with `pagePreview*` null, `pageQuadSource: "per-frame"`, + `rectifiedDownloadUsesRawSource: false`. + - `runPipeline` gained an optional 5th param `images = null` and dispatches to + `runPerFramePipeline` when `config.alignmentPipeline === "per-frame"` before the single-page + path. It still does not import `state`; `images` is passed in by the caller. +- `js/app.js`: + - `readConfig` computes `perFrameModeActive = !!dom.alignmentPipelinePerFrame?.checked || + !!state.runtime.forcePerFrameMode`. The radio ref is read with optional chaining so this is + forward-compatible with the real radio added in Phase 6. `alignmentPipeline` emits `"per-frame"` + when active, and `useCrossAlignment` is forced `false` in that mode. + - The `runPipeline` invocation site now passes `state.source.images` as the 5th argument. +- `js/dom-state.js`: added `state.runtime.forcePerFrameMode = false` — a Phase 3 dev flag so the + pipeline can be driven from the console before the Phase 6 radio exists. Set it `true` (with images + loaded) and reprocess to exercise per-frame mode. +- No other `app.js` sites were touched (e.g. the markerless stabilization-warmup gating at the + `processCurrentImage` tail). Mode-gated downstream behavior — including auto-scheduling + stabilization for per-frame — is deferred to Phase 6's `=== "markerless"` audit, per this phase's + "readConfig + the runPipeline invocation site only" scope. + --- ### Phase 4 — Multi-file upload (`handleFile`, drop zone) From dca904098e7d5c4f54d8fdadca31813d6e74112e Mon Sep 17 00:00:00 2001 From: David Vandenbogaerde Date: Thu, 11 Jun 2026 18:15:45 +0200 Subject: [PATCH 06/21] phase 4 --- index.html | 2 +- js/app.js | 5 +- js/i18n.js | 39 ++++++++++------ js/load-controller.js | 96 ++++++++++++++++++++++++++++++++------ per_frame_pipeline_plan.md | 36 ++++++++++++++ 5 files changed, 150 insertions(+), 28 deletions(-) diff --git a/index.html b/index.html index 79a1962..bccae51 100644 --- a/index.html +++ b/index.html @@ -67,7 +67,7 @@

Photo

Animation frames should be separated by crosses, dots, or empty gutters. - + diff --git a/js/app.js b/js/app.js index 7e7750e..96c52ba 100644 --- a/js/app.js +++ b/js/app.js @@ -2822,15 +2822,18 @@ async function handleFile(file, files = null) { * @param {string} src * @param {string} [filename=""] * @param {string} [mimeType="image/jpeg"] + * @param {File | null} [settingsFile=null] + * @param {File[]} [additionalImageFiles=[]] Extra images beyond the primary, loaded as per-frame entries. * @returns {Promise} */ -async function loadImageSource(src, filename = "", mimeType = "image/jpeg", settingsFile = null) { +async function loadImageSource(src, filename = "", mimeType = "image/jpeg", settingsFile = null, additionalImageFiles = []) { clearDemoQueryIfLoadingDifferentFile(filename); await loadImageSourceViaController({ src, filename, mimeType, settingsFile, + additionalImageFiles, dom, state, setStatus, diff --git a/js/i18n.js b/js/i18n.js index aaf2cdd..7967a27 100644 --- a/js/i18n.js +++ b/js/i18n.js @@ -32,6 +32,7 @@ const LOCALES = { dropLeadMobile2: "or load a demo.", dropNote: "Animation frames should be separated\nby crosses, dots, or empty gutters.", dropNoteMarkerless: "Animation frames should be separated\nby crosses, dots, or empty gutters.", + dropNotePerFrame: "Drop one image per animation frame.\nThey'll be aligned into an animation.", }, layout: { summary: "Layout", @@ -226,7 +227,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "Loads a source photo or scan for processing.", loadDemoSelect: "Loads one of the bundled demo images listed in the demo manifest, along with its accompanying settings file.", - dropZone: "Drop a photo or scan of a frame-sheet here, or click to choose a file.", + dropZone: "Drop a photo or scan of a frame-sheet here, or click to choose a file. You can also drop several images at once — one per animation frame — for the per-frame pipeline.", layoutSummary: "Sets the frame-grid dimensions and paper format assumptions.", frameCols: "Number of animation frame columns in the plotted grid.", frameRows: "Number of animation frame rows in the plotted grid.", @@ -402,6 +403,7 @@ const LOCALES = { dropLeadMobile2: "o carga una demo.", dropNote: "Los fotogramas de animación deben estar separados\npor cruces, puntos o canales vacíos.", dropNoteMarkerless: "Los fotogramas de animación deben estar separados\npor cruces, puntos o canales vacíos.", + dropNotePerFrame: "Suelta una imagen por cada fotograma.\nSe alinearán en una animación.", }, layout: { summary: "Diseño", @@ -595,7 +597,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "Carga una foto o escaneo de origen para procesarlo.", loadDemoSelect: "Carga una de las imágenes de demostración incluidas en el manifiesto de demos, junto con su archivo de configuración correspondiente.", - dropZone: "Suelta aquí una foto o escaneo de una hoja de fotogramas, o haz clic para elegir un archivo.", + dropZone: "Suelta aquí una foto o escaneo de una hoja de fotogramas, o haz clic para elegir un archivo. También puedes soltar varias imágenes a la vez, una por fotograma, para la canalización por fotograma.", layoutSummary: "Configura las dimensiones de la cuadrícula de fotogramas y las suposiciones sobre el formato del papel.", frameCols: "Número de columnas de fotogramas en la cuadrícula animada.", frameRows: "Número de filas de fotogramas en la cuadrícula animada.", @@ -770,6 +772,7 @@ const LOCALES = { dropLeadMobile2: "oppure carica una demo.", dropNote: "I fotogrammi devono essere separati\nda croci, punti o spazi vuoti.", dropNoteMarkerless: "I fotogrammi devono essere separati\nda croci, punti o spazi vuoti.", + dropNotePerFrame: "Trascina un'immagine per ogni fotogramma.\nVerranno allineate in un'animazione.", }, layout: { summary: "Layout", @@ -963,7 +966,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "Carica una foto o scansione sorgente per l’elaborazione.", loadDemoSelect: "Carica una delle immagini demo incluse nel manifesto delle demo, insieme al relativo file di impostazioni.", - dropZone: "Trascina qui una foto o scansione di un foglio di fotogrammi, oppure fai clic per scegliere un file.", + dropZone: "Trascina qui una foto o scansione di un foglio di fotogrammi, oppure fai clic per scegliere un file. Puoi anche trascinare più immagini insieme, una per fotogramma, per la pipeline per-fotogramma.", layoutSummary: "Imposta le dimensioni della griglia dei fotogrammi e le ipotesi sul formato del foglio.", frameCols: "Numero di colonne di fotogrammi nella griglia animata.", frameRows: "Numero di righe di fotogrammi nella griglia animata.", @@ -1138,6 +1141,7 @@ const LOCALES = { dropLeadMobile2: "またはデモを読み込みます。", dropNote: "フレームは区切ってください\n十字、点、または空のガターで。", dropNoteMarkerless: "フレームは区切ってください\n十字、点、または空のガターで。", + dropNotePerFrame: "1フレームにつき1枚の画像をドロップ。\n自動で位置合わせしてアニメ化します。", }, layout: { summary: "レイアウト", @@ -1331,7 +1335,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "処理する元写真またはスキャン画像を読み込みます。", loadDemoSelect: "デモ一覧にある同梱デモ画像を、対応する設定ファイルと一緒に読み込みます。", - dropZone: "フレームシートの写真またはスキャン画像をここにドラッグするか、クリックしてファイルを選択します。", + dropZone: "フレームシートの写真またはスキャン画像をここにドラッグするか、クリックしてファイルを選択します。複数の画像（1フレームにつき1枚）をまとめてドロップして、フレームごとのパイプラインを使うこともできます。", layoutSummary: "フレームグリッドの寸法と用紙サイズの前提を設定します。", frameCols: "アニメーション用フレームグリッドの列数です。", frameRows: "アニメーション用フレームグリッドの行数です。", @@ -1506,6 +1510,7 @@ const LOCALES = { dropLeadMobile2: "或加载示例。", dropNote: "动画帧之间应分隔\n用十字、圆点或空白间隔。", dropNoteMarkerless: "动画帧之间应分隔\n用十字、圆点或空白间隔。", + dropNotePerFrame: "每帧拖入一张图片。\n它们将被对齐成动画。", }, layout: { summary: "布局", @@ -1699,7 +1704,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "加载源照片或扫描图像进行处理。", loadDemoSelect: "加载演示清单中包含的示例图像及其配套设置文件。", - dropZone: "将帧表照片或扫描图像拖到这里，或点击选择文件。", + dropZone: "将帧表照片或扫描图像拖到这里，或点击选择文件。你也可以一次拖入多张图片（每帧一张），以使用逐帧管线。", layoutSummary: "设置帧网格尺寸以及纸张格式假设。", frameCols: "动画帧网格中的列数。", frameRows: "动画帧网格中的行数。", @@ -1874,6 +1879,7 @@ const LOCALES = { dropLeadMobile2: "或載入示例。", dropNote: "動畫影格之間應分隔\n以十字、圓點或空白間隔。", dropNoteMarkerless: "動畫影格之間應分隔\n以十字、圓點或空白間隔。", + dropNotePerFrame: "每格拖入一張圖片。\n它們將被對齊成動畫。", }, layout: { summary: "版面", @@ -2067,7 +2073,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "載入來源照片或掃描影像進行處理。", loadDemoSelect: "載入示例清單中包含的範例影像，以及其對應的設定檔。", - dropZone: "將幀表照片或掃描影像拖到這裡，或點擊選擇檔案。", + dropZone: "將幀表照片或掃描影像拖到這裡，或點擊選擇檔案。你也可以一次拖入多張圖片（每格一張），以使用逐幀管線。", layoutSummary: "設定影格網格尺寸以及紙張格式假設。", frameCols: "動畫影格網格中的列數。", frameRows: "動畫影格網格中的行數。", @@ -2242,6 +2248,7 @@ const LOCALES = { dropLeadMobile2: "데모를 불러오세요.", dropNote: "애니메이션 프레임은 구분되어야 합니다\n십자, 점 또는 빈 간격으로.", dropNoteMarkerless: "애니메이션 프레임은 구분되어야 합니다\n십자, 점 또는 빈 간격으로.", + dropNotePerFrame: "프레임당 이미지 한 장을 드롭하세요.\n정렬되어 애니메이션이 됩니다.", }, layout: { summary: "레이아웃", @@ -2435,7 +2442,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "처리할 원본 사진 또는 스캔 이미지를 불러옵니다.", loadDemoSelect: "데모 목록에 포함된 샘플 이미지를 해당 설정 파일과 함께 불러옵니다.", - dropZone: "프레임 시트 사진 또는 스캔 이미지를 여기에 드래그하거나 클릭해 파일을 선택하세요.", + dropZone: "프레임 시트 사진 또는 스캔 이미지를 여기에 드래그하거나 클릭해 파일을 선택하세요. 여러 이미지를 한 번에(프레임당 한 장) 드롭하여 프레임별 파이프라인을 사용할 수도 있습니다.", layoutSummary: "프레임 그리드 크기와 용지 형식 가정을 설정합니다.", frameCols: "애니메이션 프레임 그리드의 열 수입니다.", frameRows: "애니메이션 프레임 그리드의 행 수입니다.", @@ -2610,6 +2617,7 @@ const LOCALES = { dropLeadMobile2: "ou carregue uma demo.", dropNote: "Os quadros da animação devem ser\nseparados por cruzes, pontos\nou espaços vazios.", dropNoteMarkerless: "Os quadros da animação devem ser\nseparados por cruzes, pontos\nou espaços vazios.", + dropNotePerFrame: "Solte uma imagem por quadro da animação.\nElas serão alinhadas em uma animação.", }, layout: { summary: "Layout", @@ -2803,7 +2811,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "Carrega uma foto ou digitalização de origem para processamento.", loadDemoSelect: "Carrega uma das imagens de demonstração incluídas no manifesto de demos, junto com seu arquivo de configurações correspondente.", - dropZone: "Solte aqui uma foto ou digitalização de uma folha de quadros, ou clique para escolher um arquivo.", + dropZone: "Solte aqui uma foto ou digitalização de uma folha de quadros, ou clique para escolher um arquivo. Você também pode soltar várias imagens de uma vez — uma por quadro — para o pipeline por quadro.", layoutSummary: "Define as dimensões da grade de quadros e as premissas sobre o formato do papel.", frameCols: "Número de colunas de quadros na grade animada.", frameRows: "Número de linhas de quadros na grade animada.", @@ -2978,6 +2986,7 @@ const LOCALES = { dropLeadMobile2: "oder laden Sie eine Demo.", dropNote: "Animationsbilder sollten getrennt sein\ndurch Kreuze, Punkte oder leere Zwischenräume.", dropNoteMarkerless: "Animationsbilder sollten getrennt sein\ndurch Kreuze, Punkte oder leere Zwischenräume.", + dropNotePerFrame: "Ein Bild pro Animationsbild ablegen.\nSie werden zu einer Animation ausgerichtet.", }, layout: { summary: "Layout", @@ -3171,7 +3180,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "Lädt ein Quellfoto oder einen Scan zur Verarbeitung.", loadDemoSelect: "Lädt eines der im Demo-Manifest aufgeführten Beispielbilder zusammen mit der dazugehörigen Einstellungsdatei.", - dropZone: "Ziehen Sie hier ein Foto oder einen Scan eines Frame-Sheets hinein oder klicken Sie, um eine Datei auszuwählen.", + dropZone: "Ziehen Sie hier ein Foto oder einen Scan eines Frame-Sheets hinein oder klicken Sie, um eine Datei auszuwählen. Sie können auch mehrere Bilder gleichzeitig ablegen – eines pro Animationsbild – für die Per-Frame-Pipeline.", layoutSummary: "Legt die Abmessungen des Frame-Rasters und die Annahmen zum Papierformat fest.", frameCols: "Anzahl der Spalten im Animationsraster.", frameRows: "Anzahl der Zeilen im Animationsraster.", @@ -3346,6 +3355,7 @@ const LOCALES = { dropLeadMobile2: "albo wczytaj demo.", dropNote: "Klatki animacji powinny być oddzielone\nkrzyżykami, kropkami lub pustymi odstępami.", dropNoteMarkerless: "Klatki animacji powinny być oddzielone\nkrzyżykami, kropkami lub pustymi odstępami.", + dropNotePerFrame: "Upuść jeden obraz na każdą klatkę.\nZostaną wyrównane w animację.", }, layout: { summary: "Układ", @@ -3539,7 +3549,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "Wczytuje zdjęcie lub skan źródłowy do przetwarzania.", loadDemoSelect: "Wczytuje jeden z dołączonych obrazów demo wymienionych w manifeście wraz z odpowiadającym mu plikiem ustawień.", - dropZone: "Upuść tutaj zdjęcie lub skan arkusza klatek albo kliknij, aby wybrać plik.", + dropZone: "Upuść tutaj zdjęcie lub skan arkusza klatek albo kliknij, aby wybrać plik. Możesz też upuścić kilka obrazów naraz — po jednym na klatkę — dla potoku klatka po klatce.", layoutSummary: "Ustawia wymiary siatki klatek i założenia dotyczące formatu papieru.", frameCols: "Liczba kolumn klatek w siatce animacji.", frameRows: "Liczba wierszy klatek w siatce animacji.", @@ -3714,6 +3724,7 @@ const LOCALES = { dropLeadMobile2: "eller last en demo.", dropNote: "Animasjonsruter bør være skilt\nmed kryss, prikker eller tomme mellomrom.", dropNoteMarkerless: "Animasjonsruter bør være skilt\nmed kryss, prikker eller tomme mellomrom.", + dropNotePerFrame: "Slipp ett bilde per animasjonsrute.\nDe justeres til en animasjon.", }, layout: { summary: "Oppsett", @@ -3907,7 +3918,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "Laster inn et kildebilde eller en skanning for behandling.", loadDemoSelect: "Laster inn ett av demo-bildene som er oppført i demo-manifestet, sammen med den tilhørende innstillingsfilen.", - dropZone: "Slipp et bilde eller en skanning av et frame-sheet her, eller klikk for å velge en fil.", + dropZone: "Slipp et bilde eller en skanning av et frame-sheet her, eller klikk for å velge en fil. Du kan også slippe flere bilder samtidig – ett per animasjonsrute – for per-rute-rørledningen.", layoutSummary: "Setter dimensjonene for rutenettet og antakelser om papirformatet.", frameCols: "Antall kolonner i animasjonsrutenettet.", frameRows: "Antall rader i animasjonsrutenettet.", @@ -4082,6 +4093,7 @@ const LOCALES = { dropLeadMobile2: "або завантажте демо.", dropNote: "Кадри анімації мають бути розділені\nхрестиками, крапками або порожніми проміжками.", dropNoteMarkerless: "Кадри анімації мають бути розділені\nхрестиками, крапками або порожніми проміжками.", + dropNotePerFrame: "Перетягніть по одному зображенню на кадр.\nВони вирівняються в анімацію.", }, layout: { summary: "Макет", @@ -4275,7 +4287,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "Завантажує вихідне фото або скан для обробки.", loadDemoSelect: "Завантажує одне з демо-зображень із маніфесту демонстрацій разом із відповідним файлом налаштувань.", - dropZone: "Перетягніть сюди фото або скан аркуша кадрів або клацніть, щоб вибрати файл.", + dropZone: "Перетягніть сюди фото або скан аркуша кадрів або клацніть, щоб вибрати файл. Ви також можете перетягнути кілька зображень одночасно — по одному на кадр — для покадрового конвеєра.", layoutSummary: "Налаштовує розміри сітки кадрів і припущення щодо формату паперу.", frameCols: "Кількість стовпців у сітці анімації.", frameRows: "Кількість рядків у сітці анімації.", @@ -4450,6 +4462,7 @@ const LOCALES = { dropLeadMobile2: "ou chargez une démo.", dropNote: "Les images d’animation doivent être séparées\npar des croix, des points ou des gouttières vides.", dropNoteMarkerless: "Les images d’animation doivent être séparées\npar des croix, des points ou des gouttières vides.", + dropNotePerFrame: "Déposez une image par image d’animation.\nElles seront alignées en une animation.", }, layout: { summary: "Mise en page", @@ -4643,7 +4656,7 @@ const LOCALES = { appLedeSecondary: "", photoHeading: "Charge une photo ou un scan source pour le traitement.", loadDemoSelect: "Charge l’une des images de démonstration listées dans le manifeste des démos, ainsi que son fichier de réglages associé.", - dropZone: "Déposez ici une photo ou un scan d’une feuille d’images, ou cliquez pour choisir un fichier.", + dropZone: "Déposez ici une photo ou un scan d’une feuille d’images, ou cliquez pour choisir un fichier. Vous pouvez aussi déposer plusieurs images à la fois — une par image d’animation — pour le pipeline image par image.", layoutSummary: "Règle les dimensions de la grille d’images et les hypothèses de format de papier.", frameCols: "Nombre de colonnes d’images dans la grille animée.", frameRows: "Nombre de lignes d’images dans la grille animée.", diff --git a/js/load-controller.js b/js/load-controller.js index 4bdab68..fa952f2 100644 --- a/js/load-controller.js +++ b/js/load-controller.js @@ -38,6 +38,24 @@ export async function waitForNextPaint() { await new Promise((resolve) => requestAnimationFrame(() => resolve())); } +/** + * Decode an image URL into a fully-loaded `HTMLImageElement`. + * + * Used by the multi-file (per-frame) load path to decode each additional uploaded image into its + * own entry. Rejects if the image fails to decode so the caller can skip just that file. + * + * @param {string} src + * @returns {Promise} + */ +function decodeImageElement(src) { + return new Promise((resolve, reject) => { + const image = new Image(); + image.onload = () => resolve(image); + image.onerror = () => reject(new Error("Failed to decode image")); + image.src = src; + }); +} + /** * Release any blob URLs the app currently owns for raw-photo drag/download behavior, including the * per-image source entries, so nothing leaks across image reloads. @@ -93,22 +111,28 @@ function getExpectedSettingsFilename(filename) { * @param {FileList | File[] | null} [files=null] * @param {{ * state: import("./dom-state.js").state, - * loadImageSource: (src:string, filename?:string, mimeType?:string, settingsFile?:File | null) => Promise, + * loadImageSource: (src:string, filename?:string, mimeType?:string, settingsFile?:File | null, additionalImageFiles?:File[]) => Promise, * applySettingsFile: (file: File) => Promise * }} deps * @returns {Promise} */ export async function handleFile(file, files = null, { state, loadImageSource, applySettingsFile }) { const allFiles = [...(files || [file])].filter(Boolean); - // A drag payload may contain an image plus its sibling settings file, or just a settings file. - // Prefer the image when present; otherwise treat a lone settings file as an override request. - const imageFile = allFiles.find(isImageFile) || (isImageFile(file) ? file : null); - if (imageFile) { + // A drag payload may contain one image, several images (one per animation frame), and/or a + // sibling settings file. Prefer images when present; otherwise treat a lone settings file as an + // override request. + const imageFiles = allFiles.filter(isImageFile); + const primaryImageFile = imageFiles[0] || (isImageFile(file) ? file : null); + if (primaryImageFile) { releaseOwnedSourceUrl(state); - const url = URL.createObjectURL(imageFile); - const settingsFilename = getExpectedSettingsFilename(imageFile.name || ""); + const url = URL.createObjectURL(primaryImageFile); + // A sibling settings file is matched against the first image's name and applied once. + const settingsFilename = getExpectedSettingsFilename(primaryImageFile.name || ""); const siblingSettingsFile = allFiles.find((candidate) => candidate && isSettingsFile(candidate) && candidate.name === settingsFilename) || null; - await loadImageSource(url, imageFile.name || "", imageFile.type || "image/jpeg", siblingSettingsFile); + // Any images beyond the first become additional per-frame entries; the loader switches into + // per-frame mode when more than one image is present so none are silently dropped. + const additionalImageFiles = imageFiles.length > 0 ? imageFiles.slice(1) : []; + await loadImageSource(url, primaryImageFile.name || "", primaryImageFile.type || "image/jpeg", siblingSettingsFile, additionalImageFiles); return; } @@ -126,6 +150,7 @@ export async function handleFile(file, files = null, { state, loadImageSource, a * filename?: string, * mimeType?: string, * settingsFile?: File | null, + * additionalImageFiles?: File[], * dom: import("./dom-state.js").dom, * state: import("./dom-state.js").state, * setStatus: (text:string) => void, @@ -152,6 +177,7 @@ export async function loadImageSource({ filename = "", mimeType = "image/jpeg", settingsFile = null, + additionalImageFiles = [], dom, state, setStatus, @@ -212,19 +238,63 @@ export async function loadImageSource({ syncRawPhotoHeadingLink?.(); syncRawPhotoCreditDisplay?.(); state.source.rawPageContour = null; - drawImageToCanvas(image, state.source.canvas); - // Register this image as the single per-image source entry. The legacy state.source.* fields - // continue to project this active entry; later phases grow images[] to more than one entry. + // Each per-frame entry owns a distinct source-resolution canvas so runPerFramePipeline can + // rectify a different image per cell. The legacy state.source.canvas / image fields project the + // active (index 0) entry, keeping single-image markers/markerless callers working unchanged. + const activeCanvas = document.createElement("canvas"); + drawImageToCanvas(image, activeCanvas); + state.source.canvas = activeCanvas; const sourceEntry = createSourceImageEntry({ image, filename: state.source.filename, mimeType: state.source.mimeType, ownedObjectUrl: state.source.ownedObjectUrl, dragUrl: state.source.dragUrl, - canvas: state.source.canvas, + canvas: activeCanvas, }); - state.source.images = [sourceEntry]; + const entries = [sourceEntry]; + // Decode any additional uploaded images into their own entries (multi-file / per-frame upload). + // Each extra image owns its own blob URL and canvas; a failed decode is skipped rather than + // aborting the whole load. + for (const extraFile of additionalImageFiles) { + if (!extraFile) continue; + const extraUrl = URL.createObjectURL(extraFile); + let extraImage; + try { + extraImage = await decodeImageElement(extraUrl); + } catch { + try { + URL.revokeObjectURL(extraUrl); + } catch { + /* already revoked */ + } + continue; + } + const extraCanvas = document.createElement("canvas"); + drawImageToCanvas(extraImage, extraCanvas); + entries.push( + createSourceImageEntry({ + image: extraImage, + filename: extraFile.name || "", + mimeType: extraFile.type || "image/jpeg", + ownedObjectUrl: extraUrl, + dragUrl: extraUrl, + canvas: extraCanvas, + }), + ); + } + state.source.images = entries; state.source.activeImageIndex = 0; + // Multiple uploaded images mean the user wants the per-frame pipeline (image count = frame + // count), so force it on and never silently drop the extras. The real radio arrives in Phase 6; + // until then state.runtime.forcePerFrameMode drives readConfig. Ticking the radio when present + // keeps this forward-compatible. + if (entries.length > 1) { + state.runtime.forcePerFrameMode = true; + if (dom.alignmentPipelinePerFrame) { + dom.alignmentPipelinePerFrame.checked = true; + } + } syncPaperPresetUi?.(); renderRawPreview(); const hasSettingsText = !!settingsText.trim(); diff --git a/per_frame_pipeline_plan.md b/per_frame_pipeline_plan.md index aaa6e9c..90f9e39 100644 --- a/per_frame_pipeline_plan.md +++ b/per_frame_pipeline_plan.md @@ -436,6 +436,42 @@ populate `state.source.images[]`; never silently drop additional images. - Reorder/delete buttons (Phase 7). - Settings-driven per-image overrides on reload (Phase 8 plus Phase 5). +**As built (Phase 4 — COMPLETE):** +- `js/load-controller.js`: + - Added a module-level `decodeImageElement(src)` helper that resolves a loaded + `HTMLImageElement` (rejects on decode failure) for the additional per-frame images. + - `handleFile` now collects **all** image files (`imageFiles = allFiles.filter(isImageFile)`), + treats the first as the primary, and passes the rest as `additionalImageFiles` + (`imageFiles.slice(1)`). The sibling `_settings.txt` is still matched against the **first** + image's expected name and applied once. A lone settings file still routes to `applySettingsFile`. + - `loadImageSource` gained an `additionalImageFiles = []` dep. In `image.onload` each entry now + gets its **own** dedicated source-resolution canvas (`document.createElement("canvas")` + + `drawImageToCanvas`), replacing the Phase 2 aliasing of the shared `state.source.canvas`. The + legacy `state.source.canvas` is repointed at the active (index 0) entry's canvas, and + `state.source.image` continues to project the primary image. Additional images are decoded in a + loop; each owns its own blob URL + canvas, and a failed decode revokes that URL and is skipped + (it does not abort the whole load). This is one code path for the 1-image and N-image cases. + - When more than one image is loaded, per-frame mode is forced on + (`state.runtime.forcePerFrameMode = true`, plus `dom.alignmentPipelinePerFrame.checked = true` + when that radio exists — forward-compatible with Phase 6). Single-image loads do **not** touch + the mode, so a fresh single drop behaves exactly as before. The activation happens before + settings application and `processCurrentImage`; because `readConfig` OR-s `forcePerFrameMode` + with the radio, multi-image always resolves to per-frame even if a sibling settings file selected + markers/markerless. + - Mat/URL lifetime is unchanged in spirit: `releaseOwnedSourceUrl` → `releaseAllSourceImages` + already revokes every per-entry `ownedObjectUrl` and frees per-entry canvases/Mats on the next + load, so the extra blob URLs do not leak. +- `js/app.js`: the wrapper `loadImageSource(src, filename, mimeType, settingsFile, + additionalImageFiles = [])` threads the new argument into `loadImageSourceViaController`. Demo + loads (2-arg calls) are unaffected (default `[]`, single image, no per-frame switch). +- `index.html`: `#fileInput` gained the `multiple` attribute so the file picker allows multi-select. +- `js/i18n.js`: added a `photo.dropNotePerFrame` string to **all 13** locale tables (per-frame + guidance copy, wired into the visible drop note in Phase 6) and extended the generic `dropZone` + tooltip in all 13 locales to mention dropping several images at once (one per frame). +- Not touched: `js/ui-controls.js` (the existing drop/`change` listeners already forward the full + `FileList`), the strip UI, reorder/delete, and any settings persistence — all deferred to later + phases. + --- ### Phase 5 — Per-image page-corner and post-rotation overrides From 4e5471c5d41bebfdac686d13f076b74d8e6ec6dd Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 11 Jun 2026 19:39:12 +0000 Subject: [PATCH 07/21] phase 5 --- js/app.js | 121 +++++++++++++++++++++++++++++++++++-- js/source-images.js | 41 +++++++++++++ js/ui-controls.js | 5 ++ per_frame_pipeline_plan.md | 66 ++++++++++++++++++++ 4 files changed, 227 insertions(+), 6 deletions(-) diff --git a/js/app.js b/js/app.js index 96c52ba..a9346e7 100644 --- a/js/app.js +++ b/js/app.js @@ -69,6 +69,11 @@ import { extractSingleFrameToCanvas, } from "./pipeline.js"; import { applyTranslations, getTooltipText, t } from "./i18n.js"; +import { + setActiveSourceImage, + setActiveManualPageContour, + setActivePostRotationDeg, +} from "./source-images.js"; // Final output-size scaling can be done either with browser canvas drawImage() or with OpenCV. // Keep both code paths available for comparison while evaluating tiny-output quality. const bUseOpenCvOutputScaling = true; @@ -1929,6 +1934,11 @@ function init() { attachResizeHandler(); startAnimationPreviewLoop(); void initializeMp4Support(); + + // Until the Phase 7 image strip exists, expose the active-image switch on a small dev handle so + // per-frame per-image overrides can be exercised from the console. Phase 7 will call setActiveImage + // directly from the strip wiring. + window.plottimation = Object.assign(window.plottimation || {}, { setActiveImage }); } /** @@ -2022,6 +2032,7 @@ function attachUi() { beginPostRotationScrub, endPostRotationScrub, finishPostRotationScrubIfUnchanged, + commitActivePostRotationFromSlider, bumpFrameOutputEpoch, setGeometryProcessingCursor, cancelInFlightProcessing, @@ -2528,6 +2539,38 @@ function finishPostRotationScrubIfUnchanged() { return true; } +/** + * Read the Post-Rotation slider value, clamped to the slider's range. + * + * @returns {number} + */ +function readPostRotationSliderDeg() { + return Math.max( + -1.1, + Math.min( + 1.1, + Number.isFinite(Number(dom.postRotation?.value)) + ? Number(dom.postRotation.value) + : SETTINGS_DEFAULTS.detection.postRotationDeg + ) + ); +} + +/** + * Persist the current Post-Rotation slider value onto the active per-frame image entry. + * + * In per-frame mode each image carries its own Post-Rotation; the global slider edits the active + * image's value so reprocessing rectifies that image with its own rotation. In markers / markerless + * mode this is a no-op (Post-Rotation stays a single global value read from `config.postRotationDeg`), + * so legacy behavior is unchanged. + * + * @returns {void} + */ +function commitActivePostRotationFromSlider() { + if (!isPerFrameModeActive()) return; + setActivePostRotationDeg(state, readPostRotationSliderDeg(), true); +} + /** * Populate the resampling dropdown with only the interpolation modes available in this OpenCV build. * @@ -2875,6 +2918,20 @@ function scheduleProcess(delayMs = 220) { }, Math.max(0, delayMs)); } +/** + * Report whether the per-frame alignment pipeline is currently active. + * + * Per-frame mode is selected by its own radio (added in Phase 6) or, until then, by a dev/multi-file + * flag (`state.runtime.forcePerFrameMode`). The radio ref is read with optional chaining so this is + * forward-compatible with Phase 6 wiring `dom.alignmentPipelinePerFrame`. This matches the detection + * `readConfig` uses, so per-image override routing and the active config stay consistent. + * + * @returns {boolean} + */ +function isPerFrameModeActive() { + return !!dom.alignmentPipelinePerFrame?.checked || !!state.runtime.forcePerFrameMode; +} + /** * Read the current UI state and normalize it into a processing/export config object. * @@ -2945,9 +3002,8 @@ function readConfig() { const frameRows = Math.max(1, Math.min(20, Math.round(Number(dom.frameRows.value) || SETTINGS_DEFAULTS.layout.frameRows))); const sourceFrameCount = Math.max(1, frameCols * frameRows); // Per-frame mode is selected by its own radio (added in Phase 6) or, until then, by a dev flag so - // the pipeline can be exercised before the UI exists. The radio ref is read with optional chaining - // so this stays correct both before and after Phase 6 wires `dom.alignmentPipelinePerFrame`. - const perFrameModeActive = !!dom.alignmentPipelinePerFrame?.checked || !!state.runtime.forcePerFrameMode; + // the pipeline can be exercised before the UI exists. See isPerFrameModeActive for details. + const perFrameModeActive = isPerFrameModeActive(); const encodingQuality = getEncodingQualityValue(); const readSearchInset = (input, fallback) => Math.max( 0, @@ -3563,7 +3619,9 @@ function togglePageCornerEditing() { */ function clearPageCornerEdits() { if (!Array.isArray(state.source.manualPageContour) || state.source.manualPageContour.length !== 4) return; - state.source.manualPageContour = null; + // Clearing the override drops it from the active image entry too in per-frame mode (legacy-only + // otherwise). Subsequent reprocessing re-detects this image's page automatically. + setActiveManualPageContour(state, null, isPerFrameModeActive()); state.preview.activePageCornerDrag = null; state.runtime.pageCornerEditingEnabled = false; cancelPageCornerOverrideDependentWork(); @@ -7203,7 +7261,9 @@ function seedDefaultManualPageContour() { { x: right, y: bottom }, { x: inset, y: bottom }, ]; - state.source.manualPageContour = contour.map((point) => ({ x: point.x, y: point.y })); + // Seeding a default editable quad is a manual override, so mirror it to the active image entry in + // per-frame mode (legacy-only otherwise). + setActiveManualPageContour(state, contour.map((point) => ({ x: point.x, y: point.y })), isPerFrameModeActive()); state.source.rawPageContour = contour.map((point) => ({ x: point.x, y: point.y })); state.source.pageQuadSource = "manual-override"; state.source.thresholdPreviewPageContour = null; @@ -7297,7 +7357,9 @@ function updateManualPageCorner(index, point) { if (!Array.isArray(baseContour) || baseContour.length !== 4) return; const nextContour = baseContour.map((corner) => ({ x: corner.x, y: corner.y })); nextContour[index] = point; - state.source.manualPageContour = nextContour; + // In per-frame mode this also stores the override on the active image entry; in markers/markerless + // it only writes the legacy field, exactly as before. + setActiveManualPageContour(state, nextContour, isPerFrameModeActive()); state.source.rawPageContour = nextContour.map((corner) => ({ x: corner.x, y: corner.y })); state.source.pageQuadSource = "manual-override"; } @@ -7374,6 +7436,53 @@ function attachRawPageCornerEditing() { dom.rawCanvas.addEventListener("pointercancel", finishDrag); } +/** + * Switch the active per-frame image and redraw the raw photo, Page Corners overlay, and + * Post-Rotation slider to match it — without rebuilding the rectified preview or animation. + * + * Active-image switching is UI navigation, not a config change: each image's page-corner override and + * Post-Rotation are restored from its entry so the editor operates on the newly active image, but no + * reprocessing is scheduled (the existing composite stays live until a real config change). In + * markers / markerless modes there is at most one entry, so this just refreshes the raw preview. + * + * Until the Phase 7 image strip exists this is the supported active-image switch entry point (the + * dev console can call `window.plottimation.setActiveImage(1)` or set + * `state.source.activeImageIndex` and call `renderRawPreview()` directly). + * + * @param {number} index Zero-based image index (clamped to the loaded range). + * @returns {void} + */ +function setActiveImage(index) { + const entry = setActiveSourceImage(state, index); + if (!entry) return; + // Repoint the legacy projections at the newly active entry so every existing reader (raw preview, + // Page Corners editor, drag/download, status) sees this image. + if (entry.canvas) state.source.canvas = entry.canvas; + if (entry.image) state.source.image = entry.image; + state.source.filename = entry.filename || ""; + state.source.mimeType = entry.mimeType || ""; + state.source.dragUrl = entry.dragUrl || ""; + // Restore this image's manual page-corner override into the legacy field and the overlay contour. + // A live threshold preview belongs to whichever image was last reprocessed, so drop it on switch. + const contour = Array.isArray(entry.manualPageContour) && entry.manualPageContour.length === 4 + ? entry.manualPageContour.map((point) => ({ x: point.x, y: point.y })) + : null; + state.source.manualPageContour = contour; + state.source.rawPageContour = contour ? contour.map((point) => ({ x: point.x, y: point.y })) : null; + state.source.pageQuadSource = contour ? "manual-override" : ""; + state.source.thresholdPreviewPageContour = null; + state.source.thresholdPreviewSignature = ""; + state.preview.activePageCornerDrag = null; + // Restore this image's Post-Rotation onto the slider so the control reflects the active image. + if (dom.postRotation) { + dom.postRotation.value = String(Number.isFinite(entry.postRotationDeg) ? entry.postRotationDeg : 0); + updateSliderReadouts(); + } + syncRawPhotoHeadingLink(); + syncRawPhotoCreditDisplay(); + renderRawPreview(); +} + /** * Render the Page Corners preview and overlay the detected page quad in lime. * diff --git a/js/source-images.js b/js/source-images.js index f445a57..adfa2f9 100644 --- a/js/source-images.js +++ b/js/source-images.js @@ -77,6 +77,47 @@ export function setActiveSourceImage(state, index) { return images[clamped]; } +/** + * Store a manual page-corner override for the active image. + * + * The legacy `state.source.manualPageContour` field is the authoritative input for page detection in + * markers / markerless modes and is also read by many UI sites (status text, settings save, overlay + * draw). In per-frame mode the canonical per-image override lives on the active entry, so this helper + * writes the active entry's `manualPageContour` **and** mirrors to the legacy field so those legacy + * read sites keep working without per-frame-specific branching. In markers / markerless mode it only + * writes the legacy field (the per-image array is irrelevant there). + * + * @param {import("./dom-state.js").state} state + * @param {Array<{x:number,y:number}> | null} contour Source-space page quad override, or `null` to clear. + * @param {boolean} perFrameMode Whether the per-frame pipeline is currently active. + * @returns {void} + */ +export function setActiveManualPageContour(state, contour, perFrameMode) { + state.source.manualPageContour = contour; + if (perFrameMode) { + const active = getActiveSourceImage(state); + if (active) active.manualPageContour = contour; + } +} + +/** + * Store the Post-Rotation value (degrees) for the active image. + * + * In per-frame mode each image carries its own Post-Rotation, so this writes the active entry's + * `postRotationDeg`. In markers / markerless mode Post-Rotation is a single global value applied by + * the pipeline from `config.postRotationDeg`, so the per-image array is left untouched. + * + * @param {import("./dom-state.js").state} state + * @param {number} deg + * @param {boolean} perFrameMode Whether the per-frame pipeline is currently active. + * @returns {void} + */ +export function setActivePostRotationDeg(state, deg, perFrameMode) { + if (!perFrameMode) return; + const active = getActiveSourceImage(state); + if (active) active.postRotationDeg = Number.isFinite(deg) ? deg : 0; +} + /** * Release any cached rectified Mat held on a per-image entry. * diff --git a/js/ui-controls.js b/js/ui-controls.js index 4c0236d..41389b5 100644 --- a/js/ui-controls.js +++ b/js/ui-controls.js @@ -581,6 +581,7 @@ function attachMarkerlessPhaseMetricToggles({ * beginPostRotationScrub: () => void, * endPostRotationScrub: () => void, * finishPostRotationScrubIfUnchanged: () => boolean, + * commitActivePostRotationFromSlider: () => void, * bumpFrameOutputEpoch: () => void, * setGeometryProcessingCursor: (active:boolean) => void, * cancelInFlightProcessing: () => void, @@ -651,6 +652,7 @@ export function attachUi({ beginPostRotationScrub, endPostRotationScrub, finishPostRotationScrubIfUnchanged, + commitActivePostRotationFromSlider, bumpFrameOutputEpoch, cancelInFlightProcessing, invalidateFrameCaches, @@ -989,6 +991,9 @@ export function attachUi({ revokeGifUrl(); updateSliderReadouts(); if (finishPostRotationScrubIfUnchanged()) return; + // Per-frame mode stores Post-Rotation per image, so persist the slider value onto the active + // image before reprocessing. No-op (and unchanged behavior) in markers / markerless mode. + commitActivePostRotationFromSlider?.(); scheduleProcess(); }); } diff --git a/per_frame_pipeline_plan.md b/per_frame_pipeline_plan.md index 90f9e39..e45ce66 100644 --- a/per_frame_pipeline_plan.md +++ b/per_frame_pipeline_plan.md @@ -515,6 +515,72 @@ on the *active* image in per-frame mode. dev console (`state.source.activeImageIndex = 1; renderRawPreview();`). - Persisting overrides to disk (Phase 8). +**As built (Phase 5 — COMPLETE):** +- The per-image accessors this phase needs (`setActiveManualPageContour(state, + contour, perFrameMode)` and `setActivePostRotationDeg(state, deg, perFrameMode)`) + already live in `js/source-images.js` — they take an explicit `perFrameMode` + boolean rather than re-deriving the mode inside the module, so `source-images.js` + stays DOM-free. `setActiveManualPageContour` always writes the legacy + `state.source.manualPageContour` and, only when `perFrameMode`, also mirrors to + the active entry's `manualPageContour`. `setActivePostRotationDeg` writes **only** + the active entry's `postRotationDeg` and is a no-op outside per-frame mode (the + legacy global stays the slider/`config.postRotationDeg`). No new accessors were + needed. +- `js/app.js`: + - New `isPerFrameModeActive()` helper centralizes mode detection as + `!!dom.alignmentPipelinePerFrame?.checked || !!state.runtime.forcePerFrameMode` + (optional-chained for the not-yet-existing Phase 6 radio). `readConfig` now + calls it instead of inlining the same expression, so config emission and + override routing can never diverge. + - The three user-driven manual page-corner write sites now route through + `setActiveManualPageContour(state, …, isPerFrameModeActive())`: + `updateManualPageCorner` (corner drag), `seedDefaultManualPageContour` (the + inset-rectangle seed when detection fails), and `clearPageCornerEdits` (the + `null` clear). Each preserves its existing `rawPageContour` / `pageQuadSource` + bookkeeping verbatim. The `clearAllPreviews` reset (start-of-load) was left as a + direct legacy `= null` on purpose: it is a global preview reset that runs while + the `images[]` array is about to be rebuilt, not a per-image edit. + - Post-Rotation: `readPostRotationSliderDeg()` (clamped slider read, factored from + the existing duplicated clamp) and `commitActivePostRotationFromSlider()` were + added. The latter writes the slider value onto the active entry **only** in + per-frame mode and is wired into the Post-Rotation slider's `change` handler in + `ui-controls.js` (before `scheduleProcess()`, after the unchanged-scrub early + return). In markers/markerless mode it early-returns, so the legacy slider → + `config.postRotationDeg` → pipeline path is byte-for-byte unchanged. + - New `setActiveImage(index)` performs the active-image switch as **UI + navigation, not a config change**: it calls `setActiveSourceImage`, repoints the + legacy projections (`canvas`, `image`, `filename`, `mimeType`, `dragUrl`) at the + new entry, restores that entry's `manualPageContour` into the legacy field + + overlay `rawPageContour` (or clears them), drops any stale live threshold + preview, restores the entry's `postRotationDeg` onto the slider + (`dom.postRotation.value` + `updateSliderReadouts()`), refreshes the raw-photo + heading/credit, and calls `renderRawPreview()`. It deliberately does **not** + call `scheduleProcess()` — the existing composite/animation stays live until a + real config change. It is exposed as `window.plottimation.setActiveImage` so it + can be exercised before the Phase 7 strip exists (the dev-console + `activeImageIndex = …; renderRawPreview()` route still works but won't restore + the slider/contour; prefer `setActiveImage`). +- `js/ui-controls.js`: added `commitActivePostRotationFromSlider` to the `attachUi` + deps (JSDoc + destructure) and called it in the Post-Rotation `change` handler. + No other listeners changed. +- `js/dom-state.js`: **not touched** — the per-image `manualPageContour` / + `postRotationDeg` fields and `state.runtime.forcePerFrameMode` already exist from + Phases 2–4, so the plan's listing of `dom-state.js` for Phase 5 was a no-op. +- No Mats allocated, no i18n strings added, Page Detection Threshold stayed global. +- **Notes for later phases:** (6) Phase 6 makes `dom.alignmentPipelinePerFrame` + real; `isPerFrameModeActive()` already prefers it, so no app.js change is needed + there for detection. (7) The strip should call `setActiveImage(index)` on + thumbnail click (do not poke `state.source.activeImageIndex` directly — that skips + the slider/contour restore); after reorder/delete it should re-derive the active + index and call `setActiveImage` before reprocessing. (8) Settings load must + populate each entry's `manualPageContour` / `postRotationDeg`; for the active + entry it should also refresh the legacy field + slider (or just call + `setActiveImage(activeImageIndex)` afterwards) so the editor reflects the restored + values. The post-rotation scrub *preview* (Panel 3 live rotation) is still + markerless-gated (`config.alignmentPipeline !== "markerless"` skips it), so in + per-frame mode the slider commits on `change` but does not show a live scrub + preview — wiring that for per-frame is left as later polish. + --- ### Phase 6 — `alignmentPipeline` radio: add "per-frame" From 1b0e85082e291a9b72bd00dc6909125d3efcebcc Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 11 Jun 2026 20:00:42 +0000 Subject: [PATCH 08/21] phase 6 --- index.html | 4 + js/app.js | 197 ++++++++++++++++++++++++------------- js/dom-state.js | 1 + js/i18n.js | 27 +++++ js/settings-defaults.js | 3 + js/ui-controls.js | 44 ++++++--- per_frame_pipeline_plan.md | 130 ++++++++++++++++++++++++ style.css | 4 +- 8 files changed, 329 insertions(+), 81 deletions(-) diff --git a/index.html b/index.html index bccae51..ecb203c 100644 --- a/index.html +++ b/index.html @@ -82,6 +82,10 @@

Photo

Markers (crosses, dots) + diff --git a/js/app.js b/js/app.js index a9346e7..6a231d9 100644 --- a/js/app.js +++ b/js/app.js @@ -2095,7 +2095,9 @@ async function initializeMp4Support() { */ function applyManualMarkerOverrides(alignmentInfo) { if (!alignmentInfo) return; - if (readConfig().alignmentPipeline === "markerless") return; + // Non-marker pipelines (markerless + per-frame) interpret corner overrides as post-stabilization + // nudges, not in-place marker patches, so this marker-only fast path is skipped for both. + if (readConfig().alignmentPipeline !== "markers") return; for (const [key, override] of state.geometry.manualMarkerOverrides.entries()) { const marker = alignmentInfo.markerLookup.get(key); const tile = alignmentInfo.crossRoiTileMap?.get(key); @@ -2276,6 +2278,9 @@ function resetExportControls() { */ function resetNonLayoutControls() { applyNonLayoutDefaults(dom); + // `applyNonLayoutDefaults` restores the pipeline radio to the default ("markers"); clear the + // legacy per-frame force shim too so `isPerFrameModeActive()` agrees with the reset radio. + state.runtime.forcePerFrameMode = false; state.geometry.manualMarkerOverrides.clear(); state.preview.activeEditedMarker = null; state.runtime.markerEditingEnabled = false; @@ -2396,7 +2401,8 @@ function scheduleStabilizationPreviewUpdate() { * @returns {void} */ function beginStabilizationStrengthScrub() { - if (readConfig().alignmentPipeline === "markerless") { + // Stabilization is available in markerless and per-frame; warm the solver for both non-marker modes. + if (readConfig().alignmentPipeline !== "markers") { scheduleCurrentStabilizationWarmup(); } if (state.preview.stabilizationStrengthScrubbing) return; @@ -3260,13 +3266,14 @@ function syncPaperPresetUi() { } function getActiveAlignmentPipeline() { + if (isPerFrameModeActive()) return "per-frame"; return dom.alignmentPipelineMarkerless.checked ? "markerless" : "markers"; } /** * Preserve the markerless default of zero Grid Search Inset X/Y when switching away from marker mode. * - * @param {"markerless"|"markers"} pipeline + * @param {"markerless"|"markers"|"per-frame"} pipeline * @returns {void} */ function applyAlignmentPipelineDefaults(pipeline) { @@ -3286,7 +3293,7 @@ function applyAlignmentPipelineDefaults(pipeline) { /** * Ensure marker-pipeline-only controls hold a valid state before being shown again. * - * @param {"markerless"|"markers"} pipeline + * @param {"markerless"|"markers"|"per-frame"} pipeline * @returns {void} */ function sanitizeAlignmentPipelineState(pipeline) { @@ -3303,31 +3310,48 @@ function sanitizeAlignmentPipelineState(pipeline) { /** * Return which alignment-specific control groups should be visible for the active pipeline. * - * @param {"markerless"|"markers"} pipeline - * @returns {{showMarkerlessControls:boolean,showMarkersPipelineControls:boolean,showCrossOnlyControls:boolean}} + * `showMarkerlessControls` stays strictly markerless-only (markerless gutter/phase/working-image + * diagnostics must not leak into per-frame mode). `showFrameCornerControls` is the broader + * "non-marker" family (markerless + per-frame) that shares stabilization, drift compensation and the + * Frame Corners override editor. `isPerFrame` lets callers hide markerless-only controls that the + * non-marker family would otherwise reveal (e.g. gutter sliders, Grid Edge controls). + * + * @param {"markerless"|"markers"|"per-frame"} pipeline + * @returns {{showMarkerlessControls:boolean,showMarkersPipelineControls:boolean,showCrossOnlyControls:boolean,showFrameCornerControls:boolean,isPerFrame:boolean}} */ function getAlignmentUiModeFlags(pipeline) { document.body.classList.toggle("markerless-pipeline", pipeline === "markerless"); + document.body.classList.toggle("per-frame-pipeline", pipeline === "per-frame"); const markerType = dom.alignmentMarkerType.value || SETTINGS_DEFAULTS.detection.alignmentMarkerType; const resolvedAutoType = state.geometry.alignmentInfo?.resolvedMarkerType || null; const showMarkersPipelineControls = pipeline === "markers"; const showCrossOnlyControls = showMarkersPipelineControls && (markerType === "crosses" || (markerType === "auto" && resolvedAutoType === "crosses")); const showMarkerlessControls = pipeline === "markerless"; + const isPerFrame = pipeline === "per-frame"; + const showFrameCornerControls = pipeline !== "markers"; return { showMarkerlessControls, showMarkersPipelineControls, showCrossOnlyControls, + showFrameCornerControls, + isPerFrame, }; } /** * Rewrite alignment-related labels so the UI language matches the active pipeline. * - * @param {{showMarkerlessControls:boolean}} flags + * Per-frame mode shares the corner/stabilization mental model with markerless mode, so the + * non-marker label family (`showFrameCornerControls`) drives the Frame Corners heading, the + * stabilize/centers viewer tabs and the ROI-size label. `showMarkerlessControls` is reserved for + * the markerless-only "summaryMarkerless" copy that mentions gutter fitting, which does not apply + * to per-frame's synthetic exact grid. + * + * @param {{showMarkerlessControls:boolean,showFrameCornerControls:boolean,isPerFrame:boolean}} flags * @returns {void} */ function syncAlignmentPipelineLabels(flags) { - const { showMarkerlessControls } = flags; + const { showMarkerlessControls, showFrameCornerControls, isPerFrame } = flags; const frameAlignmentSummary = document.querySelector("#frameAlignmentSummary"); const frameAlignmentSummaryLabel = frameAlignmentSummary?.querySelector("[data-i18n='alignment.summary']") || @@ -3337,26 +3361,26 @@ function syncAlignmentPipelineLabels(flags) { frameAlignmentSummaryLabel.textContent = showMarkerlessControls ? t("alignment.summaryMarkerless") : t("alignment.summary"); } if (dropGuidanceNote) { - dropGuidanceNote.textContent = t("photo.dropNote"); + dropGuidanceNote.textContent = isPerFrame ? t("photo.dropNotePerFrame") : t("photo.dropNote"); } const isMobileViewerMode = state.runtime.mobileSingleViewerMode; const headingText = isMobileViewerMode - ? t(showMarkerlessControls ? "viewerTabs.centers" : "viewerTabs.markers") - : t(showMarkerlessControls ? "panels.frameCorners" : "panels.frameAlignmentMarkers"); + ? t(showFrameCornerControls ? "viewerTabs.centers" : "viewerTabs.markers") + : t(showFrameCornerControls ? "panels.frameCorners" : "panels.frameAlignmentMarkers"); if (dom.crossRegionsHeading) { const label = dom.crossRegionsHeading.querySelector("[data-panel-heading]") || dom.crossRegionsHeading.firstElementChild; if (label) label.textContent = headingText; } if (dom.viewerTabMarkers) { - const viewerTabKey = showMarkerlessControls ? "viewerTabs.centers" : "viewerTabs.markers"; + const viewerTabKey = showFrameCornerControls ? "viewerTabs.centers" : "viewerTabs.markers"; dom.viewerTabMarkers.textContent = t(viewerTabKey); } if (dom.mobileControlTabAlignment) { - const mobileControlTabKey = showMarkerlessControls ? "mobileControlTabs.stabilize" : "mobileControlTabs.markers"; + const mobileControlTabKey = showFrameCornerControls ? "mobileControlTabs.stabilize" : "mobileControlTabs.markers"; dom.mobileControlTabAlignment.textContent = t(mobileControlTabKey); } if (dom.crossRoiScaleLabel) { - dom.crossRoiScaleLabel.textContent = showMarkerlessControls + dom.crossRoiScaleLabel.textContent = showFrameCornerControls ? t("alignment.frameCornerRoiSize") : t("alignment.roiSize"); } @@ -3372,17 +3396,17 @@ function syncAlignmentPipelineLabels(flags) { if (markerlessPhaseYLabel) { markerlessPhaseYLabel.textContent = t("alignment.markerlessPhaseYOffset"); } - syncAlignmentModeTooltips(showMarkerlessControls); + syncAlignmentModeTooltips(showFrameCornerControls); } /** * Keep shared tooltip text aligned with the active pipeline when marker terminology becomes - * corner/stabilization terminology in markerless mode. + * corner/stabilization terminology in the non-marker (markerless / per-frame) pipelines. * - * @param {boolean} showMarkerlessControls + * @param {boolean} showFrameCornerControls * @returns {void} */ -function syncAlignmentModeTooltips(showMarkerlessControls) { +function syncAlignmentModeTooltips(showFrameCornerControls) { const applyTooltip = (element, key, extraElements = []) => { if (!element) return; const text = t(`tooltip.${key}`); @@ -3403,37 +3427,40 @@ function syncAlignmentModeTooltips(showMarkerlessControls) { applyTooltip( document.querySelector("#frameAlignmentSummary"), - showMarkerlessControls ? "frameAlignmentSummaryMarkerless" : "frameAlignmentSummary", + showFrameCornerControls ? "frameAlignmentSummaryMarkerless" : "frameAlignmentSummary", ); applyTooltip( dom.crossRegionsHeading, - showMarkerlessControls ? "crossRegionsHeadingMarkerless" : "crossRegionsHeading", + showFrameCornerControls ? "crossRegionsHeadingMarkerless" : "crossRegionsHeading", ); applyTooltip( dom.crossRoiScale, - showMarkerlessControls ? "crossRoiScaleMarkerless" : "crossRoiScale", + showFrameCornerControls ? "crossRoiScaleMarkerless" : "crossRoiScale", [dom.crossRoiScale?.closest("label")], ); applyTooltip( dom.toggleMarkerEditingButton, - showMarkerlessControls ? "toggleMarkerEditingButtonMarkerless" : "toggleMarkerEditingButton", + showFrameCornerControls ? "toggleMarkerEditingButtonMarkerless" : "toggleMarkerEditingButton", ); applyTooltip( dom.clearMarkerEditsButton, - showMarkerlessControls ? "clearMarkerEditsButtonMarkerless" : "clearMarkerEditsButton", + showFrameCornerControls ? "clearMarkerEditsButtonMarkerless" : "clearMarkerEditsButton", ); } /** * Keep the shared ROI-size slider in the right position for the active alignment mode. * - * @param {{showMarkerlessControls:boolean}} flags + * Per-frame mode shares the markerless Frame Corners layout, so the corner ROI slider sits at the + * bottom of the stack for both non-marker pipelines. + * + * @param {{showFrameCornerControls:boolean}} flags * @returns {void} */ function syncAlignmentSliderOrder(flags) { - const { showMarkerlessControls } = flags; + const { showFrameCornerControls } = flags; if (dom.alignmentSliderStack && dom.crossRoiScaleRow) { - if (showMarkerlessControls) { + if (showFrameCornerControls) { dom.alignmentSliderStack.appendChild(dom.crossRoiScaleRow); } else { dom.alignmentSliderStack.prepend(dom.crossRoiScaleRow); @@ -3444,27 +3471,38 @@ function syncAlignmentSliderOrder(flags) { /** * Show or hide the alignment controls appropriate for the active pipeline. * - * @param {{showMarkerlessControls:boolean,showMarkersPipelineControls:boolean,showCrossOnlyControls:boolean}} flags + * Three control families: + * - Stabilization + Vertical Drift Compensation are the shared non-marker family + * (`showFrameCornerControls`): visible in markerless AND per-frame. + * - Grid Edge Threshold / Run Length are marker-only grid controls: shown only in markers + * (`showMarkersPipelineControls`), so they are hidden in BOTH markerless and per-frame. + * - Markerless gutter/phase sliders (`markerlessPhase*Row`) stay markerless-only + * (`showMarkerlessControls`): hidden in per-frame because the synthetic grid needs no phase sweep. + * + * @param {{showMarkerlessControls:boolean,showMarkersPipelineControls:boolean,showCrossOnlyControls:boolean,showFrameCornerControls:boolean}} flags * @returns {void} */ function syncAlignmentPipelineVisibility(flags) { - const { showMarkerlessControls, showMarkersPipelineControls, showCrossOnlyControls } = flags; - dom.boundarySensitivityRow.hidden = showMarkerlessControls; - dom.boundaryPersistenceRow.hidden = showMarkerlessControls; + const { showMarkerlessControls, showMarkersPipelineControls, showCrossOnlyControls, showFrameCornerControls } = flags; + // Grid Edge Threshold / Run Length are marker-grid controls: keep visible only in markers mode so + // per-frame (which has no detected grid edges) hides them alongside markerless. + dom.boundarySensitivityRow.hidden = !showMarkersPipelineControls; + dom.boundaryPersistenceRow.hidden = !showMarkersPipelineControls; if (dom.stabilizationMethodGroup) { - dom.stabilizationMethodGroup.hidden = !showMarkerlessControls; + dom.stabilizationMethodGroup.hidden = !showFrameCornerControls; } if (dom.stabilizationEnabledRow) { - dom.stabilizationEnabledRow.hidden = !showMarkerlessControls; + dom.stabilizationEnabledRow.hidden = !showFrameCornerControls; } if (dom.stabilizationStrengthRow) { - dom.stabilizationStrengthRow.hidden = !showMarkerlessControls; + dom.stabilizationStrengthRow.hidden = !showFrameCornerControls; } dom.alignmentMarkerTypeField.hidden = !showMarkersPipelineControls; // Keep marker subpixel alignment as a settings-file/default option, not a visible UI control. dom.useCrossAlignmentRow.hidden = true; dom.detectCrossesWithConvolutionRow.hidden = !showCrossOnlyControls; - dom.stabilizationLambdaRow.hidden = !showMarkerlessControls; + dom.stabilizationLambdaRow.hidden = !showFrameCornerControls; + // Markerless gutter/phase sweep sliders remain markerless-only; per-frame's exact grid skips them. dom.markerlessPhaseXRow.hidden = !showMarkerlessControls; dom.markerlessPhaseYRow.hidden = !showMarkerlessControls; if (dom.markerlessPhaseDebugRow) { @@ -3479,23 +3517,24 @@ function syncAlignmentPipelineVisibility(flags) { if (dom.markerlessUseVarianceRow) { dom.markerlessUseVarianceRow.hidden = true; } - dom.verticalDriftCompensationRow.hidden = !showMarkerlessControls; + dom.verticalDriftCompensationRow.hidden = !showFrameCornerControls; } /** - * Enable only the stabilization controls that apply to the current markerless method. + * Enable only the stabilization controls that apply to the current method. * * `Stabilization Rigidity` (`lambda`) only affects the pairwise/cyclic least-squares solve. The * alternate average-reference method does not use it, so the slider should be visibly inactive in - * that mode to avoid implying that it has any effect. + * that mode to avoid implying that it has any effect. Per-frame mode runs the same stabilization + * solver as markerless, so it shares this gating via `showFrameCornerControls`. * - * @param {{showMarkerlessControls:boolean}} flags + * @param {{showFrameCornerControls:boolean}} flags * @returns {void} */ function syncStabilizationMethodUi(flags) { - const { showMarkerlessControls } = flags; + const { showFrameCornerControls } = flags; const usesLambda = - showMarkerlessControls && + showFrameCornerControls && (dom.stabilizationMethodPairwise?.checked || !dom.stabilizationMethodAverage?.checked); if (dom.stabilizationLambda) { dom.stabilizationLambda.disabled = !usesLambda; @@ -3705,8 +3744,10 @@ function clearMarkerEdits() { state.geometry.manualMarkerOverrides.clear(); state.preview.activeEditedMarker = null; state.runtime.markerEditingEnabled = false; - const isMarkerless = readConfig().alignmentPipeline === "markerless"; - if (state.geometry.alignmentInfo && !isMarkerless) { + // Non-marker pipelines (markerless + per-frame) store overrides as corner nudges, so they take the + // same cache-clearing revert path; only true marker mode patches the live marker objects in place. + const usesCornerNudges = readConfig().alignmentPipeline !== "markers"; + if (state.geometry.alignmentInfo && !usesCornerNudges) { // Original auto-detected positions are cached on the live alignment objects so edits can revert instantly // without rerunning the whole detector. for (const [key, marker] of state.geometry.alignmentInfo.markerLookup.entries()) { @@ -3724,7 +3765,7 @@ function clearMarkerEdits() { } } revokeGifUrl(); - if (isMarkerless) { + if (usesCornerNudges) { state.frames.base = new Array(state.geometry.frameCount); state.frames.baseOutputEpoch = new Array(state.geometry.frameCount); state.frames.stabilizedCache.clear(); @@ -3901,7 +3942,7 @@ function updateSliderReadouts() { return; } const config = readConfig(); - const roiSizePx = config.alignmentPipeline === "markerless" + const roiSizePx = config.alignmentPipeline !== "markers" ? estimateMarkerlessCornerTileSidePx( state.geometry.alignmentInfo.rectifiedWidth, state.geometry.alignmentInfo.rectifiedHeight, @@ -4310,7 +4351,8 @@ async function processCurrentImage(requestId = state.processing.requestId) { updatePageGridDetectionHeading(false); setStatus(buildStatusWithTiming(result.statusText)); schedulePreviewFrameWarmup(requestId); - if (config.alignmentPipeline === "markerless") { + // Stabilization runs in both non-marker pipelines, so warm its solver after per-frame too. + if (config.alignmentPipeline !== "markers") { scheduleMarkerlessStabilizationWarmup(requestId); } } catch (error) { @@ -4847,7 +4889,9 @@ function getPreviewFrameQuadForSourceIndex(sourceIndex) { const row = Math.floor(sourceIndex / cols); if (row < 0 || row >= alignmentInfo.rows) return null; const extractionInfo = - readConfig().alignmentPipeline === "markerless" + // Per-frame uses the same corner-cross lattice as markerless, so resolve preview frame quads + // through the markerless extraction builder for both non-marker pipelines. + readConfig().alignmentPipeline !== "markers" ? buildMarkerlessExtractionInfoForFrame(alignmentInfo, col, row) : alignmentInfo; const quad = resolveFrameQuadForPreview(extractionInfo, col, row); @@ -4932,7 +4976,8 @@ function drawOmittedFrameQuads(ctx, mapRectifiedPointToPreview) { */ function resolveDisplayedAlignmentPoint(alignmentInfo, col, row) { const key = getMarkerKey(col, row); - if (readConfig().alignmentPipeline === "markerless") { + // Per-frame shares the markerless corner-display model (phase + stabilization + manual nudge). + if (readConfig().alignmentPipeline !== "markers") { const sourceMarker = alignmentInfo?.markerLookup?.get(key); if (sourceMarker) { const displayed = getMarkerlessDisplayedCorner(sourceMarker, col, row, alignmentInfo); @@ -5604,7 +5649,10 @@ function getFrameGridCoords(index, alignmentInfo) { * @returns {object} */ function getFrameExtractionAlignmentInfo(config, alignmentInfo, col, row, includeMarkerlessNudges) { - if (config.alignmentPipeline === "markerless" && includeMarkerlessNudges) { + // Per-frame extracts from the same corner-cross lattice as markerless, so both non-marker + // pipelines resolve frame quads via the markerless extraction builder (phase nudges are 0 in + // per-frame, so this only contributes stabilization/drift/manual-nudge offsets there). + if (config.alignmentPipeline !== "markers" && includeMarkerlessNudges) { return buildMarkerlessExtractionInfoForFrame(alignmentInfo, col, row); } return alignmentInfo; @@ -6094,7 +6142,8 @@ function scheduleMarkerlessStabilizationWarmup(requestId) { */ function scheduleCurrentStabilizationWarmup() { window.setTimeout(() => { - if (readConfig().alignmentPipeline !== "markerless") { + // Stabilization warmup applies to both non-marker pipelines (markerless + per-frame). + if (readConfig().alignmentPipeline === "markers") { return; } if (!readConfig().stabilizationEnabled) { @@ -6737,7 +6786,8 @@ function combineSourceOffsets(baseOffset, extraOffset) { * @returns {{x:number,y:number}} */ function getMarkerlessVerticalDriftSourceOffset(config, alignmentInfo, frameIndex) { - if (!alignmentInfo || config.alignmentPipeline !== "markerless") { + // Vertical Drift Compensation is enabled in both non-marker pipelines (markerless + per-frame). + if (!alignmentInfo || config.alignmentPipeline === "markers") { return { x: 0, y: 0 }; } const cellHeight = alignmentInfo.gridBounds.height / Math.max(1, alignmentInfo.rows); @@ -6786,7 +6836,8 @@ function getAutomaticMarkerlessSourceOffset(config, alignmentInfo, frameIndex, i * @returns {{x:number,y:number}} */ function getMarkerlessCornerStabilizationOffset(col, row, alignmentInfo) { - if (!alignmentInfo || readConfig().alignmentPipeline !== "markerless") { + // Stabilization and the Frame Corners panel are shared by both non-marker pipelines. + if (!alignmentInfo || readConfig().alignmentPipeline === "markers") { return { x: 0, y: 0 }; } const offsets = getStabilizationOffsets(); @@ -6848,7 +6899,8 @@ function getFrameStabilizationSourceOffset(index) { * @returns {{x:number,y:number}} */ function getMarkerlessCornerManualNudge(col, row) { - if (readConfig().alignmentPipeline !== "markerless") { + // Frame Corners overrides (stored as corner nudges) are enabled in markerless and per-frame. + if (readConfig().alignmentPipeline === "markers") { return { x: 0, y: 0 }; } const override = state.geometry.manualMarkerOverrides.get(getMarkerKey(col, row)); @@ -7087,8 +7139,12 @@ function buildPostRotationPreviewTile(previewCanvas, expected, alignmentInfo, te } /** - * Build a display-only alignment view for markerless mode that incorporates the current manual - * phase offset while leaving the underlying stored marker coordinates unphased. + * Build a display-only alignment view for the non-marker pipelines (markerless + per-frame) that + * incorporates the current corner display offsets while leaving the stored coordinates unphased. + * + * Per-frame shares the markerless corner-tile display path so its Frame Corners panel renders the + * same stabilized/nudged corner tiles. Only true marker mode short-circuits to the raw alignment + * (or, while scrubbing post-rotation, the marker post-rotation preview tiles). * * @param {object | null} alignmentInfo * @returns {object | null} @@ -7097,7 +7153,7 @@ function getDisplayAlignmentInfo(alignmentInfo) { if (!alignmentInfo) return null; const config = readConfig(); const showingPostRotationPreview = state.preview.postRotationScrubbing && !!state.preview.rectifiedCanvas; - if (config.alignmentPipeline !== "markerless" && !showingPostRotationPreview) { + if (config.alignmentPipeline === "markers" && !showingPostRotationPreview) { return alignmentInfo; } const crossRoiScale = config.crossRoiScale; @@ -7105,7 +7161,7 @@ function getDisplayAlignmentInfo(alignmentInfo) { ? getPostRotationPreviewCanvas(state.preview.rectifiedCanvas, getPostRotationPreviewDeg()) : null; - if (config.alignmentPipeline !== "markerless") { + if (config.alignmentPipeline === "markers") { const markerLookup = new Map(alignmentInfo.markerLookup); const crossRoiTiles = alignmentInfo.crossRoiTiles.map((tile) => buildPostRotationPreviewTile( previewCanvas, @@ -7650,9 +7706,12 @@ function applyMarkerOverride(tile, local, finalize) { let detectedX = roiCenterX + (local.x - center); let detectedY = roiCenterY + (local.y - center); const config = readConfig(); - const isMarkerless = config.alignmentPipeline === "markerless"; + // Per-frame shares the markerless corner-nudge override model (stored as deltas from the displayed + // corner), so both non-marker pipelines take the nudge path; only true marker mode stores absolute + // detected positions and patches the marker objects in place. + const usesCornerNudges = config.alignmentPipeline !== "markers"; const key = getMarkerKey(tile.col, tile.row); - if (isMarkerless && state.geometry.alignmentInfo) { + if (usesCornerNudges && state.geometry.alignmentInfo) { const sourceMarker = state.geometry.alignmentInfo.markerLookup.get(key); if (sourceMarker) { const displayed = getMarkerlessDisplayedCorner(sourceMarker, tile.col, tile.row, state.geometry.alignmentInfo, false); @@ -7665,37 +7724,37 @@ function applyMarkerOverride(tile, local, finalize) { state.geometry.manualMarkerOverrides.set(key, { x: detectedX, y: detectedY }); } state.preview.activeEditedMarker = finalize ? null : { col: tile.col, row: tile.row }; - if (state.geometry.alignmentInfo && !isMarkerless) { + if (state.geometry.alignmentInfo && !usesCornerNudges) { // Manual overrides patch the already-detected alignment object in place, which lets preview/extraction // update lazily from the edited marker positions without another CV pass. applyManualMarkerOverrides(state.geometry.alignmentInfo); } revokeGifUrl(); if (!finalize) { - if (isMarkerless) { + if (usesCornerNudges) { beginMarkerOverrideScrub(); } else { state.preview.markerOverrideScrubbing = true; } } let affectedMarkerFrames = null; - if (isMarkerless && !finalize) { + if (usesCornerNudges && !finalize) { invalidateCurrentPreviewFrameForMarker(tile.col, tile.row); - } else if (isMarkerless) { + } else if (usesCornerNudges) { invalidateMarkerlessNudgedFramesForMarker(tile.col, tile.row); } else { affectedMarkerFrames = invalidateFramesForMarker(tile.col, tile.row); } syncMarkerEditingUi(); if (finalize) { - if (isMarkerless) { + if (usesCornerNudges) { endMarkerOverrideScrub(); } else { state.preview.markerOverrideScrubbing = false; } renderCrossRoiGrid(state.geometry.alignmentInfo); scheduleStabilizationPreviewUpdate(); - if (!isMarkerless) { + if (!usesCornerNudges) { if (affectedMarkerFrames) { scheduleCurrentPreviewFrameWarmupForSourceIndices(affectedMarkerFrames); } else { @@ -7717,8 +7776,10 @@ function restoreMarkerOverride(tile) { const key = getMarkerKey(tile.col, tile.row); state.geometry.manualMarkerOverrides.delete(key); state.preview.activeEditedMarker = null; - const isMarkerless = readConfig().alignmentPipeline === "markerless"; - if (state.geometry.alignmentInfo && !isMarkerless) { + // Non-marker pipelines (markerless + per-frame) clear corner-nudge overrides via cache + // invalidation; only marker mode restores auto-detected positions on the live marker objects. + const usesCornerNudges = readConfig().alignmentPipeline !== "markers"; + if (state.geometry.alignmentInfo && !usesCornerNudges) { const marker = state.geometry.alignmentInfo.markerLookup.get(key); const liveTile = state.geometry.alignmentInfo.crossRoiTileMap?.get(key); if (marker && Number.isFinite(marker.autoDetectedX)) { @@ -7734,7 +7795,7 @@ function restoreMarkerOverride(tile) { } revokeGifUrl(); let affectedMarkerFrames = null; - if (isMarkerless) { + if (usesCornerNudges) { invalidateMarkerlessNudgedFramesForMarker(tile.col, tile.row); } else { affectedMarkerFrames = invalidateFramesForMarker(tile.col, tile.row); @@ -7742,7 +7803,7 @@ function restoreMarkerOverride(tile) { syncMarkerEditingUi(); renderCrossRoiGrid(state.geometry.alignmentInfo); scheduleStabilizationPreviewUpdate(); - if (!isMarkerless) { + if (!usesCornerNudges) { if (affectedMarkerFrames) { scheduleCurrentPreviewFrameWarmupForSourceIndices(affectedMarkerFrames); } else { diff --git a/js/dom-state.js b/js/dom-state.js index c709ba9..0816d23 100644 --- a/js/dom-state.js +++ b/js/dom-state.js @@ -97,6 +97,7 @@ const domGroups = { alignmentPipelineField: q("#alignmentPipelineField"), alignmentPipelineMarkerless: q("#alignmentPipelineMarkerless"), alignmentPipelineMarkers: q("#alignmentPipelineMarkers"), + alignmentPipelinePerFrame: q("#alignmentPipelinePerFrame"), stabilizationMethodGroup: q("#stabilizationMethodGroup"), stabilizationMethodField: q("#stabilizationMethodField"), stabilizationMethodPairwise: q("#stabilizationMethodPairwise"), diff --git a/js/i18n.js b/js/i18n.js index 7967a27..77d3ed0 100644 --- a/js/i18n.js +++ b/js/i18n.js @@ -87,6 +87,7 @@ const LOCALES = { pipelineOptions: { markerless: "Markerless (gutters, frames)", markers: "Markers (crosses, dots)", + perFrame: "Per-frame (one image per frame)", }, stabilizationMethod: "Stabilization Method", stabilizationMethodOptions: { @@ -249,6 +250,7 @@ const LOCALES = { alignmentPipelineField: "Choose between markerless frame estimation and marker-based alignment.", alignmentPipelineMarkerless: "Estimate frame divisions without registration marks by fitting a straight grid from image autocorrelation and gutter evidence.", alignmentPipelineMarkers: "Use registration markers between frames to refine the grid alignment.", + alignmentPipelinePerFrame: "Upload one image per animation frame; each is page-rectified and stacked into the animation. Drop several images at once to fill the frames.", stabilizationMethodField: "Choose which translation-only stabilization strategy is used in markerless mode.", stabilizationMethodPairwise: "Compare each frame with neighboring frames in the sheet or loop and solve one weighted global offset field.", stabilizationMethodAverage: "Compare each frame independently against one median reference frame built from the whole animation.", @@ -458,6 +460,7 @@ const LOCALES = { pipelineOptions: { markerless: "Sin marcadores (canales, fotogramas)", markers: "Marcadores (cruces, puntos)", + perFrame: "Por fotograma (una imagen por fotograma)", }, stabilizationMethod: "Método de estabilización", stabilizationMethodOptions: { @@ -619,6 +622,7 @@ const LOCALES = { alignmentPipelineField: "Elige entre la estimación de fotogramas sin marcadores y la alineación basada en marcadores.", alignmentPipelineMarkerless: "Estima las divisiones de los fotogramas sin marcas de registro ajustando una cuadrícula recta a partir de la autocorrelación de la imagen y la evidencia de los canales.", alignmentPipelineMarkers: "Usa marcadores de registro entre fotogramas para refinar la alineación de la cuadrícula.", + alignmentPipelinePerFrame: "Sube una imagen por cada fotograma de la animación; cada una se rectifica y se apila en la animación. Suelta varias imágenes a la vez para llenar los fotogramas.", stabilizationMethodField: "Elige qué estrategia de estabilización solo por traslación se usa en el modo sin marcadores.", stabilizationMethodPairwise: "Compara cada fotograma con los fotogramas vecinos de la hoja o del bucle y resuelve un único campo global de desplazamientos ponderados.", stabilizationMethodAverage: "Compara cada fotograma de forma independiente con un fotograma de referencia mediano construido a partir de toda la animación.", @@ -827,6 +831,7 @@ const LOCALES = { pipelineOptions: { markerless: "Senza marcatori (spazi, fotogrammi)", markers: "Marcatori (croci, punti)", + perFrame: "Per fotogramma (un'immagine per fotogramma)", }, stabilizationMethod: "Metodo di stabilizzazione", stabilizationMethodOptions: { @@ -988,6 +993,7 @@ const LOCALES = { alignmentPipelineField: "Scegli tra la stima dei fotogrammi senza marcatori e l’allineamento basato sui marcatori.", alignmentPipelineMarkerless: "Stima le divisioni dei fotogrammi senza segni di registro adattando una griglia diritta dall’autocorrelazione dell’immagine e dall’evidenza degli spazi vuoti.", alignmentPipelineMarkers: "Usa marcatori di registro tra i fotogrammi per rifinire l’allineamento della griglia.", + alignmentPipelinePerFrame: "Carica un'immagine per ogni fotogramma dell'animazione; ognuna viene rettificata e impilata nell'animazione. Trascina più immagini insieme per riempire i fotogrammi.", stabilizationMethodField: "Scegli quale strategia di stabilizzazione solo per traslazione usare in modalità senza marcatori.", stabilizationMethodPairwise: "Confronta ogni fotogramma con i fotogrammi vicini nel foglio o nel loop e risolve un unico campo globale di offset pesati.", stabilizationMethodAverage: "Confronta ogni fotogramma in modo indipendente con un fotogramma di riferimento mediano costruito dall'intera animazione.", @@ -1196,6 +1202,7 @@ const LOCALES = { pipelineOptions: { markerless: "マーカーレス（ガター、フレーム）", markers: "マーカー使用（十字、点）", + perFrame: "フレームごと（1フレームに1枚）", }, stabilizationMethod: "スタビライズ方式", stabilizationMethodOptions: { @@ -1357,6 +1364,7 @@ const LOCALES = { alignmentPipelineField: "マーカーレスのフレーム推定と、マーカーを使う整列のどちらを使うか選びます。", alignmentPipelineMarkerless: "登録マークなしで、画像の自己相関とガターの手掛かりから直線的な格子を当てはめてフレーム境界を推定します。", alignmentPipelineMarkers: "フレーム間の登録マーカーを使って格子の整列を微調整します。", + alignmentPipelinePerFrame: "アニメの1フレームにつき1枚の画像をアップロードします。各画像はページ補正されてアニメに積み重ねられます。複数枚をまとめてドロップしてフレームを埋められます。", stabilizationMethodField: "マーカーレスモードで使う並進のみのスタビライズ手法を選びます。", stabilizationMethodPairwise: "各フレームをシート内またはループ内の近傍フレームと比較し、重み付きの全体オフセット場を解きます。", stabilizationMethodAverage: "各フレームを、アニメーション全体から作った中央値参照フレームと個別に比較します。", @@ -1565,6 +1573,7 @@ const LOCALES = { pipelineOptions: { markerless: "无标记（空隙、帧）", markers: "标记（十字、圆点）", + perFrame: "逐帧（每帧一张图片）", }, stabilizationMethod: "稳定化方法", stabilizationMethodOptions: { @@ -1726,6 +1735,7 @@ const LOCALES = { alignmentPipelineField: "选择无标记帧估计或基于标记的对齐。", alignmentPipelineMarkerless: "不使用注册标记，而是根据图像自相关和空隙证据拟合直线网格来估计帧分割。", alignmentPipelineMarkers: "使用帧之间的注册标记来细化网格对齐。", + alignmentPipelinePerFrame: "为动画的每一帧上传一张图片；每张图片都会经过页面矫正并叠加成动画。可一次拖入多张图片来填充各帧。", stabilizationMethodField: "选择在无标记模式下使用哪种仅平移的稳定化策略。", stabilizationMethodPairwise: "将每一帧与纸面或循环中的邻近帧比较，并求解一个加权的全局偏移场。", stabilizationMethodAverage: "将每一帧独立地与由整段动画构建出的中值参考帧进行比较。", @@ -1934,6 +1944,7 @@ const LOCALES = { pipelineOptions: { markerless: "無標記（空隙、影格）", markers: "標記（十字、圓點）", + perFrame: "逐格（每格一張圖片）", }, stabilizationMethod: "穩定化方法", stabilizationMethodOptions: { @@ -2095,6 +2106,7 @@ const LOCALES = { alignmentPipelineField: "選擇無標記影格估計或基於標記的對齊。", alignmentPipelineMarkerless: "不使用註冊標記，而是根據影像自相關與空隙證據擬合直線網格來估計影格分割。", alignmentPipelineMarkers: "使用影格之間的註冊標記來細化網格對齊。", + alignmentPipelinePerFrame: "為動畫的每一格上傳一張圖片；每張圖片都會經過頁面矯正並疊加成動畫。可一次拖入多張圖片來填滿各格。", stabilizationMethodField: "選擇在無標記模式下使用哪種僅平移的穩定化策略。", stabilizationMethodPairwise: "將每個影格與紙面或循環中的鄰近影格比較，並求解一個加權的全域偏移場。", stabilizationMethodAverage: "將每個影格獨立地與由整段動畫建立出的中值參考影格進行比較。", @@ -2303,6 +2315,7 @@ const LOCALES = { pipelineOptions: { markerless: "마커 없음 (간격, 프레임)", markers: "마커 사용 (십자, 점)", + perFrame: "프레임별 (프레임당 이미지 한 장)", }, stabilizationMethod: "안정화 방법", stabilizationMethodOptions: { @@ -2464,6 +2477,7 @@ const LOCALES = { alignmentPipelineField: "마커 없는 프레임 추정과 마커 기반 정렬 중에서 선택합니다.", alignmentPipelineMarkerless: "등록 마크 없이 이미지 자기상관과 간격 단서를 바탕으로 직선 격자를 맞춰 프레임 경계를 추정합니다.", alignmentPipelineMarkers: "프레임 사이의 등록 마커를 사용해 격자 정렬을 세밀하게 맞춥니다.", + alignmentPipelinePerFrame: "애니메이션 프레임마다 이미지 한 장을 업로드합니다. 각 이미지는 페이지 보정 후 애니메이션으로 쌓입니다. 여러 장을 한 번에 드롭해 프레임을 채울 수 있습니다.", stabilizationMethodField: "마커 없는 모드에서 사용할 평행이동 전용 안정화 방식을 선택합니다.", stabilizationMethodPairwise: "각 프레임을 시트나 루프에서 이웃한 프레임과 비교하고 가중 전역 오프셋 필드를 풉니다.", stabilizationMethodAverage: "각 프레임을 전체 애니메이션으로 만든 중앙값 참조 프레임과 독립적으로 비교합니다.", @@ -2672,6 +2686,7 @@ const LOCALES = { pipelineOptions: { markerless: "Sem marcadores (espaços, quadros)", markers: "Marcadores (cruzes, pontos)", + perFrame: "Por quadro (uma imagem por quadro)", }, stabilizationMethod: "Método de estabilização", stabilizationMethodOptions: { @@ -2833,6 +2848,7 @@ const LOCALES = { alignmentPipelineField: "Escolha entre a estimativa de quadros sem marcadores e o alinhamento baseado em marcadores.", alignmentPipelineMarkerless: "Estima as divisões dos quadros sem marcas de registro, ajustando uma grade reta a partir da autocorrelação da imagem e da evidência dos espaços vazios.", alignmentPipelineMarkers: "Usa marcadores de registro entre os quadros para refinar o alinhamento da grade.", + alignmentPipelinePerFrame: "Envie uma imagem por quadro da animação; cada uma é retificada e empilhada na animação. Solte várias imagens de uma vez para preencher os quadros.", stabilizationMethodField: "Escolha qual estratégia de estabilização apenas por translação será usada no modo sem marcadores.", stabilizationMethodPairwise: "Compara cada quadro com os quadros vizinhos na folha ou no loop e resolve um único campo global de deslocamentos ponderados.", stabilizationMethodAverage: "Compara cada quadro independentemente com um quadro de referência mediano construído a partir de toda a animação.", @@ -3041,6 +3057,7 @@ const LOCALES = { pipelineOptions: { markerless: "Markerlos (Zwischenräume, Frames)", markers: "Marker (Kreuze, Punkte)", + perFrame: "Pro Bild (ein Bild pro Animationsbild)", }, stabilizationMethod: "Stabilisierungsmethode", stabilizationMethodOptions: { @@ -3202,6 +3219,7 @@ const LOCALES = { alignmentPipelineField: "Wählen Sie zwischen markerloser Frame-Schätzung und markerbasierter Ausrichtung.", alignmentPipelineMarkerless: "Schätzt Frame-Unterteilungen ohne Registrierungsmarken, indem aus Bildautokorrelation und Zwischenraum-Hinweisen ein gerades Raster angepasst wird.", alignmentPipelineMarkers: "Verwendet Registrierungsmarker zwischen den Frames, um die Rasterausrichtung zu verfeinern.", + alignmentPipelinePerFrame: "Lade ein Bild pro Animationsbild hoch; jedes wird seitenkorrigiert und zur Animation gestapelt. Mehrere Bilder auf einmal ablegen, um die Frames zu füllen.", stabilizationMethodField: "Wählen Sie, welche reine Translations-Stabilisierungsstrategie im markerlosen Modus verwendet wird.", stabilizationMethodPairwise: "Vergleicht jeden Frame mit benachbarten Frames im Blatt oder in der Schleife und löst ein gewichtetes globales Offset-Feld.", stabilizationMethodAverage: "Vergleicht jeden Frame unabhängig mit einem Median-Referenzframe, der aus der gesamten Animation gebildet wird.", @@ -3410,6 +3428,7 @@ const LOCALES = { pipelineOptions: { markerless: "Bez znaczników (odstępy, klatki)", markers: "Znaczniki (krzyżyki, kropki)", + perFrame: "Na klatkę (jeden obraz na klatkę)", }, stabilizationMethod: "Metoda stabilizacji", stabilizationMethodOptions: { @@ -3571,6 +3590,7 @@ const LOCALES = { alignmentPipelineField: "Wybierz między bezznacznikowym szacowaniem klatek a wyrównaniem opartym na znacznikach.", alignmentPipelineMarkerless: "Szacuje podziały klatek bez znaków rejestracyjnych, dopasowując prostą siatkę na podstawie autokorelacji obrazu i sygnałów z odstępów.", alignmentPipelineMarkers: "Używa znaczników rejestracyjnych między klatkami do doprecyzowania wyrównania siatki.", + alignmentPipelinePerFrame: "Prześlij jeden obraz na każdą klatkę animacji; każdy zostaje wyprostowany i ułożony w animację. Upuść kilka obrazów naraz, aby wypełnić klatki.", stabilizationMethodField: "Wybierz, która strategia stabilizacji oparta wyłącznie na przesunięciu ma być używana w trybie bez znaczników.", stabilizationMethodPairwise: "Porównuje każdą klatkę z sąsiednimi klatkami na arkuszu lub w pętli i wyznacza jedno ważone globalne pole przesunięć.", stabilizationMethodAverage: "Porównuje każdą klatkę niezależnie z medianową klatką odniesienia zbudowaną z całej animacji.", @@ -3779,6 +3799,7 @@ const LOCALES = { pipelineOptions: { markerless: "Uten markører (mellomrom, ruter)", markers: "Markører (kryss, prikker)", + perFrame: "Per rute (ett bilde per rute)", }, stabilizationMethod: "Stabiliseringsmetode", stabilizationMethodOptions: { @@ -3940,6 +3961,7 @@ const LOCALES = { alignmentPipelineField: "Velg mellom markørløs ruteberegning og markørbasert justering.", alignmentPipelineMarkerless: "Estimerer ruteinndelingen uten registreringsmerker ved å tilpasse et rett rutenett fra bildeautokorrelasjon og tegn på tomme mellomrom.", alignmentPipelineMarkers: "Bruker registreringsmarkører mellom rutene for å finjustere rutenettjusteringen.", + alignmentPipelinePerFrame: "Last opp ett bilde per animasjonsrute; hvert bilde sidekorrigeres og stables til animasjonen. Slipp flere bilder samtidig for å fylle rutene.", stabilizationMethodField: "Velg hvilken stabiliseringsstrategi som kun bruker translasjon i markørløs modus.", stabilizationMethodPairwise: "Sammenligner hver rute med nabobildene i arket eller løkken og løser ett vektet globalt forskyvningsfelt.", stabilizationMethodAverage: "Sammenligner hver rute uavhengig med en medianreferanserute bygget fra hele animasjonen.", @@ -4148,6 +4170,7 @@ const LOCALES = { pipelineOptions: { markerless: "Без міток (проміжки, кадри)", markers: "Мітки (хрестики, точки)", + perFrame: "Покадрово (одне зображення на кадр)", }, stabilizationMethod: "Метод стабілізації", stabilizationMethodOptions: { @@ -4309,6 +4332,7 @@ const LOCALES = { alignmentPipelineField: "Виберіть між безмітковим оцінюванням кадрів і вирівнюванням на основі міток.", alignmentPipelineMarkerless: "Оцінює поділ кадрів без реєстраційних міток, підганяючи пряму сітку за автокореляцією зображення та ознаками порожніх проміжків.", alignmentPipelineMarkers: "Використовує реєстраційні мітки між кадрами для уточнення вирівнювання сітки.", + alignmentPipelinePerFrame: "Завантажте по одному зображенню на кадр анімації; кожне вирівнюється за сторінкою та складається в анімацію. Перетягніть кілька зображень одразу, щоб заповнити кадри.", stabilizationMethodField: "Виберіть, яку стратегію стабілізації лише за зсувом використовувати в режимі без міток.", stabilizationMethodPairwise: "Порівнює кожен кадр із сусідніми кадрами на аркуші або в циклі та розв’язує одне зважене глобальне поле зміщень.", stabilizationMethodAverage: "Порівнює кожен кадр окремо з медіанним опорним кадром, побудованим з усієї анімації.", @@ -4517,6 +4541,7 @@ const LOCALES = { pipelineOptions: { markerless: "Sans repères (gouttières, images)", markers: "Repères (croix, points)", + perFrame: "Par image (une image par image d’animation)", }, stabilizationMethod: "Méthode de stabilisation", stabilizationMethodOptions: { @@ -4678,6 +4703,7 @@ const LOCALES = { alignmentPipelineField: "Choisissez entre l’estimation des images sans repères et l’alignement basé sur des repères.", alignmentPipelineMarkerless: "Estime les divisions d’images sans marques de repérage en ajustant une grille droite à partir de l’autocorrélation de l’image et des indices de gouttières.", alignmentPipelineMarkers: "Utilise des repères de registration entre les images pour affiner l’alignement de la grille.", + alignmentPipelinePerFrame: "Importez une image par image d’animation ; chacune est redressée puis empilée dans l’animation. Déposez plusieurs images à la fois pour remplir les images.", stabilizationMethodField: "Choisissez quelle stratégie de stabilisation uniquement par translation utiliser en mode sans repères.", stabilizationMethodPairwise: "Compare chaque image aux images voisines sur la feuille ou dans la boucle et résout un unique champ global de décalages pondérés.", stabilizationMethodAverage: "Compare chaque image indépendamment à une image de référence médiane construite à partir de l’ensemble de l’animation.", @@ -4926,6 +4952,7 @@ const TOOLTIP_SELECTOR_KEYS = { "#alignmentPipelineField": "alignmentPipelineField", "#alignmentPipelineMarkerless": ["alignmentPipelineMarkerless", "alignmentPipelineField"], "#alignmentPipelineMarkers": ["alignmentPipelineMarkers", "alignmentPipelineField"], + "#alignmentPipelinePerFrame": ["alignmentPipelinePerFrame", "alignmentPipelineField"], "#stabilizationMethodField": "stabilizationMethodField", "#stabilizationMethodPairwise": ["stabilizationMethodPairwise", "stabilizationMethodField"], "#stabilizationMethodAverage": ["stabilizationMethodAverage", "stabilizationMethodField"], diff --git a/js/settings-defaults.js b/js/settings-defaults.js index a49048e..1bb9c27 100644 --- a/js/settings-defaults.js +++ b/js/settings-defaults.js @@ -115,6 +115,9 @@ export function applyCropGeometryDefaults(dom) { export function applyNonLayoutDefaults(dom) { dom.alignmentPipelineMarkers.checked = SETTINGS_DEFAULTS.detection.alignmentPipeline === "markers"; dom.alignmentPipelineMarkerless.checked = SETTINGS_DEFAULTS.detection.alignmentPipeline === "markerless"; + if (dom.alignmentPipelinePerFrame) { + dom.alignmentPipelinePerFrame.checked = SETTINGS_DEFAULTS.detection.alignmentPipeline === "per-frame"; + } if (dom.stabilizationMethodPairwise) { dom.stabilizationMethodPairwise.checked = SETTINGS_DEFAULTS.detection.stabilizationMethod === "pairwise-cyclic"; } diff --git a/js/ui-controls.js b/js/ui-controls.js index 41389b5..f9daad1 100644 --- a/js/ui-controls.js +++ b/js/ui-controls.js @@ -92,6 +92,7 @@ export function initializeTooltips({ tooltipText, state, dom, applyTooltipState * * @param {{ * dom: import("./dom-state.js").dom, + * state: import("./dom-state.js").state, * revokeGifUrl: () => void, * updateSliderReadouts: () => void, * scheduleProcess: (delayMs?: number) => void, @@ -101,25 +102,43 @@ export function initializeTooltips({ tooltipText, state, dom, applyTooltipState */ function attachAlignmentPipelineControls({ dom, + state, revokeGifUrl, updateSliderReadouts, scheduleProcess, syncAlignmentMarkerUi, }) { const alignmentDom = dom.alignment; - [alignmentDom.alignmentPipelineMarkerless, alignmentDom.alignmentPipelineMarkers].forEach((input) => { - input.addEventListener("input", () => { - syncAlignmentMarkerUi(); - revokeGifUrl(); - updateSliderReadouts(); - scheduleProcess(); - }); - input.addEventListener("change", () => { - syncAlignmentMarkerUi(); - revokeGifUrl(); - scheduleProcess(); + // The radio group is now the authoritative source of pipeline mode. A multi-image load set the + // legacy `forcePerFrameMode` shim (and ticked the per-frame radio); once the user interacts with + // the radio, reconcile the shim to the radio so switching *out* of per-frame actually sticks. + const reconcilePerFrameForceFlag = () => { + state.runtime.forcePerFrameMode = !!alignmentDom.alignmentPipelinePerFrame?.checked; + }; + [ + alignmentDom.alignmentPipelineMarkerless, + alignmentDom.alignmentPipelineMarkers, + alignmentDom.alignmentPipelinePerFrame, + ] + .filter(Boolean) + .forEach((input) => { + input.addEventListener("input", () => { + reconcilePerFrameForceFlag(); + syncAlignmentMarkerUi(); + revokeGifUrl(); + updateSliderReadouts(); + // Switching into per-frame mode before any images are uploaded must not process — there is + // nothing to rectify yet. `scheduleProcess` already no-ops when `state.source.image` is null + // (which is the empty-`images[]` case), so just wait for the upload to drive processing. + scheduleProcess(); + }); + input.addEventListener("change", () => { + reconcilePerFrameForceFlag(); + syncAlignmentMarkerUi(); + revokeGifUrl(); + scheduleProcess(); + }); }); - }); } /** @@ -816,6 +835,7 @@ export function attachUi({ attachAlignmentPipelineControls({ dom, + state, revokeGifUrl, updateSliderReadouts, scheduleProcess, diff --git a/per_frame_pipeline_plan.md b/per_frame_pipeline_plan.md index e45ce66..27bf2ec 100644 --- a/per_frame_pipeline_plan.md +++ b/per_frame_pipeline_plan.md @@ -640,6 +640,136 @@ and wire mode-gated visibility. - The image strip itself (Phase 7). - Settings persistence (Phase 8). +**As built (Phase 6 — COMPLETE):** + +*New radio + DOM/i18n wiring* +- `index.html`: added a third radio `#alignmentPipelinePerFrame` (`name="alignmentPipeline"`, + `value="per-frame"`) to `#alignmentPipelineField`, with a `` + label matching the two siblings. +- `js/dom-state.js`: added `alignmentPipelinePerFrame: q("#alignmentPipelinePerFrame")` to the + `alignment` group (auto-flattened to `dom.alignmentPipelinePerFrame`). No new container refs were + needed — visibility is gated entirely through the existing alignment rows plus a new body class. +- `js/settings-defaults.js`: default stays `"markers"`. `applyNonLayoutDefaults` now also sets + `dom.alignmentPipelinePerFrame.checked = (default === "per-frame")` (guarded by existence), so + reset/sync drives all three radios consistently. +- `js/ui-controls.js`: the per-frame radio joins the `attachAlignmentPipelineControls` `input`/`change` + listeners (`.filter(Boolean)` so a missing ref is harmless). Switching into per-frame with no images + loaded does NOT process — `scheduleProcess` already no-ops when `state.source.image` is null (the + empty-`images[]` case), so the handler just waits for upload. The handler also calls + `reconcilePerFrameForceFlag()` which sets `state.runtime.forcePerFrameMode` to the per-frame radio's + checked state, so once the user interacts with the radio it becomes authoritative and switching + *out* of per-frame after a multi-image load actually sticks (the Phase 4 shim no longer pins the + mode). `attachAlignmentPipelineControls` gained `state` in its deps for this. +- `js/app.js`: `readConfig`/`isPerFrameModeActive` already emit `"per-frame"` from Phase 5; verified + they now resolve through the real radio. `getActiveAlignmentPipeline()` now returns `"per-frame"` + when `isPerFrameModeActive()` (else markerless/markers as before). `resetNonLayoutControls` clears + `state.runtime.forcePerFrameMode = false` so a reset truly returns to the default markers pipeline. + +*Mode flags (new in `getAlignmentUiModeFlags`)* +- `showMarkerlessControls` (`=== "markerless"`) — unchanged, strictly markerless-only. +- `showMarkersPipelineControls` (`=== "markers"`) — unchanged. +- `showCrossOnlyControls` — unchanged (markers + cross marker type). +- `showFrameCornerControls` (`!== "markers"`) — NEW: the shared non-marker family (markerless + + per-frame) for stabilization, drift compensation, Frame Corners labels/slider/tooltips. +- `isPerFrame` (`=== "per-frame"`) — NEW: drives the per-frame drop note and the + `per-frame-pipeline` body class. +- Body classes: `markerless-pipeline` (unchanged) plus new `per-frame-pipeline`. `style.css` + extended the markerless flat-background viewport selectors to also match `per-frame-pipeline` (the + Frame Corners + preview viewports), preventing a checkerboard mismatch in per-frame mode. (Note: + `style.css` was not in the Phase 6 file list; this is a 2-line additive selector only.) + +*New i18n keys (added to ALL 13 locale tables; same locale set as Phase 4's `photo.dropNotePerFrame`)* +- `alignment.pipelineOptions.perFrame` — the radio label (locale-translated in all 13 tables). +- `tooltip.alignmentPipelinePerFrame` — the radio tooltip (locale-translated in all 13 tables), + plus a `TOOLTIP_SELECTOR_KEYS` entry `"#alignmentPipelinePerFrame": ["alignmentPipelinePerFrame", + "alignmentPipelineField"]`. +- The per-frame drop note reuses the existing `photo.dropNotePerFrame` (added in Phase 4); no new + drop-note key was needed. `syncAlignmentPipelineLabels` now shows `photo.dropNotePerFrame` when + `isPerFrame`, else `photo.dropNote`. +- Verified: `perFrame: "` appears 13×, `alignmentPipelinePerFrame` appears 14× (13 tooltip values + + 1 selector key), `dropNotePerFrame` appears 13×. + +*Classification checklist — every `alignmentPipeline === "markerless"` / `!== "markerless"` site +audited in `js/app.js` (line numbers approximate, post-edit).* Reclassification preserves +byte-identical behavior for markers and markerless: for markerless, `!== "markers"` ≡ old +`=== "markerless"` (true) and `=== "markers"` ≡ old `!== "markerless"` (false); only per-frame newly +joins the non-marker branch. + +| Site | Function | Old guard | Classification | New guard | +|------|----------|-----------|----------------|-----------| +| ~1111 | inverted marker vision | `=== "markers"` | markers-only (untouched) | `=== "markers"` | +| ~2100 | `applyManualMarkerOverrides` | `=== "markerless"` return | non-marker (skip in-place marker patch) | `!== "markers"` | +| ~2405 | `beginStabilizationStrengthScrub` | `=== "markerless"` warmup | non-marker (stabilization shared) | `!== "markers"` | +| ~3688 | `toggleMarkerlessPhaseDebug` | `!== "markerless"` return | **markerless-only** (phase debug) | unchanged | +| ~3717 | `toggleMarkerlessWorkingImage` | `!== "markerless"` return | **markerless-only** (working-image debug) | unchanged | +| ~3749 | `clearMarkerEdits` (`usesCornerNudges`) | `=== "markerless"` | non-marker (corner-nudge revert path) | `!== "markers"` | +| ~3945 | `updateSliderReadouts` ROI estimate | `=== "markerless"` | non-marker (corner-tile estimate) | `!== "markers"` | +| ~4355 | post-process stabilization warmup | `=== "markerless"` | non-marker (warm stabilization for per-frame) | `!== "markers"` | +| ~4434 | `renderRectifiedPreview` working canvas | `=== "markerless"` | **markerless-only** (working-blur canvas) | unchanged | +| ~4560 | grid-search-inset overlay | `=== "markerless"` | **markerless-only** (search-inset visual) | unchanged | +| ~4603 | markerless phase-debug chart | `=== "markerless"` | **markerless-only** (phase debug viz) | unchanged | +| ~4894 | `getPreviewFrameQuadForSourceIndex` | `=== "markerless"` | non-marker (corner-lattice extraction) | `!== "markers"` | +| ~4980 | `resolveDisplayedAlignmentPoint` | `=== "markerless"` | non-marker (corner display model) | `!== "markers"` | +| ~5655 | `getFrameExtractionAlignmentInfo` | `=== "markerless"` | non-marker (extraction lattice; phase=0 in per-frame) | `!== "markers"` | +| ~6146 | `scheduleCurrentStabilizationWarmup` | `!== "markerless"` return | non-marker (stabilization shared) | `=== "markers"` | +| ~6748 | `getMarkerlessPhaseSourceOffset` | `!== "markerless"` → {0,0} | **markerless-only** (per-frame has no phase sweep; returns 0) | unchanged | +| ~6790 | `getMarkerlessVerticalDriftSourceOffset` | `!== "markerless"` → {0,0} | non-marker (Vertical Drift Compensation enabled) | `=== "markers"` | +| ~6840 | `getMarkerlessCornerStabilizationOffset` | `!== "markerless"` → {0,0} | non-marker (stabilization + Frame Corners) | `=== "markers"` | +| ~6903 | `getMarkerlessCornerManualNudge` | `!== "markerless"` → {0,0} | non-marker (Frame Corners overrides) | `=== "markers"` | +| ~7156 | `getDisplayAlignmentInfo` early return | `!== "markerless" && !preview` | non-marker (markers-only short-circuit) | `=== "markers" && !preview` | +| ~7164 | `getDisplayAlignmentInfo` marker post-rot branch | `!== "markerless"` | non-marker (markers-only marker preview; per-frame falls to corner builder) | `=== "markers"` | +| ~7712 | `applyMarkerOverride` (`usesCornerNudges`) | `=== "markerless"` | non-marker (corner-nudge override semantics) | `!== "markers"` | +| ~7781 | `restoreMarkerOverride` (`usesCornerNudges`) | `=== "markerless"` | non-marker (corner-nudge clear path) | `!== "markers"` | +| (n/a) | `getPageBoundaryPreviewSignature` (~7211) | `config.alignmentPipeline` in cache key | passthrough (per-frame yields distinct signature naturally) | unchanged | +| (n/a) | `syncAlignmentMarkerUi` working-image reset | `pipeline !== "markerless"` | **markerless-only** (resets working-image flag; correct for per-frame) | unchanged | + +*Visibility audit (`syncAlignmentPipelineVisibility`, label/slider/tooltip syncs)* +- **Disabled in per-frame** (hidden): Grid Edge Threshold (`boundarySensitivityRow`) and Grid Edge + Run Length (`boundaryPersistenceRow`) changed `hidden = showMarkerlessControls` → + `hidden = !showMarkersPipelineControls` (marker-grid-only, so hidden in markerless AND per-frame); + marker-type field (`alignmentMarkerTypeField`, already markers-only); markerless gutter/phase + sliders (`markerlessPhaseXRow`/`markerlessPhaseYRow`, kept `!showMarkerlessControls`); the + Rectified Grid Pre/Post (`Rectified Grid` toggle) is not pipeline-radio-gated here — the marker + editor's blob view (`toggleMarkerBlobViewButton`) is already always hidden. +- **Enabled in per-frame** (visible via `showFrameCornerControls`): stabilization method group + + enable + strength + lambda rows (Neighbor / Median both work); Vertical Drift Compensation row; + ROI-size slider position (`syncAlignmentSliderOrder`); Frame Corners override editor + (`toggleMarkerEditingButton`/`clearMarkerEditsButton` — not pipeline-gated, inherits markerless + behavior via the `usesCornerNudges` override sites above). `syncStabilizationMethodUi` lambda + gating moved to `showFrameCornerControls`. +- **Labels/tooltips**: `syncAlignmentPipelineLabels` uses `showFrameCornerControls` for the Frame + Corners heading, the Centers/Stabilize viewer + mobile-control tab labels, and the ROI-size label + (so per-frame reads "Frame Corners"/"Centers"/"Stabilize", not marker wording); + `showMarkerlessControls` is still used only for the gutter-specific `summaryMarkerless` copy. + `syncAlignmentModeTooltips` now takes `showFrameCornerControls`. Drop note uses `isPerFrame ? + dropNotePerFrame : dropNote`. + +*Mobile checkpoint:* viewer tabs and mobile control tabs are always present (not pipeline-gated); +only their text changes, now via `showFrameCornerControls`, so per-frame shows Centers/Stabilize on +mobile single-viewer just like markerless. No viewer-tab visibility branch keys off pipeline mode, so +all three pipelines work in mobile single-viewer mode. + +*Mat lifetime:* Phase 6 allocates no OpenCV Mats (UI-only). No `.delete()` changes. + +*Validation:* `node --check` passes on `js/app.js`, `js/ui-controls.js`, `js/dom-state.js`, +`js/settings-defaults.js`, `js/i18n.js`. i18n key counts verified (see above). Markers/markerless +conditionals confirmed byte-identical by the `markers`/`markerless` truth-table above. + +*Notes for Phases 7–9:* +- (7) The strip should call `setActiveImage(index)` (Phase 5) on thumbnail click and re-derive the + active index after reorder/delete. Strip visibility can key off the `per-frame-pipeline` body class + or `isPerFrameModeActive()`. The strip belongs in the Photo group; nothing in Phase 6 reserved a + container, so Phase 7 adds its own (`#perFrameStripPanel`) and DOM refs. +- (8) Settings load currently still routes `alignment_pipeline = per-frame` through + `settings-io.js`'s markers/markerless branch (lines ~147-151) — **left untouched on purpose** + (Phase 8 owns per-frame settings round-trip). When Phase 8 lands, that load branch must recognize + `per-frame` and set `dom.alignmentPipelinePerFrame.checked` + `state.runtime.forcePerFrameMode` + rather than falling back to markerless. Save side already emits `config.alignmentPipeline` + (`per-frame`) correctly. +- (9) Per-frame's post-rotation live scrub preview is still markerless-gated (Phase 5 note); wiring a + per-frame scrub preview is later polish. No memory work was added here (Phase 9 owns the cell-size + ceiling and per-image cache trimming). + --- ### Phase 7 — Image strip UI diff --git a/style.css b/style.css index b1e2caf..a4f6aed 100644 --- a/style.css +++ b/style.css @@ -772,7 +772,9 @@ body.has-loaded-image .output-preview { } body.has-loaded-image.markerless-pipeline .viewer-card[data-view="preview"] .output-preview, -body.has-loaded-image.markerless-pipeline .viewer-card[data-view="markers"] .cross-roi-viewport { +body.has-loaded-image.markerless-pipeline .viewer-card[data-view="markers"] .cross-roi-viewport, +body.has-loaded-image.per-frame-pipeline .viewer-card[data-view="preview"] .output-preview, +body.has-loaded-image.per-frame-pipeline .viewer-card[data-view="markers"] .cross-roi-viewport { background-color: rgb(243, 244, 246); background-image: none; } From abcdcf2a89738eaa89986a029e5e3fb7550c4efc Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 11 Jun 2026 20:14:16 +0000 Subject: [PATCH 09/21] phase 7 --- index.html | 11 ++ js/app.js | 79 ++++++++ js/dom-state.js | 7 + js/i18n.js | 104 +++++++++++ js/load-controller.js | 2 +- js/per-frame-strip.js | 368 +++++++++++++++++++++++++++++++++++++ js/ui-controls.js | 16 ++ per_frame_pipeline_plan.md | 95 ++++++++++ style.css | 142 ++++++++++++++ 9 files changed, 823 insertions(+), 1 deletion(-) create mode 100644 js/per-frame-strip.js diff --git a/index.html b/index.html index ecb203c..dbe43ab 100644 --- a/index.html +++ b/index.html @@ -68,6 +68,17 @@

Photo

by crosses, dots, or empty gutters. + + diff --git a/js/app.js b/js/app.js index 6a231d9..d10abbc 100644 --- a/js/app.js +++ b/js/app.js @@ -35,6 +35,7 @@ import { releaseOwnedSourceUrl as releaseSourceUrl, handleFile as loadFileSource, loadImageSource as loadImageSourceViaController, + decodeImageElement, } from "./load-controller.js"; import { releaseRectifiedDragUrl as releaseRectifiedDragAsset, @@ -73,7 +74,9 @@ import { setActiveSourceImage, setActiveManualPageContour, setActivePostRotationDeg, + createSourceImageEntry, } from "./source-images.js"; +import { renderPerFrameStrip } from "./per-frame-strip.js"; // Final output-size scaling can be done either with browser canvas drawImage() or with OpenCV. // Keep both code paths available for comparison while evaluating tiny-output quality. const bUseOpenCvOutputScaling = true; @@ -2033,6 +2036,9 @@ function attachUi() { endPostRotationScrub, finishPostRotationScrubIfUnchanged, commitActivePostRotationFromSlider, + setActiveImage, + isPerFrameModeActive, + addPerFrameImages, bumpFrameOutputEpoch, setGeometryProcessingCursor, cancelInFlightProcessing, @@ -2924,6 +2930,74 @@ function scheduleProcess(delayMs = 220) { }, Math.max(0, delayMs)); } +/** + * Decode and append additional images as per-frame entries, then reprocess with the new frame count. + * + * Reuses the Phase 4 decode path (`decodeImageElement`) so the strip's `+` tile and additional drops + * share one code path. Each new entry owns its own blob URL + source-resolution canvas. If the strip + * was empty (e.g. after deleting every image), the first added entry becomes the active image and the + * legacy projections are repointed at it via `setActiveImage`. Forces per-frame mode on so adding + * images never silently changes pipelines. + * + * @param {File[]} files + * @returns {Promise} + */ +async function addPerFrameImages(files) { + const imageFiles = (files || []).filter((file) => file && String(file.type || "").startsWith("image/")); + if (imageFiles.length === 0) return; + + setGeometryProcessingCursor(true); + const startedEmpty = !Array.isArray(state.source.images) || state.source.images.length === 0; + if (!Array.isArray(state.source.images)) state.source.images = []; + + let addedAny = false; + for (const file of imageFiles) { + const url = URL.createObjectURL(file); + let image; + try { + image = await decodeImageElement(url); + } catch { + try { + URL.revokeObjectURL(url); + } catch { + /* already revoked */ + } + continue; + } + const canvas = document.createElement("canvas"); + drawImageToCanvas(image, canvas); + state.source.images.push( + createSourceImageEntry({ + image, + filename: file.name || "", + mimeType: file.type || "image/jpeg", + ownedObjectUrl: url, + dragUrl: url, + canvas, + }), + ); + addedAny = true; + } + if (!addedAny) { + setGeometryProcessingCursor(false); + return; + } + + // Adding images is an explicit per-frame action; make sure the pipeline is in per-frame mode. + state.runtime.forcePerFrameMode = true; + if (dom.alignmentPipelinePerFrame) dom.alignmentPipelinePerFrame.checked = true; + document.body.classList.add("has-loaded-image"); + + if (startedEmpty) { + // No active image existed (fresh or fully-emptied strip): adopt the first new entry as active and + // repoint the legacy projections so processCurrentImage has a source image to read. + setActiveImage(0); + } + syncAlignmentMarkerUi(); + renderPerFrameStrip(); + scheduleProcess(0); +} + /** * Report whether the per-frame alignment pipeline is currently active. * @@ -3600,6 +3674,9 @@ function syncAlignmentMarkerUi() { } } syncMarkerlessPhaseDebugUi(); + // Keep the per-frame strip's visibility + thumbnails in sync with the active pipeline. Idempotent: + // hides/empties the strip in markers/markerless modes and only rebuilds on real image[] changes. + renderPerFrameStrip(); } /** @@ -7537,6 +7614,8 @@ function setActiveImage(index) { syncRawPhotoHeadingLink(); syncRawPhotoCreditDisplay(); renderRawPreview(); + // Refresh the strip's active-thumbnail highlight to match the newly selected image. + renderPerFrameStrip(); } /** diff --git a/js/dom-state.js b/js/dom-state.js index 0816d23..ec5a656 100644 --- a/js/dom-state.js +++ b/js/dom-state.js @@ -47,6 +47,13 @@ const domGroups = { fileInput: q("#fileInput"), loadDemoSelect: q("#loadDemoSelect"), }, + perFrameStripRefs: { + perFrameStripPanel: q("#perFrameStripPanel"), + perFrameStripHeading: q("#perFrameStripHeading"), + perFrameStripCount: q("#perFrameStripCount"), + perFrameStrip: q("#perFrameStrip"), + perFrameStripFileInput: q("#perFrameStripFileInput"), + }, groups: { layoutGroup: q("#layoutGroup"), pageGridDetectionGroup: q("#pageGridDetectionGroup"), diff --git a/js/i18n.js b/js/i18n.js index 77d3ed0..43a58a9 100644 --- a/js/i18n.js +++ b/js/i18n.js @@ -33,6 +33,14 @@ const LOCALES = { dropNote: "Animation frames should be separated\nby crosses, dots, or empty gutters.", dropNoteMarkerless: "Animation frames should be separated\nby crosses, dots, or empty gutters.", dropNotePerFrame: "Drop one image per animation frame.\nThey'll be aligned into an animation.", + strip: { + heading: "Frames", + frameCount: "{count} images", + frameCountOne: "1 image", + addLabel: "Add images", + deleteLabel: "Remove frame {index}", + selectLabel: "Select frame {index}", + }, }, layout: { summary: "Layout", @@ -406,6 +414,14 @@ const LOCALES = { dropNote: "Los fotogramas de animación deben estar separados\npor cruces, puntos o canales vacíos.", dropNoteMarkerless: "Los fotogramas de animación deben estar separados\npor cruces, puntos o canales vacíos.", dropNotePerFrame: "Suelta una imagen por cada fotograma.\nSe alinearán en una animación.", + strip: { + heading: "Fotogramas", + frameCount: "{count} imágenes", + frameCountOne: "1 imagen", + addLabel: "Añadir imágenes", + deleteLabel: "Quitar fotograma {index}", + selectLabel: "Seleccionar fotograma {index}", + }, }, layout: { summary: "Diseño", @@ -777,6 +793,14 @@ const LOCALES = { dropNote: "I fotogrammi devono essere separati\nda croci, punti o spazi vuoti.", dropNoteMarkerless: "I fotogrammi devono essere separati\nda croci, punti o spazi vuoti.", dropNotePerFrame: "Trascina un'immagine per ogni fotogramma.\nVerranno allineate in un'animazione.", + strip: { + heading: "Fotogrammi", + frameCount: "{count} immagini", + frameCountOne: "1 immagine", + addLabel: "Aggiungi immagini", + deleteLabel: "Rimuovi fotogramma {index}", + selectLabel: "Seleziona fotogramma {index}", + }, }, layout: { summary: "Layout", @@ -1148,6 +1172,14 @@ const LOCALES = { dropNote: "フレームは区切ってください\n十字、点、または空のガターで。", dropNoteMarkerless: "フレームは区切ってください\n十字、点、または空のガターで。", dropNotePerFrame: "1フレームにつき1枚の画像をドロップ。\n自動で位置合わせしてアニメ化します。", + strip: { + heading: "フレーム", + frameCount: "{count} 枚", + frameCountOne: "1 枚", + addLabel: "画像を追加", + deleteLabel: "フレーム {index} を削除", + selectLabel: "フレーム {index} を選択", + }, }, layout: { summary: "レイアウト", @@ -1519,6 +1551,14 @@ const LOCALES = { dropNote: "动画帧之间应分隔\n用十字、圆点或空白间隔。", dropNoteMarkerless: "动画帧之间应分隔\n用十字、圆点或空白间隔。", dropNotePerFrame: "每帧拖入一张图片。\n它们将被对齐成动画。", + strip: { + heading: "帧", + frameCount: "{count} 张图片", + frameCountOne: "1 张图片", + addLabel: "添加图片", + deleteLabel: "移除第 {index} 帧", + selectLabel: "选择第 {index} 帧", + }, }, layout: { summary: "布局", @@ -1890,6 +1930,14 @@ const LOCALES = { dropNote: "動畫影格之間應分隔\n以十字、圓點或空白間隔。", dropNoteMarkerless: "動畫影格之間應分隔\n以十字、圓點或空白間隔。", dropNotePerFrame: "每格拖入一張圖片。\n它們將被對齊成動畫。", + strip: { + heading: "影格", + frameCount: "{count} 張圖片", + frameCountOne: "1 張圖片", + addLabel: "新增圖片", + deleteLabel: "移除第 {index} 格", + selectLabel: "選擇第 {index} 格", + }, }, layout: { summary: "版面", @@ -2261,6 +2309,14 @@ const LOCALES = { dropNote: "애니메이션 프레임은 구분되어야 합니다\n십자, 점 또는 빈 간격으로.", dropNoteMarkerless: "애니메이션 프레임은 구분되어야 합니다\n십자, 점 또는 빈 간격으로.", dropNotePerFrame: "프레임당 이미지 한 장을 드롭하세요.\n정렬되어 애니메이션이 됩니다.", + strip: { + heading: "프레임", + frameCount: "이미지 {count}장", + frameCountOne: "이미지 1장", + addLabel: "이미지 추가", + deleteLabel: "프레임 {index} 제거", + selectLabel: "프레임 {index} 선택", + }, }, layout: { summary: "레이아웃", @@ -2632,6 +2688,14 @@ const LOCALES = { dropNote: "Os quadros da animação devem ser\nseparados por cruzes, pontos\nou espaços vazios.", dropNoteMarkerless: "Os quadros da animação devem ser\nseparados por cruzes, pontos\nou espaços vazios.", dropNotePerFrame: "Solte uma imagem por quadro da animação.\nElas serão alinhadas em uma animação.", + strip: { + heading: "Quadros", + frameCount: "{count} imagens", + frameCountOne: "1 imagem", + addLabel: "Adicionar imagens", + deleteLabel: "Remover quadro {index}", + selectLabel: "Selecionar quadro {index}", + }, }, layout: { summary: "Layout", @@ -3003,6 +3067,14 @@ const LOCALES = { dropNote: "Animationsbilder sollten getrennt sein\ndurch Kreuze, Punkte oder leere Zwischenräume.", dropNoteMarkerless: "Animationsbilder sollten getrennt sein\ndurch Kreuze, Punkte oder leere Zwischenräume.", dropNotePerFrame: "Ein Bild pro Animationsbild ablegen.\nSie werden zu einer Animation ausgerichtet.", + strip: { + heading: "Einzelbilder", + frameCount: "{count} Bilder", + frameCountOne: "1 Bild", + addLabel: "Bilder hinzufügen", + deleteLabel: "Einzelbild {index} entfernen", + selectLabel: "Einzelbild {index} auswählen", + }, }, layout: { summary: "Layout", @@ -3374,6 +3446,14 @@ const LOCALES = { dropNote: "Klatki animacji powinny być oddzielone\nkrzyżykami, kropkami lub pustymi odstępami.", dropNoteMarkerless: "Klatki animacji powinny być oddzielone\nkrzyżykami, kropkami lub pustymi odstępami.", dropNotePerFrame: "Upuść jeden obraz na każdą klatkę.\nZostaną wyrównane w animację.", + strip: { + heading: "Klatki", + frameCount: "{count} obrazów", + frameCountOne: "1 obraz", + addLabel: "Dodaj obrazy", + deleteLabel: "Usuń klatkę {index}", + selectLabel: "Wybierz klatkę {index}", + }, }, layout: { summary: "Układ", @@ -3745,6 +3825,14 @@ const LOCALES = { dropNote: "Animasjonsruter bør være skilt\nmed kryss, prikker eller tomme mellomrom.", dropNoteMarkerless: "Animasjonsruter bør være skilt\nmed kryss, prikker eller tomme mellomrom.", dropNotePerFrame: "Slipp ett bilde per animasjonsrute.\nDe justeres til en animasjon.", + strip: { + heading: "Ruter", + frameCount: "{count} bilder", + frameCountOne: "1 bilde", + addLabel: "Legg til bilder", + deleteLabel: "Fjern rute {index}", + selectLabel: "Velg rute {index}", + }, }, layout: { summary: "Oppsett", @@ -4116,6 +4204,14 @@ const LOCALES = { dropNote: "Кадри анімації мають бути розділені\nхрестиками, крапками або порожніми проміжками.", dropNoteMarkerless: "Кадри анімації мають бути розділені\nхрестиками, крапками або порожніми проміжками.", dropNotePerFrame: "Перетягніть по одному зображенню на кадр.\nВони вирівняються в анімацію.", + strip: { + heading: "Кадри", + frameCount: "{count} зображень", + frameCountOne: "1 зображення", + addLabel: "Додати зображення", + deleteLabel: "Видалити кадр {index}", + selectLabel: "Вибрати кадр {index}", + }, }, layout: { summary: "Макет", @@ -4487,6 +4583,14 @@ const LOCALES = { dropNote: "Les images d’animation doivent être séparées\npar des croix, des points ou des gouttières vides.", dropNoteMarkerless: "Les images d’animation doivent être séparées\npar des croix, des points ou des gouttières vides.", dropNotePerFrame: "Déposez une image par image d’animation.\nElles seront alignées en une animation.", + strip: { + heading: "Images", + frameCount: "{count} images", + frameCountOne: "1 image", + addLabel: "Ajouter des images", + deleteLabel: "Retirer l’image {index}", + selectLabel: "Sélectionner l’image {index}", + }, }, layout: { summary: "Mise en page", diff --git a/js/load-controller.js b/js/load-controller.js index fa952f2..42626a0 100644 --- a/js/load-controller.js +++ b/js/load-controller.js @@ -47,7 +47,7 @@ export async function waitForNextPaint() { * @param {string} src * @returns {Promise} */ -function decodeImageElement(src) { +export function decodeImageElement(src) { return new Promise((resolve, reject) => { const image = new Image(); image.onload = () => resolve(image); diff --git a/js/per-frame-strip.js b/js/per-frame-strip.js new file mode 100644 index 0000000..1ceab39 --- /dev/null +++ b/js/per-frame-strip.js @@ -0,0 +1,368 @@ +/** + * Per-frame image strip UI (Phase 7). + * + * The strip is only meaningful in the per-frame alignment pipeline, where each uploaded image is one + * animation frame. It renders one thumbnail per `state.source.images[i]` and lets the user: + * - switch the active image (click) — UI navigation only, no reprocessing; + * - reorder frames (HTML5 drag-and-drop within the strip) — reprocesses (frame order changed); + * - delete a frame (× on hover) — releases that entry's blob URL / canvas / cached Mat and + * reprocesses with N-1 frames; + * - add more frames (the trailing `+` tile) — reuses the Phase 4 decode path and reprocesses. + * + * This module deliberately holds all strip rendering + event handling so app.js / ui-controls.js stay + * lean. It is wired in from ui-controls.js with the small set of callbacks it needs (active-image + * select, reprocess trigger, add-images load path) plus the shared `dom` / `state` handles. + */ +import { t } from "./i18n.js"; +import { + getActiveSourceImage, + releaseEntryRectifiedCache, +} from "./source-images.js"; + +/** @type {StripDeps | null} Bound dependencies, set once by attachPerFrameStrip. */ +let deps = null; + +/** Signature of the last render so re-renders only rebuild when images[]/activeIndex changed. */ +let lastRenderSignature = ""; + +/** Index of the thumbnail currently being dragged (reorder), or -1 when not dragging. */ +let dragSourceIndex = -1; + +/** + * @typedef {Object} StripDeps + * @property {import("./dom-state.js").dom} dom + * @property {import("./dom-state.js").state} state + * @property {(index:number) => void} setActiveImage Switch the active image (no reprocess). + * @property {() => void} reprocess Trigger a debounced reprocess (reorder/delete). + * @property {(files: File[]) => Promise} addImageFiles Decode + append images, then reprocess. + * @property {() => boolean} isPerFrameModeActive Whether the per-frame pipeline is currently active. + */ + +/** + * Wire the per-frame strip's add-images file input and remember the shared dependencies. + * + * Safe to call once during app startup. The strip itself is rendered lazily via `renderPerFrameStrip`. + * + * @param {StripDeps} boundDeps + * @returns {void} + */ +export function attachPerFrameStrip(boundDeps) { + deps = boundDeps; + const { dom } = deps; + const addInput = dom.perFrameStripFileInput; + if (addInput) { + addInput.addEventListener("change", () => { + const files = Array.from(addInput.files || []).filter(Boolean); + // Reset the input so picking the same file again still fires `change`. + addInput.value = ""; + if (files.length === 0) return; + void deps.addImageFiles(files); + }); + } +} + +/** + * Build a cheap signature describing the current strip contents so we only rebuild on real changes. + * + * @returns {string} + */ +function computeSignature() { + const { state } = deps; + const images = Array.isArray(state.source.images) ? state.source.images : []; + const parts = images.map((entry) => (entry && entry.canvas ? "1" : "0")); + return `${state.source.activeImageIndex}|${images.length}|${parts.join(",")}`; +} + +/** + * Re-render the strip if the per-frame mode visibility or the images[]/activeIndex changed. + * + * Hidden (and emptied) in markers / markerless modes so legacy flows never see strip markup. + * Idempotent: a no-op when neither the visibility nor the signature changed since the last render. + * + * @returns {void} + */ +export function renderPerFrameStrip() { + if (!deps) return; + const { dom, state, isPerFrameModeActive } = deps; + const panel = dom.perFrameStripPanel; + const container = dom.perFrameStrip; + if (!panel || !container) return; + + const active = isPerFrameModeActive(); + if (!active) { + // Leave no strip markup behind in legacy modes. + if (!panel.hidden || container.childElementCount > 0) { + panel.hidden = true; + container.replaceChildren(); + lastRenderSignature = ""; + } + return; + } + panel.hidden = false; + + const signature = computeSignature(); + if (signature === lastRenderSignature && container.childElementCount > 0) { + // Nothing visible changed (count + per-entry presence + activeIndex all match); avoid thrashing + // the DOM. Reorder/delete reset `lastRenderSignature` to "" first to force a rebuild, since a + // reorder can leave the signature unchanged while the visible order differs. + return; + } + lastRenderSignature = signature; + + const images = Array.isArray(state.source.images) ? state.source.images : []; + const activeIndex = state.source.activeImageIndex; + const fragment = document.createDocumentFragment(); + + images.forEach((entry, index) => { + fragment.appendChild(buildThumbnail(entry, index, index === activeIndex)); + }); + fragment.appendChild(buildAddTile()); + container.replaceChildren(fragment); + + // Frame-count readout (singular / plural). + const count = images.length; + if (dom.perFrameStripCount) { + dom.perFrameStripCount.textContent = + count === 1 ? t("photo.strip.frameCountOne") : t("photo.strip.frameCount", { count }); + } +} + +/** + * Build one thumbnail tile for a source entry. + * + * @param {import("./source-images.js").SourceImageEntry} entry + * @param {number} index + * @param {boolean} isActive + * @returns {HTMLElement} + */ +function buildThumbnail(entry, index, isActive) { + const tile = document.createElement("div"); + tile.className = "per-frame-thumb"; + tile.setAttribute("role", "listitem"); + tile.dataset.index = String(index); + tile.draggable = true; + if (isActive) tile.classList.add("is-active"); + tile.setAttribute("aria-label", t("photo.strip.selectLabel", { index: index + 1 })); + + const img = document.createElement("img"); + // Prefer a drag/display URL; fall back to drawing from the entry's canvas via a data URL is avoided + // for cost — the dragUrl (blob/demo) is always present for loaded entries. + if (entry && entry.dragUrl) { + img.src = entry.dragUrl; + } else if (entry && entry.canvas) { + try { + img.src = entry.canvas.toDataURL("image/jpeg", 0.5); + } catch { + /* tainted/empty canvas — leave src empty */ + } + } + img.alt = ""; + tile.appendChild(img); + + const number = document.createElement("span"); + number.className = "per-frame-thumb-number"; + number.textContent = String(index + 1); + tile.appendChild(number); + + const del = document.createElement("button"); + del.type = "button"; + del.className = "per-frame-thumb-delete"; + del.textContent = "×"; // × + del.setAttribute("aria-label", t("photo.strip.deleteLabel", { index: index + 1 })); + del.addEventListener("click", (event) => { + event.stopPropagation(); + deleteImageAt(index); + }); + tile.appendChild(del); + + tile.addEventListener("click", () => { + selectImageAt(index); + }); + + attachReorderHandlers(tile, index); + return tile; +} + +/** + * Build the trailing `+` tile that adds more images. + * + * @returns {HTMLElement} + */ +function buildAddTile() { + const tile = document.createElement("button"); + tile.type = "button"; + tile.className = "per-frame-thumb-add"; + tile.textContent = "+"; + tile.setAttribute("aria-label", t("photo.strip.addLabel")); + tile.addEventListener("click", () => { + deps.dom.perFrameStripFileInput?.click(); + }); + // Allow dropping additional images directly onto the add tile. + tile.addEventListener("dragover", (event) => { + if (event.dataTransfer?.types?.includes("Files")) { + event.preventDefault(); + tile.classList.add("is-drag-over"); + } + }); + tile.addEventListener("dragleave", () => { + tile.classList.remove("is-drag-over"); + }); + tile.addEventListener("drop", (event) => { + tile.classList.remove("is-drag-over"); + const files = Array.from(event.dataTransfer?.files || []).filter((file) => + String(file.type || "").startsWith("image/"), + ); + if (files.length === 0) return; + event.preventDefault(); + event.stopPropagation(); + void deps.addImageFiles(files); + }); + return tile; +} + +/** + * Wire HTML5 drag-and-drop reorder handlers on a thumbnail tile. Scoped to the strip: we only react + * to drags whose source index we recorded on `dragstart`. + * + * @param {HTMLElement} tile + * @param {number} index + * @returns {void} + */ +function attachReorderHandlers(tile, index) { + tile.addEventListener("dragstart", (event) => { + dragSourceIndex = index; + tile.classList.add("is-dragging"); + if (event.dataTransfer) { + event.dataTransfer.effectAllowed = "move"; + // Some browsers require data to be set for the drag to begin. + try { + event.dataTransfer.setData("text/plain", String(index)); + } catch { + /* ignore */ + } + } + }); + tile.addEventListener("dragend", () => { + dragSourceIndex = -1; + tile.classList.remove("is-dragging"); + }); + tile.addEventListener("dragover", (event) => { + // Only handle in-strip thumbnail reorders (a recorded source index). File drags fall through to + // the add tile / drop zone. + if (dragSourceIndex < 0) return; + event.preventDefault(); + if (event.dataTransfer) event.dataTransfer.dropEffect = "move"; + tile.classList.add("is-drag-over"); + }); + tile.addEventListener("dragleave", () => { + tile.classList.remove("is-drag-over"); + }); + tile.addEventListener("drop", (event) => { + tile.classList.remove("is-drag-over"); + if (dragSourceIndex < 0) return; + event.preventDefault(); + event.stopPropagation(); + const from = dragSourceIndex; + dragSourceIndex = -1; + reorderImages(from, index); + }); +} + +/** + * Select the active image at `index` (UI navigation; no reprocess). + * + * @param {number} index + * @returns {void} + */ +function selectImageAt(index) { + deps.setActiveImage(index); + renderPerFrameStrip(); +} + +/** + * Move the entry at `from` to occupy position `to`, keeping the same logical entry active, and + * reprocess because frame order changed. + * + * @param {number} from + * @param {number} to + * @returns {void} + */ +function reorderImages(from, to) { + const { state } = deps; + const images = state.source.images; + if (!Array.isArray(images)) return; + if (from === to || from < 0 || to < 0 || from >= images.length || to >= images.length) return; + + // Track which entry is currently active so the same logical image stays selected after reorder. + const activeEntry = getActiveSourceImage(state); + const [moved] = images.splice(from, 1); + images.splice(to, 0, moved); + + // Re-derive the active index from the moved entry's new position. + const newActiveIndex = activeEntry ? images.indexOf(activeEntry) : 0; + // Repoint legacy projections at the (unchanged logical) active entry without redundant work. + deps.setActiveImage(newActiveIndex >= 0 ? newActiveIndex : 0); + + // Force a rebuild even though signature length is unchanged (order changed but counts did not). + lastRenderSignature = ""; + renderPerFrameStrip(); + deps.reprocess(); +} + +/** + * Delete the entry at `index`: release its blob URL + canvas + cached rectified Mat, adjust the + * active index, and reprocess with N-1 frames. Deleting the last image returns to an empty state. + * + * @param {number} index + * @returns {void} + */ +function deleteImageAt(index) { + const { state } = deps; + const images = state.source.images; + if (!Array.isArray(images) || index < 0 || index >= images.length) return; + + const [removed] = images.splice(index, 1); + releaseEntry(removed); + + if (images.length === 0) { + // Empty state: clear the active index and the legacy projections so the app reads "no image". + state.source.activeImageIndex = 0; + state.source.image = null; + lastRenderSignature = ""; + renderPerFrameStrip(); + deps.reprocess(); + return; + } + + // Keep the active selection sensible: clamp to the new range, biasing toward the prior neighbor. + let nextActive = state.source.activeImageIndex; + if (index < nextActive) nextActive -= 1; + if (nextActive >= images.length) nextActive = images.length - 1; + if (nextActive < 0) nextActive = 0; + deps.setActiveImage(nextActive); + + lastRenderSignature = ""; + renderPerFrameStrip(); + deps.reprocess(); +} + +/** + * Release every owned resource of a removed entry: its blob URL, cached rectified Mat, and canvas. + * + * @param {import("./source-images.js").SourceImageEntry | null | undefined} entry + * @returns {void} + */ +function releaseEntry(entry) { + if (!entry) return; + if (entry.ownedObjectUrl) { + try { + URL.revokeObjectURL(entry.ownedObjectUrl); + } catch { + /* already revoked */ + } + entry.ownedObjectUrl = ""; + } + releaseEntryRectifiedCache(entry); + entry.image = null; + entry.canvas = null; +} diff --git a/js/ui-controls.js b/js/ui-controls.js index f9daad1..c055a99 100644 --- a/js/ui-controls.js +++ b/js/ui-controls.js @@ -5,6 +5,7 @@ * can supply callbacks without carrying the full DOM-listener implementation inline. */ import { t } from "./i18n.js"; +import { attachPerFrameStrip } from "./per-frame-strip.js"; /** * Wire a small header reset button without toggling the parent details element. @@ -601,6 +602,9 @@ function attachMarkerlessPhaseMetricToggles({ * endPostRotationScrub: () => void, * finishPostRotationScrubIfUnchanged: () => boolean, * commitActivePostRotationFromSlider: () => void, + * setActiveImage: (index:number) => void, + * isPerFrameModeActive: () => boolean, + * addPerFrameImages: (files: File[]) => Promise, * bumpFrameOutputEpoch: () => void, * setGeometryProcessingCursor: (active:boolean) => void, * cancelInFlightProcessing: () => void, @@ -672,6 +676,9 @@ export function attachUi({ endPostRotationScrub, finishPostRotationScrubIfUnchanged, commitActivePostRotationFromSlider, + setActiveImage, + isPerFrameModeActive, + addPerFrameImages, bumpFrameOutputEpoch, cancelInFlightProcessing, invalidateFrameCaches, @@ -707,6 +714,15 @@ export function attachUi({ makeLivePreviewDragCue(); makeGifImageDraggable(); + attachPerFrameStrip({ + dom, + state, + setActiveImage, + reprocess: scheduleProcess, + addImageFiles: addPerFrameImages, + isPerFrameModeActive, + }); + dom.dropZone.addEventListener("dragover", (event) => { event.preventDefault(); dom.dropZone.classList.add("dragging"); diff --git a/per_frame_pipeline_plan.md b/per_frame_pipeline_plan.md index 27bf2ec..d90faad 100644 --- a/per_frame_pipeline_plan.md +++ b/per_frame_pipeline_plan.md @@ -817,6 +817,101 @@ reorder, and delete. - Cross-strip drag (e.g. dragging a thumbnail out of the app). - Bulk operations (multi-select delete). +**As built (Phase 7 — COMPLETE):** + +*New module `js/per-frame-strip.js`* — exports `attachPerFrameStrip(deps)` and +`renderPerFrameStrip()`. +- `attachPerFrameStrip({ dom, state, setActiveImage, reprocess, addImageFiles, + isPerFrameModeActive })` binds deps in module scope (single instance) and wires the hidden + `#perFrameStripFileInput` `change` handler (the `+` tile's file picker). Called once from + `attachUi`. +- `renderPerFrameStrip()` rebuilds the strip. It hides + empties the panel (sets `panel.hidden = + true`, clears children, resets the cached signature) whenever `isPerFrameModeActive()` is false, so + markers/markerless modes never see strip markup. When active it diffs a cheap signature + (`activeImageIndex | length | per-entry-canvas-presence`) against the last render and early-returns + when nothing visible changed, to avoid DOM thrash/flicker. Reorder and delete reset the cached + signature to `""` first so a same-length reorder still rebuilds in the new order. +- Each thumbnail shows the entry image (`` from `entry.dragUrl`, `object-fit: cover` square + crop), the 1-based frame number, and a hover/focus delete (`×`) button. The trailing `+` tile opens + the file picker and also accepts image drops directly. Click selects active via the app's + `setActiveImage(index)` (NO reprocess). Reorder uses HTML5 DnD scoped to the strip (a module-level + `dragSourceIndex` recorded on `dragstart`; file drags fall through to the add tile). Reorder splices + `state.source.images[]`, keeps the same logical entry active (re-derives its new index via + `images.indexOf(activeEntry)` and calls `setActiveImage`), then calls `reprocess()`. Delete splices + the entry, releases it (see below), clamps the active index toward the prior neighbor, calls + `setActiveImage`, then `reprocess()`; deleting the last image nulls `state.source.image` and leaves + only the `+` tile (empty state). + +*Delete release path (no leaks):* the strip's `releaseEntry(entry)` revokes `entry.ownedObjectUrl`, +calls the existing `releaseEntryRectifiedCache(entry)` from `js/source-images.js` (frees a bare `Mat` +or a `{ visionMat, styledMat }` cache), and nulls `entry.image` / `entry.canvas`. Active index is +re-clamped via `setActiveSourceImage` (through `setActiveImage`) so it always stays in range. + +*Reprocess vs select:* selecting an image routes through the app's `setActiveImage` (Phase 5) → no +`scheduleProcess`. Reorder and delete both call the `reprocess` dep, which is the app's existing +`scheduleProcess` (debounced reprocess entry point). `scheduleProcess` no-ops when `state.source.image` +is null, so deleting the last frame does not reprocess (stale preview lingers until images are +re-added — acceptable empty state). + +*`js/app.js`:* added `addPerFrameImages(files)` — decodes each image via the exported +`decodeImageElement` (Phase 4), builds an entry with its own blob URL + source canvas via +`createSourceImageEntry` + `drawImageToCanvas`, pushes to `state.source.images`, forces per-frame mode +on (`forcePerFrameMode` + ticks the radio), adopts entry 0 as active via `setActiveImage` when the +strip started empty, then `syncAlignmentMarkerUi()` + `renderPerFrameStrip()` + `scheduleProcess(0)`. +`renderPerFrameStrip()` is now also called at the tail of `syncAlignmentMarkerUi()` (covers mode +switch / post-process visibility) and at the tail of `setActiveImage()` (active-highlight refresh). +`setActiveImage`, `isPerFrameModeActive`, and `addPerFrameImages` are threaded into `wireUiControls`. +`decodeImageElement` is now exported from `js/load-controller.js`. + +*`js/ui-controls.js`:* imports `attachPerFrameStrip` and calls it near the top of `attachUi`, passing +`reprocess: scheduleProcess` and `addImageFiles: addPerFrameImages`. The three new deps +(`setActiveImage`, `isPerFrameModeActive`, `addPerFrameImages`) were added to the `attachUi` JSDoc + +destructure. + +*`index.html`:* `#perFrameStripPanel` (a `