Skip to content

Improve image collection chaining#9168

Merged
lstein merged 6 commits into
invoke-ai:mainfrom
JPPhoto:image-collection-improvements
Jul 5, 2026
Merged

Improve image collection chaining#9168
lstein merged 6 commits into
invoke-ai:mainfrom
JPPhoto:image-collection-improvements

Conversation

@JPPhoto

@JPPhoto JPPhoto commented May 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

Give the Image Collection Primitive node the ability to chain that a normal Collection node has.

Related Issues / Discussions

QA Instructions

Try upgrading an old workflow that uses this node.

Also try dropping it into a new workflow and make sure you can chain as well as add new images to the collection. It should work with no inputs, one, and both.

Merge Plan

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@JPPhoto JPPhoto added backend PRs that change backend files frontend PRs that change frontend files labels May 14, 2026
@github-actions github-actions Bot added python PRs that change python files invocations PRs that change invocations python-tests PRs that change python tests labels May 14, 2026
@JPPhoto JPPhoto force-pushed the image-collection-improvements branch 5 times, most recently from e2021e9 to 0c29921 Compare May 15, 2026 21:23
@lstein lstein self-assigned this May 16, 2026
@JPPhoto JPPhoto force-pushed the image-collection-improvements branch 13 times, most recently from 81f9695 to 1de55b7 Compare May 22, 2026 03:37
@JPPhoto JPPhoto force-pushed the image-collection-improvements branch 2 times, most recently from 41007f4 to 9de21b9 Compare May 27, 2026 01:39
@JPPhoto JPPhoto force-pushed the image-collection-improvements branch 12 times, most recently from e0f65ec to ee18966 Compare July 1, 2026 11:10
@JPPhoto JPPhoto force-pushed the image-collection-improvements branch 2 times, most recently from 0d998c3 to 58ed149 Compare July 4, 2026 00:26
@JPPhoto JPPhoto force-pushed the image-collection-improvements branch from 58ed149 to 7ee2ca0 Compare July 5, 2026 12:06
@lstein

lstein commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Code review findings

Automated multi-agent review of this branch (8 finder angles, each candidate independently verified against the code). The chainable-collection design and the node-update migration are sound overall, but the migration has two paths where it destroys or hides user data, and the workflow-form / linear-view surface isn't migrated. Findings ranked by severity.

Data loss / hidden data

1. Migration trusts unvalidated edges — permanent data loss on load (validateWorkflow.ts:84)
connectedInputNames is built from the raw edges array, but edge validation runs later in the same function (lines ~160–247). A 1.0.1 workflow with direct images in collection plus an edge into it from a missing/uninstalled node (the classic shared-community-workflow case): migration sees "connected" → skips copying to images, wipes collection.value = [], and then edge validation deletes the edge. The user's image list is permanently destroyed and persisted on next save. Fix: validate edges first, or compute connectedness only from edges that survive validation.

2. graphToWorkflow bypasses the migration entirely (graphToWorkflow.ts:84)
It stamps nodes with the current template version while copying legacy collection values verbatim, so getNeedsUpdate is false and the migration never runs. Loading a pre-PR graph (image metadata without embedded workflow, or the paste-graph modal) yields a 1.0.2 node with collection.value = [imgs] — invisible and uneditable in the UI (connection-only field renders no widget), yet buildNodesGraph.ts:70 still serializes it and the backend concatenates it into the output. The node silently emits images the user cannot see or remove.

3. Workflow-form fields aren't remapped from collection to images (validateWorkflow.ts:282)
The form-element cleanup only removes elements whose field no longer exists — collection still exists — and the legacy exposedFields migration (lines ~251–279) even builds a fresh form element for the now connection-only field. Linear-view users see an empty image picker after upgrade (their images moved to the hidden images field); anything they re-add writes collection.value, which the backend concatenates with the hidden migrated images → duplicated/phantom images.

Behavioral regressions and in-scope bugs

4. Required→optional removes the loud failure (primitives.py:302)
Pre-PR, an image_collection node with neither connection nor value raised MissingInputException. Now invoke() returns collection=[], a downstream iterate node prepares zero iterations, and the session completes "successfully" with no outputs and no error.

5. Shadowed direct value erased when collection is validly connected (nodeUpdate.ts:53)
Pre-PR the field was Input.Any: a direct value hidden behind a connection was retained and restored on disconnect (nodesSlice never clears values on connect). The migration wipes it. This looks deliberate (the test names the value 'stale') and moving it to images would change execution output — but it's silent, permanent-on-save loss; consider surfacing a warning.

6. Catch swallows non-NodeUpdateError and still shows the success toast (UpdateNodesButton.tsx:51)
A node version like "1e1.0.0" passes the loose zSemVer coercion but makes compare-versions' satisfies() throw a plain Error inside getMayUpdateNode; it's swallowed, unableToUpdateCount stays 0, and "all nodes updated" fires anyway. The sibling call site in validateWorkflow.ts catches everything, so the two paths diverge.

7. Copy-paste bug in the touched function (validateWorkflow.ts:235)
The deleted-invalid-edge warning builds its target descriptor from edge.source instead of edge.target (`${edge.source}.${edge.targetHandle}`), misidentifying which node lost the connection. Pre-existing, but cheap to fix here.

Design hardening (latent, cheap to fix now)

8. Migration is version-blind (nodeUpdate.ts:37)
Gated on node type only — and updateNode stamps clone.data.version = template.version before calling it, so it can't check the source version even if it wanted to. Any future 1.0.3 bump re-runs the migration on already-migrated 1.0.2 nodes (which can carry collection values via finding 2). Version-blind migrations don't compose.

9. Missing options.connectedInputNames means "nothing connected" (nodeUpdate.ts:50)
The two-arg updateNode(node, template) form still type-checks (the first unit test uses it). A future caller — e.g. a per-node "update this node" action; getNeedsUpdate is already consumed per-node elsewhere — would silently promote a stale shadowed collection value into the direct images field, appending phantom images at runtime. Passing edges (or keying off the template's input kind) would make the wrong call unrepresentable.

10. Duplicated edge-scan, rebuilt per node (UpdateNodesButton.tsx:39, validateWorkflow.ts:84)
The identical 5-line new Set(edges.flatMap(...)) lives verbatim at both call sites, inside each per-node loop (O(nodes × edges)). Extract a getConnectedInputNames(nodeId, edges) helper next to updateNode so the connectedness predicate can't drift between the update-all-button and workflow-load paths.

Minor (cut by the finding cap)

  • The test fixtures in nodeUpdate.test.ts invert the real parsed template defaults (new collection fixture: default: [] vs the pipeline's undefined; old fixture: undefined/required: false vs the real []/required: true), so the tests exercise template shapes production never generates. Relatedly, migrated nodes serialize collection: [] while fresh nodes omit the key.
  • Backend: Optional[...] default=None + or [] diverges from every sibling collection primitive and from CollectInvocation, which uses exactly list[Any] = InputField(default=[], input=Input.Connection). default=[] non-Optional validates identically and drops the | null noise from the schema.

🤖 Generated with Claude Code

@lstein

lstein commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

See above for Claude's code review. I performed functional testing and the image collection chaining works as advertised however I didn't have suitable preexisting workflows to functionally test the migration issues identified in the review.

@JPPhoto

JPPhoto commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator Author

@lstein First, a note about the loud failure comment: yes, this intentionally turns the former missing-input failure into a current valid case. An empty image collection is now supported and outputs an empty collection.

Next, A summary of the latest changes:

  • Addressed invalid-edge data loss in invokeai/frontend/web/src/features/nodes/util/workflow/validateWorkflow.ts: edges are validated before node updates, and migration connectedness now uses only surviving valid edges.

  • Addressed graph import migration in invokeai/frontend/web/src/features/nodes/util/workflow/graphToWorkflow.ts: legacy collection graph values now move to visible images when unconnected.

  • Addressed form/linear-view migration in invokeai/frontend/web/src/features/nodes/util/workflow/validateWorkflow.ts: legacy collection field identifiers are remapped to images.

  • Addressed hidden-value loss in invokeai/frontend/web/src/features/nodes/util/node/nodeUpdate.ts: validly connected legacy collection values are preserved instead of erased.

  • Addressed update button false-success behavior in invokeai/frontend/web/src/features/nodes/components/flow/panels/TopPanel/UpdateNodesButton.tsx: all update failures now count as failures.

  • Addressed version-blind and optional-call migration risks in invokeai/frontend/web/src/features/nodes/util/node/nodeUpdate.ts: migration is source-version gated, and updateNode() now requires connected-input context.

  • Addressed duplicate connected-input scans by adding getConnectedInputNames() in invokeai/frontend/web/src/features/nodes/util/node/nodeUpdate.ts.

  • Added targeted tests in invokeai/frontend/web/src/features/nodes/util/workflow/validateWorkflow.test.ts, invokeai/frontend/web/src/features/nodes/util/workflow/graphToWorkflow.test.ts, and invokeai/frontend/web/src/features/nodes/util/node/nodeUpdate.test.ts.

@lstein

lstein commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Follow-up: version gate in the migration misses 1.0.0 nodes

Thanks for the fixes — re-reviewed a9f2dd05d9/b9c84c7bcc and the migration-path issues are resolved (all 15 tests pass locally). One gap remains in the new version gate:

migrateImageCollectionInputValues now skips the migration unless the source version is exactly 1.0.1:

if (options.sourceVersion && options.sourceVersion !== '1.0.1') {
  return;
}

But image_collection shipped at 1.0.0 for a while (added in d9148fb619, bumped to 1.0.1 in 29b04b7e83), and 1.0.0 nodes still pass getMayUpdateNode (satisfies('1.0.0', '^1')). Verified with a repro test: loading a workflow containing a 1.0.0 image_collection node with direct collection values produces a 1.0.2 node whose values are stranded on the now connection-only collection field — invisible and uneditable in the UI, yet still serialized into the graph and concatenated into the output at runtime. This is exactly the hidden-phantom-images failure the migration exists to prevent.

Suggested fix — gate on "source predates 1.0.2" instead of an exact match, e.g. using compare from compare-versions:

if (options.sourceVersion && compare(options.sourceVersion, '1.0.2', '>=')) {
  return;
}

This keeps the intended protection (future 1.0.3+ bumps won't re-run the migration on already-migrated nodes) while covering pre-1.0.1 workflows.

🤖 Generated with Claude Code

@JPPhoto

JPPhoto commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator Author

@lstein Fixed, good catch. Ready for approval.

@lstein lstein left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as advertised.

@lstein lstein enabled auto-merge (squash) July 5, 2026 18:29
@lstein lstein merged commit 69c5882 into invoke-ai:main Jul 5, 2026
17 checks passed
@JPPhoto JPPhoto deleted the image-collection-improvements branch July 5, 2026 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.14.x backend PRs that change backend files frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files python-tests PRs that change python tests

Projects

Status: 6.14.x Theme: USER EXPERIENCE

Development

Successfully merging this pull request may close these issues.

2 participants