Write modules.json incrementally during installation

openspec/changes/archive/2026-03-03-install-test-infrastructure/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-03-03

openspec/changes/archive/2026-03-03-install-test-infrastructure/design.md

@@ -0,0 +1,58 @@
+## Context
+
+`install_modules_with_installer` is the core function that:
+1. Iterates through `InstallGraph` nodes in topological order
+2. Calls the installer for each missing module
+3. Updates `is_installed` flag based on success/failure
+4. Writes `modules.json` after each module
+
+Currently untestable because `InstallGraph` requires a real `Release` from Unity's API.
+
+## Goals / Non-Goals
+
+**Goals:**
+- Enable unit testing of `install_modules_with_installer`
+- Test error collection and continuation behavior
+- Test incremental `modules.json` writes
+- Keep test fixtures minimal and maintainable
+
+**Non-Goals:**
+- Mocking the entire Unity API
+- Testing actual download/install behavior (that's integration testing)
+- Changing production code structure significantly
+
+## Decisions
+
+### 1. Create Release from JSON fixtures
+
+**Decision**: Use JSON deserialization to create test `Release` instances.
+
+**Rationale**: `Release`, `Download`, and `Module` all derive `Deserialize`. We can create minimal JSON fixtures that produce valid `InstallGraph` instances without hitting Unity's API.
+
+### 2. Test fixture location
+
+**Decision**: Place fixtures inline in test code as `const` strings or in a `test_fixtures` module.
+
+**Rationale**: Keeps tests self-contained and readable. External fixture files add complexity for minimal benefit.
+
+### 3. Use existing MockModuleInstaller
+
+**Decision**: The `MockModuleInstaller` already exists with the right interface. Wire it up to actual tests.
+
+**Rationale**: No new abstractions needed - just connect existing pieces.
+
+## Risks / Trade-offs
+
+**[Risk] Fixture maintenance**: If `Release` structure changes, fixtures break → Mitigation: Minimal fixtures with only required fields, rely on `#[serde(default)]`.
+
+**[Risk] Test coverage gaps**: Unit tests can't catch all integration issues → Mitigation: These complement, not replace, real installation testing.
+
+## Future Improvements
+
+### Extract test utilities to separate crate
+
+**When to consider**: If test fixtures and mocks grow significantly, or if multiple crates need to share test infrastructure.
+
+**Approach**: Create a `uvm-test-utils` crate with test fixtures and mocks. To avoid circular dependencies, the `ModuleInstaller` trait would need to move to a lower-level crate (e.g., `uvm-install-core` or `uvm_install_graph`), or use a feature flag approach where `uvm_install` exposes test utilities behind a `test-utils` feature.
+
+**Current state**: Test utilities live inline in `uvm_install/src/lib.rs` within `#[cfg(test)]` modules. This is sufficient for the current scope and avoids premature abstraction.

openspec/changes/archive/2026-03-03-install-test-infrastructure/proposal.md

@@ -0,0 +1,26 @@
+## Why
+
+The `install_modules_with_installer` function in `uvm_install` cannot be properly unit tested. It requires an `InstallGraph` which is built from Unity release data, making it difficult to test error handling, incremental state updates, and partial failure scenarios without real installations.
+
+## What Changes
+
+- Add test fixture helpers to create mock `Release` and `Module` data via JSON deserialization
+- Add a test helper module in `uvm_install` for constructing test `InstallGraph` instances
+- Write integration tests for `install_modules_with_installer` using `MockModuleInstaller`
+- Test the actual production code paths: iteration, error collection, state updates, and incremental writes
+
+## Capabilities
+
+### New Capabilities
+
+- `install-graph-test-helpers`: Test utilities for creating mock `InstallGraph` instances from JSON fixtures, enabling unit tests for installation logic
+
+### Modified Capabilities
+
+None
+
+## Impact
+
+- **Code**: `uvm_install/src/lib.rs` (test module), possibly `uvm_install_graph` if we need to expose internals
+- **Dependencies**: None - uses existing `serde_json` for fixture deserialization
+- **Tests**: Adds meaningful tests for `install_modules_with_installer` covering success, failure, and partial failure scenarios

openspec/changes/archive/2026-03-03-install-test-infrastructure/specs/install-graph-test-helpers/spec.md

@@ -0,0 +1,63 @@
+## ADDED Requirements
+
+### Requirement: Test helpers can create InstallGraph from minimal data
+
+The test module SHALL provide helpers to create `InstallGraph` instances from minimal JSON fixtures without requiring real Unity API data.
+
+#### Scenario: Create graph with single module
+
+- **WHEN** a test creates a Release fixture with one module
+- **THEN** an InstallGraph can be built from it
+- **AND** the graph contains the expected module node
+
+#### Scenario: Create graph with multiple modules
+
+- **WHEN** a test creates a Release fixture with modules A, B, C
+- **THEN** an InstallGraph can be built containing all three modules
+- **AND** iteration yields modules in topological order
+
+### Requirement: MockModuleInstaller controls per-module success/failure
+
+The test module SHALL provide a `MockModuleInstaller` that implements `ModuleInstaller` trait and allows configuring which modules succeed or fail.
+
+#### Scenario: Module not in fail set succeeds
+
+- **WHEN** MockModuleInstaller has fail_modules = {"ios"}
+- **AND** install_module is called for "android"
+- **THEN** the call returns Ok(())
+
+#### Scenario: Module in fail set fails
+
+- **WHEN** MockModuleInstaller has fail_modules = {"ios"}
+- **AND** install_module is called for "ios"
+- **THEN** the call returns Err(InstallError::InstallFailed)
+
+### Requirement: Tests can verify installation behavior via install_modules_with_installer
+
+Tests SHALL call `install_modules_with_installer` directly with MockModuleInstaller to test the production loop logic.
+
+#### Scenario: Single module fails, others continue
+
+- **WHEN** MockModuleInstaller is configured to fail module "ios"
+- **AND** install_modules_with_installer is called with graph containing [android, ios, webgl]
+- **THEN** android succeeds and is marked installed in modules list
+- **AND** ios fails and remains not installed
+- **AND** webgl succeeds and is marked installed (installation continued past failure)
+- **AND** the returned errors vector contains the ios failure
+
+#### Scenario: Multiple modules fail
+
+- **WHEN** MockModuleInstaller is configured to fail modules "ios" and "webgl"
+- **AND** install_modules_with_installer is called
+- **THEN** errors for both ios and webgl are returned
+- **AND** only android is marked installed in modules list
+
+### Requirement: Tests verify modules.json is written after each module
+
+The test module SHALL verify that `modules.json` is written incrementally during installation.
+
+#### Scenario: State persisted after each module
+
+- **WHEN** install_modules_with_installer processes 3 modules
+- **THEN** modules.json exists after the function returns
+- **AND** the file reflects the final installation state

openspec/changes/archive/2026-03-03-install-test-infrastructure/tasks.md

@@ -0,0 +1,28 @@
+## 1. Create Test Fixtures
+
+- [x] 1.1 Create helper function to build minimal `Release` JSON for testing
+- [x] 1.2 Create helper function to build `Module` JSON with configurable id and properties
+- [x] 1.3 Verify fixtures can be deserialized into valid `Release` instances
+
+## 2. Wire Up InstallGraph for Testing
+
+- [x] 2.1 Create helper to build `InstallGraph` from test `Release`
+- [x] 2.2 Mark all modules as `Missing` status for installation testing
+- [x] 2.3 Verify graph iteration yields expected modules
+
+## 3. MockModuleInstaller for Controlled Success/Failure
+
+- [x] 3.1 Implement `MockModuleInstaller` with `fail_modules: HashSet<String>` to control which modules fail
+- [x] 3.2 MockModuleInstaller returns `Ok(())` for modules not in fail set
+- [x] 3.3 MockModuleInstaller returns `Err(InstallError::InstallFailed)` for modules in fail set
+- [x] 3.4 Add tracking to record which modules were attempted (install order)
+
+## 4. Test `install_modules_with_installer` Directly
+
+Tests call `install_modules_with_installer` directly with MockModuleInstaller to bypass RealModuleInstaller.
+
+- [x] 4.1 Test: all modules succeed - MockModuleInstaller with empty fail set, verify all marked installed
+- [x] 4.2 Test: single module fails - fail set contains "ios", verify android/webgl succeed, ios fails
+- [x] 4.3 Test: multiple modules fail - fail set contains "ios" and "webgl", verify errors collected
+- [x] 4.4 Test: verify modules.json reflects correct state (installed modules true, failed modules false)
+- [x] 4.5 Test: verify install continues after failure (all modules attempted, not just up to first failure)

openspec/changes/archive/2026-03-03-modules-json-sync/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-03-03

openspec/changes/archive/2026-03-03-modules-json-sync/design.md

@@ -0,0 +1,64 @@
+## Context
+
+Currently, `modules.json` is written once at the end of `Installer::install()` (lib.rs:363). The installation flow is:
+
+1. Build dependency graph from Unity release
+2. Call `install_module_and_dependencies()` which loops through modules
+3. Each module: download → create installer → install
+4. After ALL modules succeed, update `is_installed` flags and write `modules.json`
+
+If any module fails in step 3, the function returns an error immediately and `modules.json` is never written. Physically installed modules are not reflected in the state file.
+
+## Goals / Non-Goals
+
+**Goals:**
+- Write `modules.json` after each module installation completes (success or failure)
+- Maintain accurate installation state even when individual modules fail
+- Continue installing remaining modules when one fails (collect errors)
+
+**Non-Goals:**
+- Changing the modules.json file format
+- Adding retry logic for failed modules
+- Parallel module installation (current sequential flow is maintained)
+
+## Decisions
+
+### 1. Incremental Write Strategy
+
+**Decision**: Write `modules.json` after each module completes installation.
+
+**Alternatives considered**:
+- Write on error only: Simpler but loses sync on crash
+- Write at checkpoints: More complex, unclear what checkpoints mean
+
+**Rationale**: Writing after each module is simple, ensures state is always current, and the performance cost is negligible (one small file write per module).
+
+### 2. Error Handling Approach
+
+**Decision**: Collect all module installation errors and continue with remaining modules.
+
+**Alternatives considered**:
+- Fail fast (current behavior): Loses progress information
+- Interactive mode asking user to skip/retry: Too complex for CLI tool
+
+**Rationale**: Users can see what succeeded and retry just the failed modules. This matches user expectations for package managers.
+
+### 3. State Initialization
+
+**Decision**: Load or create `modules.json` before the installation loop starts, then mutate and write incrementally.
+
+**Rationale**: The modules list needs to exist before we can update individual module states. Current code already handles this (lines 340-354), we just need to move the write into the loop.
+
+### 4. Installation Reference
+
+**Decision**: Create the `UnityInstallation` handle before the installation loop instead of after.
+
+**Rationale**: We need the installation handle to call `write_modules()` inside the loop. Currently it's created at line 339 after installation completes. Moving it earlier means we need to handle the case where the base directory doesn't exist yet - we can create the directory first if needed.
+
+## Risks / Trade-offs
+
+**[Risk] More disk I/O** → Mitigation: One small JSON write per module is negligible compared to module download/install time.
+
+**[Risk] Partial state on crash during write** → Mitigation: File writes are typically atomic on modern filesystems. Could add write-to-temp-then-rename pattern if needed.
+
+**[Risk] Installation continues despite failures** → Mitigation: Clearly report all failures at the end. Users who want fail-fast can use existing behavior via a flag (future work).

openspec/changes/archive/2026-03-03-modules-json-sync/proposal.md

@@ -0,0 +1,26 @@
+## Why
+
+When a module installation fails mid-way, `modules.json` is never written because it only gets written once at the end of a successful installation. This leaves the installation in an inconsistent state where physically installed modules aren't reflected in `modules.json`, causing uvm to misreport installation status.
+
+## What Changes
+
+- Write `modules.json` incrementally after each module finishes installing (success or failure)
+- Track partial installation state so users can see what actually installed
+- Handle module installation errors gracefully - continue with remaining modules and update state accordingly
+
+## Capabilities
+
+### New Capabilities
+
+- `incremental-modules-sync`: Write modules.json after each module installation step completes, ensuring installation state is always persisted to disk
+
+### Modified Capabilities
+
+None - this change modifies implementation details within the existing installation flow, not external requirements.
+
+## Impact
+
+- **Code**: `uvm_install/src/lib.rs` (installation flow), `unity-hub/src/unity/installation.rs` (write_modules)
+- **Behavior**: modules.json will be written multiple times during installation instead of once at the end
+- **Reliability**: Partial installations will have accurate state, improving recovery from failures
+- **Performance**: Minimal - one small file write per module installed

openspec/changes/archive/2026-03-03-modules-json-sync/specs/incremental-modules-sync/spec.md

@@ -0,0 +1,50 @@
+## ADDED Requirements
+
+### Requirement: System writes modules.json after each module installation
+
+The system SHALL write `modules.json` to disk after each module completes installation, updating the `is_installed` flag for that module.
+
+#### Scenario: Successful module installation updates state
+
+- **WHEN** a module installation completes successfully
+- **THEN** the system writes `modules.json` with that module's `is_installed` set to `true`
+
+#### Scenario: Failed module installation updates state
+
+- **WHEN** a module installation fails
+- **THEN** the system writes `modules.json` with that module's `is_installed` remaining `false`
+- **AND** the system continues installing remaining modules
+
+### Requirement: Installation continues despite individual module failures
+
+The system SHALL continue installing remaining modules when one module fails, collecting all errors for final reporting.
+
+#### Scenario: Multiple modules with one failure
+
+- **WHEN** the user installs modules A, B, and C
+- **AND** module B fails during installation
+- **THEN** the system installs module A successfully
+- **AND** the system attempts module B and records the failure
+- **AND** the system continues to install module C
+- **AND** the system reports module B's failure at the end
+
+#### Scenario: All errors reported at completion
+
+- **WHEN** multiple modules fail during installation
+- **THEN** the system reports all failures after attempting all modules
+- **AND** the return value indicates installation had failures
+
+### Requirement: modules.json reflects accurate installation state
+
+The system SHALL ensure `modules.json` accurately reflects which modules are physically installed at all times during and after installation.
+
+#### Scenario: Partial installation state is accurate
+
+- **WHEN** installation is interrupted after module A succeeds but before module B completes
+- **THEN** `modules.json` shows module A with `is_installed: true`
+- **AND** `modules.json` shows module B with `is_installed: false`
+
+#### Scenario: State is queryable immediately after write
+
+- **WHEN** a module installation completes and `modules.json` is written
+- **THEN** running `uvm list` or similar commands shows the updated state

openspec/changes/archive/2026-03-03-modules-json-sync/tasks.md

@@ -0,0 +1,29 @@
+## 1. Refactor Installation Setup
+
+- [x] 1.1 Move UnityInstallation creation before the installation loop in `Installer::install()`
+- [x] 1.2 Initialize modules list before installation loop (load existing or create from release)
+- [x] 1.3 Ensure base directory exists before creating UnityInstallation handle
+
+## 2. Implement Incremental State Updates
+
+- [x] 2.1 Refactor `install_module_and_dependencies` to accept mutable modules list and installation handle
+- [x] 2.2 Update module's `is_installed` flag after each successful installation
+- [x] 2.3 Call `write_modules()` after each module completes (success or failure)
+
+## 3. Implement Error Collection
+
+- [x] 3.1 Change `install_module_and_dependencies` to collect errors instead of returning early
+- [x] 3.2 Return collected errors after all modules are attempted
+- [x] 3.3 Update `Installer::install()` to handle multiple errors and report them
+
+## 4. Testing
+
+- [x] 4.1 Add test for incremental modules.json updates during installation
+- [x] 4.2 Add test for partial failure scenario (some modules fail, others succeed)
+- [x] 4.3 Verify modules.json state matches physical installation after partial failure
+
+## 5. Multi-Error Reporting
+
+- [x] 5.1 Add `MultipleInstallFailures` variant to `InstallError` that holds a `Vec<InstallError>`
+- [x] 5.2 Update `Installer::install()` to return `MultipleInstallFailures` when multiple modules fail
+- [x] 5.3 Implement `Display` for the new variant to show all failures