Skip to content

Evaluations by iteration#296

Open
FlxPo wants to merge 25 commits intomainfrom
evaluations-by-iteration
Open

Evaluations by iteration#296
FlxPo wants to merge 25 commits intomainfrom
evaluations-by-iteration

Conversation

@FlxPo
Copy link
Copy Markdown
Contributor

@FlxPo FlxPo commented Apr 2, 2026

Motivation

This PR refactors how transport costs and congestion are represented across GroupDayTrips iterations.

The previous implementation relied on TransportCostsAggregator, step-based resolution, and several dedicated snapshot classes. That made iteration-specific inspection, congestion-state persistence, and resume behavior hard to reason about. The goal of this refactor is to make asset_for_iteration(...) the consistent mental model everywhere: transport costs, graphs, congestion, and evaluation should all resolve through the same iteration-scoped asset composition pattern.

This PR also addresses a limitation of the previous design: TransportCostsAggregator was an InMemoryAsset, so the transport state needed to resume a run was not persisted on disk. With the new structure, the relevant state can be reconstructed from persisted iteration artifacts and OD flow assets.

See #296

Changes

Transport-cost architecture

  • Replaced TransportCostsAggregator with a first-class TransportCosts file asset.
  • Added CongestionStateManager to build and reload persisted congestion states from OD flow assets.
  • Removed internal snapshot-specific classes:
    • path-travel-cost snapshots
    • congested/contracted graph snapshots
    • detailed carpool travel-cost snapshots
  • Removed internal step-based transport-cost resolution in favor of iteration-based asset resolution.
  • Introduced asset_for_iteration(...) / get_for_iteration(...) as the canonical access pattern for iteration-specific transport state.
  • Refactored path and carpool travel costs to resolve congestion variants through reusable assets built from persisted flow assets.

GroupDayTrips execution and resume

  • Updated GroupDayTrips, Run, ModeSequences, and PlanUpdater to use iteration-aware transport-cost assets during execution and resume.
  • Simplified run state by removing explicit in-memory congestion-state storage from RunState.
  • Resume compatibility with older cached runs is not preserved for this refactor. Affected runs must be recomputed from scratch because the previous TransportCostsAggregator did not persist the required transport state to disk.

Evaluation access pattern

  • Updated migrated evaluation helpers to use iteration-aware access, defaulting to the last iteration of the run when no explicit iteration is provided.
  • This migration is not yet complete across all evaluations; only the migrated evaluation helpers follow the new iteration-aware pattern in this PR.

Tests and configuration

  • Updated tests to the new iteration-based API.
  • Added pytest marker registration in pyproject.toml.

Example (if relevant)

results = group_day_trips.weekday_run.results()

# Default: use the last iteration of the run
results.travel_costs(ref_costs)

# Explicitly inspect a specific iteration
results.travel_costs(ref_costs, iteration=2)

car_mode = next(
    mode for mode in results.modes
    if mode.inputs["parameters"].name == "car"
)

travel_costs = car_mode.inputs["travel_costs"]
congested_graph_asset = (
    travel_costs
    .asset_for_iteration(results.run, iteration=2)
    .inputs["congested_path_graph"]
)

AI-assisted contribution

  • AI used for substantial parts of this PR

If substantial, briefly describe:

  • Scope of AI-assisted content: propositions of architectural / naming / optimizations options to reach the desired PR result (ability to get assets artifacts for any given iteration), refactoring changes to implement these propositions.
  • What you reviewed or changed: all propositions were reviewed, challenged and changed to match the overall architecture and style of the package, and keep things as simple as possible.
  • How you validated it (tests, checks, manual verification): local unit and integration tests with pytest + tests on an actual mobility model of a small region.

Checklist

  • I have reviewed all code in this PR
  • I understand the code and can maintain it
  • I added or ran appropriate tests/checks for the changed behavior

FlxPo added 25 commits March 20, 2026 18:10
…ration so that travel costs recompute when a new GTFS feed is added + add a GTFS builder class + integration test for time varying GTFS parameter
… GTFS when testing for GTFS changes during a simulation
@FlxPo FlxPo self-assigned this Apr 2, 2026
@FlxPo FlxPo added the enhancement New feature or request label Apr 2, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 2, 2026

Codecov Report

❌ Patch coverage is 84.87435% with 319 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.38%. Comparing base (79769a7) to head (634a436).

Files with missing lines Patch % Lines
mobility/trips/group_day_trips/core/results.py 44.57% 143 Missing ⚠️
.../carpool/detailed/detailed_carpool_travel_costs.py 30.00% 28 Missing ⚠️
...ips/group_day_trips/plans/destination_sequences.py 86.52% 19 Missing ⚠️
mobility/transport/costs/path/path_travel_costs.py 85.14% 15 Missing ⚠️
mobility/transport/costs/transport_costs.py 89.09% 12 Missing ⚠️
...lity/trips/group_day_trips/core/group_day_trips.py 81.53% 12 Missing ⚠️
...ips/group_day_trips/iterations/iteration_assets.py 92.03% 9 Missing ⚠️
...bility/trips/group_day_trips/plans/plan_updater.py 90.90% 9 Missing ⚠️
...y/transport/modes/public_transport/gtfs_builder.py 93.57% 7 Missing ⚠️
...oup_day_trips/evaluation/car_traffic_evaluation.py 0.00% 7 Missing ⚠️
... and 21 more
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #296       +/-   ##
===========================================
+ Coverage   59.06%   74.38%   +15.32%     
===========================================
  Files         129      143       +14     
  Lines        6738     6392      -346     
===========================================
+ Hits         3980     4755      +775     
+ Misses       2758     1637     -1121     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@FlxPo FlxPo mentioned this pull request Apr 3, 2026
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant