Refactor get_trace and extend get_opt_trace_by_steps to MOO/constrained
#4884
+346
−129
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
We have a method
get_opt_trace_by_stepsthat was used extensively during our Ax 1.0 benchmarking campaign. It duplicates the basic logic ofget_tracebut differs in that it operates along(trial_index, MAP_KEY)pairs and respects ordering by timestamp (i.e. chronological order). However, it is limited to single-objective unconstrained problems, and our current needs (multi-objective and/or constrained) have outgrown it. We reconcile the two by extracting three core building blocks ofget_trace:_pivot_data_with_feasibility: Pivots data to wide format with feasibility information and metric completeness checks._compute_trace_values: Computes per-observation trace values (hypervolume for MOO, objective value for SOO), with cumulative best support._aggregate_and_cumulate_trace: Aggregates values by groups and computes the cumulative best across groups.These are implemented in a more general way that respects arbitrary groupings and orderings. We then refactor
get_trace(and its helpers_prepare_data_for_traceandget_trace_by_arm_pull_from_data) to use these building blocks, and leverage them inget_opt_trace_by_stepsto extend its support to multi-objective and constrained problems.Additionally:
get_opt_trace_by_stepsis preserved, which is critical for correct cumulative hypervolume computation (without this, observations would be processed in(trial_index, arm_name, MAP_KEY)order instead of chronological order).NotImplementedErrorchecks with actual MOO and constrained test cases that verify correctness of the new functionality.Differential Revision: D79581270