Fix minor bugs across api, benchmark, generators, metrics, runners, and other modules #4899
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Fix multiple bugs in ax/core and ax/orchestration: - `Arm.md5hash` hashes unconverted parameters instead of the numpy-type-converted `new_parameters` (core/arm.py:94) - `GeneratorRun.clone()` doesn't copy `gen_metadata` or `candidate_metadata_by_arm_signature`, causing mutations on the clone to affect the original (core/generator_run.py:304-346) - Module-level dict mutation in `OutcomeConstraint`: writing `fmt_data["name"] = ...` mutates the shared module-level dicts `UPPER_BOUND_MISMATCH` / `LOWER_BOUND_THRESHOLD` etc. (core/outcome_constraint.py:123,159) - Typo in method name: `is_reconverable_fetch_e` → `is_recoverable_fetch_e` (core/metric.py:168, orchestration/orchestrator.py:2098) - `force_refit` parameter hardcoded to True instead of using the passed argument (orchestration/orchestrator.py:1230) - Grammar fix: "values is better" → "values are better" (core/outcome_constraint.py) - Missing spaces between concatenated strings (core/parameter.py, core/parameter_constraint.py, core/experiment.py) Differential Revision: D92879435
Summary: Fix multiple bugs in ax/storage: - Missing `raise` before `SQADecodeError` silently ignores JSON mismatches during `copy_db_ids` (sqa_store/utils.py:114) - `int(None)` crash in SQA encoder: `metric_registry.get()` can return None, but the result was immediately passed to `int()` before the None check could run (sqa_store/encoder.py:393) - `AttributeError` on None kwargs in JSON decoder: `kwargs.pop()` was called before the `if kwargs is not None` guard (json_store/decoder.py:1092) - `decode_args_list` length mismatch in SQA save: used `len(trials)` instead of `len(trials_to_reduce_state)` and `[...] * len(trials)` instead of a single-element list (sqa_store/save.py:245,254) - SQLAlchemy filter no-op: `SQAExperiment.id is not None` uses Python identity check instead of SQLAlchemy's `.isnot(None)` (sqa_store/load.py:760) - Inconsistent threshold in error message: says "> 10" but the actual check is `> 15` (sqa_store/utils.py:77) - Garbled error message in with_db_settings_base.py: sentence fragments in wrong order (sqa_store/with_db_settings_base.py:106) - Doubled path `ax/ax/storage` in error message (sqa_store/encoder.py:131) - Backslash line-continuation inside string literal causes 16 extra spaces in error message (sqa_store/encoder.py:654) - Missing space between concatenated strings (sqa_store/db.py) - Typo "SCalarized" → "Scalarized" (sqa_store/encoder.py) - Missing space in SKIP_ATTRS_ERROR_SUFFIX (sqa_store/utils.py) Differential Revision: D92879499
Summary: Fix multiple bugs in ax/generation_strategy: - `min_trials_observed or ceil(...)` fails when `min_trials_observed=0` is explicitly passed: Python truthiness treats 0 as falsy, so the explicit value is discarded in favor of the default. Changed to `is None` check (dispatch_utils.py:61,127) - `max_parallelism_override and max_parallelism_cap` truthiness check misses explicit 0 values for the same reason. Changed to `is not None` checks (dispatch_utils.py:441) - Wrong variable in dedup check: `ObservationFeatures` object compared against list of `TParameterization` dicts instead of comparing `o.parameters` (external_generation_node.py:199) - Generation node setting wrong `_previous_node_name`: was setting it to the node being transitioned *to* instead of the node being transitioned *from* (generation_strategy.py:546) - Grammar fixes: "you are should" → "you should", "more is are available" → "more data is available" (generation_strategy.py, transition_criterion.py) Differential Revision: D92879532
Summary: Fix multiple bugs in ax/adapter: - `IntRangeToChoice` transform using index instead of value for `target_value`: `next(i for i, v in enumerate(values) if v == p.target_value)` returns the index, but the target_value should remain the value itself since ChoiceParameter stores actual values (transforms/int_range_to_choice.py:70) - Wrong exception type caught in `relativize.py`: `list.index()` raises `ValueError`, not `IndexError` or `StopIteration` (transforms/relativize.py:338) - Error message says "True" when the value is actually False: `use_model_predictions` is False in the branch where the error is raised (adapter_utils.py) - Missing space between concatenated strings (torch.py, adapter_utils.py) Differential Revision: D92879576
Summary: Fix multiple bugs in ax/service: - `max_parallelism or num_trials` fails when `max_parallelism` is 0: Python truthiness treats 0 as falsy, so an explicit 0 is replaced with `num_trials`. Changed to `is not None` check (ax_client.py) - `_is_all_noiseless` operator precedence + NaN comparison always returning False: `(sems == 0) | sems == nan` parses as `((sems == 0) | sems) == nan`, and NaN never equals anything. Fixed by using `.isna()` (utils/best_point.py:649) - Removed unused `from numpy import nan` import after the `_is_all_noiseless` fix (utils/best_point.py) - Typos: "exhaused" → "exhausted" (managed_loop.py), "Hieararchical" → "Hierarchical" (utils/instantiation.py) - Missing spaces between concatenated strings (utils/instantiation.py, utils/report_utils.py, ax_client.py) - Grammar fix: "by setting passing" → "by passing" (ax_client.py) Differential Revision: D92879656
Summary: Fix multiple bugs in ax/analysis and ax/plot: - De Morgan's law error in scatter NaN guard: `not isnan(x) or not isnan(y)` should be `not isnan(x) and not isnan(y)` — the original condition is true when *either* value is non-NaN, but both must be non-NaN to draw crosshairs (analysis/plotly/scatter.py:528) - Wrong variable `rel` instead of `rel_x`/`rel_y` for scatter axis titles: both axes used the same `rel` variable instead of the axis-specific ones (plot/scatter.py:523,528) - `trial_index=0` treated as falsy: `if trial_index` skips trial 0, changed to `if trial_index is not None` (plot/pareto_utils.py:390) - Double period in arm_effects title when trial_index is not None (analysis/plotly/arm_effects.py) - Missing spaces between concatenated strings (analysis/healthcheck/regression_detection_utils.py, analysis/utils.py) Differential Revision: D92879692
Summary: Fix multiple bugs in ax/utils: - Claude fix: Wrong variable `other_val` used twice instead of `one_val` and `other_val` in equality comparison: `not hasattr(other_val, "model")` checked `other_val` twice instead of checking `one_val` first (common/equality.py:217) - Manual fix: `_model` attribute has been deprecated. Just removed this special case. If it was needed, we would've fixed it and replaced it with the `generator` attribute already. - Missing `abs()` in datetime comparison: `(dt1 - dt2).total_seconds() < 1.0` can be negative when dt2 > dt1, allowing far-apart datetimes to compare as equal (common/equality.py:107) - `next` used instead of `continue` in statstools loop: bare `next` is a no-op expression (evaluates to the builtin `next` function and discards it), so the loop body runs even when it should be skipped (stats/statstools.py:204) - Missing `KENDALL_TAU_RANK_CORRELATION` in metric directions dict (stats/model_fit_stats.py:344) - Missing spaces between concatenated strings (common/testutils.py) Differential Revision: D92879733
…nd other modules
Summary:
Fix bugs in smaller modules with 1-2 files each:
- Client JSON snapshot crash when no GenerationStrategy exists: `"generation_strategy" in snapshot` is True for the key with a None value, causing a decode crash. Changed to `snapshot.get("generation_strategy") is not None` (api/client.py:1351)
- Benchmark ternary discarding input in both branches: `{} if target_fidelity_and_task is None else {}` always produces an empty dict regardless of input (benchmark/benchmark.py:701)
- String `"False"` default (truthy) preventing trials from being marked CANDIDATE: `t.run_metadata.get(STARTED_KEY, "False")` returns the string `"False"` which is truthy, so `not "False"` is always False (runners/map_replay.py:44)
- Operator precedence making `noisy=False` set all means to 0.0: `item["mean"] + noise if noisy else 0.0` parses as `(item["mean"] + noise) if noisy else 0.0`, replacing the entire mean with 0.0 when `noisy=False` (metrics/noisy_function_map.py:86, metrics/branin_map.py:119)
- Misleading "total number of trials" vs actual trial index (global_stopping/strategies/improvement.py:132) + test update
- Typos: "trial rial" → "trial" (fb/tutorials/auto_tuning.py), "prefererd" → "preferred" and stray `f` in f-string (fb/utils/preference/preference.py), "acquistion" → "acquisition" (generators/torch/botorch_modular/utils.py + test)
Differential Revision: D92879402
meta-cla
bot
added
the
CLA Signed
|
@saitcakmak has exported this pull request. If you are a Meta employee, you can view the originating Diff in D92879402. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4899 +/- ##
=======================================
Coverage 96.76% 96.76%
=======================================
Files 593 593
Lines 62035 62033 -2
=======================================
- Hits 60030 60029 -1
+ Misses 2005 2004 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Fix bugs in smaller modules with 1-2 files each:
"generation_strategy" in snapshotis True for the key with a None value, causing a decode crash. Changed tosnapshot.get("generation_strategy") is not None(api/client.py:1351){} if target_fidelity_and_task is None else {}always produces an empty dict regardless of input (benchmark/benchmark.py:701)"False"default (truthy) preventing trials from being marked CANDIDATE:t.run_metadata.get(STARTED_KEY, "False")returns the string"False"which is truthy, sonot "False"is always False (runners/map_replay.py:44)noisy=Falseset all means to 0.0:item["mean"] + noise if noisy else 0.0parses as(item["mean"] + noise) if noisy else 0.0, replacing the entire mean with 0.0 whennoisy=False(metrics/noisy_function_map.py:86, metrics/branin_map.py:119)fin f-string (fb/utils/preference/preference.py), "acquistion" → "acquisition" (generators/torch/botorch_modular/utils.py + test)Differential Revision: D92879402