Skip to content

2510 hi goodtimes set up cli call to processing 1#2779

Open
subagonsouth wants to merge 10 commits intoIMAP-Science-Operations-Center:devfrom
subagonsouth:2510-hi-goodtimes---set-up-cli-call-to-processing-1
Open

2510 hi goodtimes set up cli call to processing 1#2779
subagonsouth wants to merge 10 commits intoIMAP-Science-Operations-Center:devfrom
subagonsouth:2510-hi-goodtimes---set-up-cli-call-to-processing-1

Conversation

@subagonsouth
Copy link
Contributor

@subagonsouth subagonsouth commented Feb 26, 2026

Change Summary

Overview

This PR adds the top-level hi_goodtimes orchestrator function and integrates it with the CLI for IMAP-Hi L1B goodtimes processing.

Changes

imap_processing/hi/hi_goodtimes.py

  • Added hi_goodtimes() top-level function that orchestrates all goodtimes culling operations
  • Added _find_current_pointing_index() helper to locate the current pointing in the datasets list
  • Added _apply_goodtimes_filters() helper that applies all 6 culling filters (incomplete spin sets, DRF times, overflow packets, statistical filters 0-2)
  • Added finalize_dataset() accessor function that prepares goodtimes dataset for CDF writing
  • Implements repoint availability check: processing only proceeds if current_repointing + 3 has occurred
  • Handles incomplete DE file sets by marking all times as bad (CullCode.LOOSE)

imap_processing/cli.py

  • Added Hi L1B goodtimes processing branch in Hi.do_processing()
  • Loads L1B DE and HK CDFs before passing to hi_goodtimes
  • Validates required dependencies (DE files, 1 HK file, 1 cal-prod ancillary)
  • Passes start_date and version from CLI arguments to hi_goodtimes

imap_processing/tests/hi/test_hi_goodtimes.py

  • Added TestFindCurrentPointingIndex - tests for finding current pointing index in datasets
  • Added TestApplyGoodtimesFilters - tests all 6 filters are called and errors are handled
  • Added TestHiGoodtimes - tests the main orchestrator including repoint checks, incomplete DE handling, and filter application

imap_processing/tests/test_cli.py

  • Added test_hi_l1c_goodtimes - tests CLI integration with proper mocking of load_cdf and hi_goodtimes

Closes: #2510

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements the top-level Hi goodtimes orchestrator function and integrates it with the CLI for IMAP-Hi L1C goodtimes processing. The goodtimes processing identifies and culls bad time intervals for Hi sensor data using 6 different filtering algorithms (incomplete spin sets, DRF times, overflow packets, and 3 statistical filters). The implementation includes repointing availability checks to ensure sufficient surrounding pointings exist for statistical analysis before proceeding with processing.

Changes:

  • Added hi_goodtimes() orchestrator function and helper functions (_find_current_pointing_index, _apply_goodtimes_filters, _write_goodtimes_output) to manage the goodtimes culling workflow
  • Integrated goodtimes processing into the CLI with proper dependency validation and CDF loading
  • Updated sensor naming convention from "Hi45"/"Hi90" to "sensor45"/"sensor90" to maintain consistency
  • Added comprehensive test coverage for the new orchestrator, helpers, and CLI integration

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
imap_processing/hi/hi_goodtimes.py Adds hi_goodtimes() orchestrator and helper functions for goodtimes processing; updates sensor naming convention in create_goodtimes_dataset()
imap_processing/cli.py Adds L1C goodtimes processing branch in Hi.do_processing() with dependency validation and CDF loading; refactors L1C pset processing into explicit elif branch
imap_processing/tests/hi/test_hi_goodtimes.py Adds comprehensive test coverage for new functions (TestFindCurrentPointingIndex, TestWriteGoodtimesOutput, TestApplyGoodtimesFilters, TestHiGoodtimes); updates sensor names in existing tests
imap_processing/tests/test_cli.py Adds test_hi_l1c_goodtimes for CLI integration testing; updates parametrize decorator in test_hi to include data_descriptor parameter

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 773 to 775
def do_processing( # noqa: PLR0912
self, dependencies: ProcessingInputCollection
) -> list[xr.Dataset]:
Copy link

Copilot AI Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return type annotation for Hi.do_processing should be list[xr.Dataset | Path] instead of list[xr.Dataset] because the goodtimes processing path returns a list of Path objects (line 858). This is consistent with the Spacecraft instrument's do_processing method which also returns list[xr.Dataset | Path] and aligns with the post_processing method's signature which accepts list[xr.Dataset | Path].

Copilot uses AI. Check for mistakes.
@subagonsouth subagonsouth marked this pull request as draft February 26, 2026 22:42
@subagonsouth subagonsouth force-pushed the 2510-hi-goodtimes---set-up-cli-call-to-processing-1 branch from 4b8d4c3 to 3d022e1 Compare March 3, 2026 21:16
@subagonsouth subagonsouth requested a review from Copilot March 3, 2026 21:20
@subagonsouth subagonsouth marked this pull request as ready for review March 3, 2026 21:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copy link
Contributor

@tech3371 tech3371 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

goodtimes file has lot going on but it was easy to follow and understand on high-level. Looks good to me.

Notes
-----
The output filename follows the pattern:
imap_hi_sensor{45|90}-goodtimes_{start_date}_{start_date}_{version}.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is two start_date intentional or type? And are you going to convert goodtimes to CDF in future PR or is that still being decided?

subagonsouth and others added 3 commits March 4, 2026 08:52
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Hi Goodtimes - Set up cli call to processing

3 participants