Skip to content

Implement adaptive threshold calculation in video_describer.extract_significant_frames for dynamic frame selection #9

@chigwell

Description

@chigwell

User Story
As a FrameStory user,
I want adaptive threshold calculation in extract_significant_frames
so that frame extraction accuracy improves across videos with varying lighting and motion dynamics.

Background
The current fixed threshold in video_describer.py (line 31: threshold=50000) causes inconsistent frame selection. Videos with rapid scene changes or subtle lighting variations either miss key frames (high threshold) or capture redundant ones (low threshold). This limitation is evident in extract_significant_frames where cv2.absdiff comparisons use a static value rather than adapting to content characteristics. Risks include inaccurate video summaries and reduced model training effectiveness if frame selection quality deteriorates.

Acceptance Criteria

  • Modify extract_significant_frames in frame_story/video_describer.py to implement dynamic threshold calculation
  • Replace threshold parameter with logic analyzing local frame differences (e.g., moving average of 5-frame window)
  • Add optional adaptive=True parameter to VideoDescriber.__init__ to enable/disable this feature
  • Update tests in tests/test_video_describer.py to validate threshold adaptation using low-variation and high-variation test videos
  • Document the adaptive threshold in the README's "Customizing Extraction Threshold" example
  • Verify through manual testing that:
    • Darker videos no longer require manual threshold adjustments
    • Action sequences with rapid cuts capture scene transitions precisely
    • Threshold values appear in debug logs when show_progress=True

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions