Skip to content

FrontierCS 2.0 Vector db ann task#121

Merged
joyemang33 merged 2 commits into
mainfrom
vector-db-ann-task
May 29, 2026
Merged

FrontierCS 2.0 Vector db ann task#121
joyemang33 merged 2 commits into
mainfrom
vector-db-ann-task

Conversation

@joyemang33
Copy link
Copy Markdown
Contributor

Summary

Adds a new Frontier-CS 2.0 vector_db_ann task for SIFT1M-scale approximate nearest-neighbor vector search, with a Rust project skeleton, hidden black-box evaluator, reference baseline, and Harbor support.

Also updates the Frontier-CS 2.0 Harbor adapter/environment so tasks can submit whole projects, configure runtime/judge dependencies, expose judge readiness, preserve iterative best submissions, and report task-specific metrics such as QPS, effective QPS, recall, load time, tokens, and cost.

Please read CONTRIBUTING.md before submitting.

Type of Change

  • New research problem
  • New algorithmic problem
  • New Frontier-CS 2.0 problem
  • Bug fix
  • Documentation update
  • Other: Harbor adapter/runtime support for project-based 2.0 tasks

Testing

  • PYTHONPYCACHEPREFIX=/private/tmp/frontier-cs-pycache python3 -m py_compile ...
  • uv run --no-sync frontier list 2.0
  • uv run --no-sync frontier show 2.0 vector_db_ann
  • Generated the Harbor task via frontier_cs_2_0.main --task-ids vector_db_ann
  • Ran Harbor trials for vector_db_ann; best observed valid submission reached recall 0.9871, QPS 370.24, and score 20.65

Checklist

  • Code follows the project structure and conventions
  • Self-review completed
  • Documentation updated (if applicable)

CI Validation (for new problems)

When adding new problems, CI will automatically validate that your reference solution achieves score > 0.

  • Algorithmic problems: Include reference.cpp in your problem directory
  • Research problems: Include reference.py (or reference.cpp if language: cpp in config.yaml)
  • 2.0 problems: Include reference.py unless the problem config declares another language

@joyemang33 joyemang33 marked this pull request as ready for review May 29, 2026 10:34
@joyemang33 joyemang33 merged commit 230d20e into main May 29, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant