feat: Annotation search for Pipeline Runs

[yuechao-qin]

TODO

• How do I confirm that the unit test will be part of CI/CD?
• Black formatter, should it align with Google's Python style guide here?
• Review expected behavior of search with multiple annotations
  • Should/Where to save end-2-end test script
• Test in oasis staging
  • Test SQL in KateSQL staging to confirm indexes and all operations

Description

Closes #45

Implemented a new API to search annotations for runs.

Background

Annotations are key-value pairs. The following examples of annotations (e.g. key = value):

• env = production
• team = backend

This PR allows searches for key/value strings.

Features

• Updated API (GET /api/pipeline_runs/) to search key/value in annotations.
• Key and value searchable operations
  • CONTAINS: If key/value contains a substring
  • IN_SET: If key/value string matches set of strings
  • EQUALS: If key/value string equals string
• Keys and Values search operations can be negated (i.e. NOT ).
• Default is OR'ing multiple key/value. Is able to do ANDs
  • Example (default): (key=key1 and value=value1) or (key=key2 and value=value2)
  • Example (advance): (key=key1 and value=value1) and (key=key2 and value=value2)

Use Cases and Examples

1. Key equals a string

Find runs where annotation key equals "environment":

{"annotation_filters": [{"key": {"operator": "equals", "text": "environment"}}]}

2. Key equals string AND value in set

Find runs where key equals "environment" AND value is "prod" or "staging":

{
  "annotation_filters": [
    {
      "key": {"operator": "equals", "text": "environment"},
      "value": {"operator": "in_set", "texts": ["prod", "staging"]}
    }
  ]
}

3. Complex: (key equals AND value contains) AND key NOT contains

Find runs where (key equals "environment" AND any value contains "prod") AND key NOT contains "team":

{
  "operator": "and",
  "annotation_filters": [
    {
      "key": {"operator": "equals", "text": "environment"},
      "value": {"operator": "contains", "text": "prod"}
    },
    {"key": {"operator": "contains", "text": "team", "negate": true}}
  ]
}

Test Plan

• Unit test
  • uv run pytest tests/test_pipeline_run_search.py -v
• E2E Manual Testing
  • Watch demo videos below
  • Add annotations for testing (PUT /api/pipeline_runs/<ID>/annotations/<KEY>/)
  • Query annotations for ID (GET /api/pipeline_runs/<ID>/annotations/)
  • Test with new search (POST /api/pipeline_runs/search/)
• Test on Staging Oasis
• Test on Staging KateSQL

Video of E2E Testing

Screen.Recording.2026-01-28.at.1.18.33.PM.mov

[Ark-kun]

annotation_filters:
 and:
 - in_set:
    key: environment
    items: ["prod", "staging"]

 - value_equals:
    key: hidden
    value: "true"

 - value_not_equals:
    key: hidden
    value: "false"

 - not:
     key_exists:
       key: hidden

 - key_matches:
       key_pattern: official

[Ark-kun]

Same (value.negate ignored)

I consider this to be a code smell.
I do not mean that you have to change the behavior here. Rather I think that this might indicates a problem with the approach.

[Ark-kun]

Maybe just log it.

[yuechao-qin]

annotation_filters:
 and:
 - in_set:
    key: environment
    items: ["prod", "staging"]

 - value_equals:
    key: hidden
    value: "true"

 - value_not_equals:
    key: hidden
    value: "false"

 - not:
     key_exists:
       key: hidden

 - key_matches:
       key_pattern: official

Predicates:

key1 [not] exists
key1 value [not] equals value1
key1 value is [not] in set: value1, value2
key1 value [not] contains substr1
and

• Key Search (Search for a particular key, regardless of value)
  • Key [not] Exist

  SELECT *
  FROM pipeline_runs
  WHERE [NOT] EXISTS (SELECT * FROM pipeline_run_annotation WHERE key = "key1”)
  

• Value Search (Search for value for a particular key)
  • Key Exist (must be specified, doesn't handle "not")
  • AND (one option below)
    • key1 value [not] equals value1
    • key1 value is [not] in set: value1, value2
    • key1 value [not] contains substr1