feat: Search annotations in pipeline run API

[yuechao-qin]

TL;DR

Implemented annotation-based filtering for Pipeline Run API and proper pagination handling.

What changed?

API GET /api/pipeline_runs/

Functionality

• Implemented support for filter_query which handles annotation searches in pipeline runs.
• Added new database index for annotation-based queries. Covering index (key, pipeline_run_id, value).
• Enhanced pagination to preserve both legacy filter and new filter_query across pages.

Other

• Created new filter_query_sql.py module with comprehensive filter query processing:
  • PageToken class for encoding/decoding pagination state
  • Support for annotation-based predicates: key_exists, value_equals, value_contains, value_in
  • Logical operators: and, or, not with proper nesting
  • Integration with existing legacy filter system
• Refactored pipeline run listing to use the new filter query system while maintaining backward compatibility

How to test?

uv run pytest tests/test_api_server_sql.py tests/test_filter_query_sql.py

• Test basic annotation filtering: {"and": [{"key_exists": {"key": "team"}}]}
• Test value-based filtering: {"and": [{"value_equals": {"key": "env", "value": "prod"}}]}
• Test logical combinations with nested and/or/not predicates
• Verify mutual exclusivity validation between filter and filter_query parameters
• Test pagination preservation of filter_query state across multiple pages
• Confirm legacy filter functionality remains unchanged

Why make this change?

Annotation search support is the backbone for all the upcoming search features (create_by, pipeline name, status) for Pipeline Runs. The other searches will be search against the annotations table.

[yuechao-qin]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: Add include_sql to Search Pipeline Run API #135
• feat: Search pipeline run API uses pagination cursor #131
• feat: Search within date range in pipeline run API #130
• feat: Search pipeline name in pipeline run API #129
• refactor: Convert legacy filter to be filter_query #128
• feat: Search created by user in pipeline run API #108
• feat: Search annotations in pipeline run API #107 👈 (View in Graphite)
• feat: Create Pydantic models for Search Pipeline Run API #106
• refactor: Search pipeline run refactoring and baseline unit tests #105
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Ark-kun]

Will you be moving to cursor way of filtering? If you plan to make changes soon, this class should not be public. Otherwise it's a breaking change.
Maybe you should just make the whole module private. Like _api_server_search_utils.py. ("utils" is an anti-pattern, but I do not have a better idea yet.)

[Ark-kun]

This indentation was correct (it's a class variable inside a class).

[Ark-kun]

I don't like that we construct all these different objects just for comparison.

You can do:

    match type(predicate):
        case filter_query_models.AndPredicate:

[Ark-kun]

Maybe we can assert the whole query in some of these tests? It would be nice to see the full queries here.
I guess, there is a possibility that this query could be brittle and change with new version of SqlAlchemy. But we can wait for that moment and then make tests more robust if that happens.