feat: Add include_sql to Search Pipeline Run API

[yuechao-qin]

TL;DR

Added optional SQL query debugging to the pipeline runs list API endpoint.

What changed?

Functional

• Added include_sql parameter to API

GET /api/pipeline_runs/?include_sql=true&filter_query=<QUERY>

• Added sql field to ListPipelineJobsResponse to optionally return the generated SQL query

{
  "pipeline_runs": [...],
  "next_page_token": <TOKEN>,
  "sql": "SELECT * FROM pipeline_run WHERE ... ORDER BY ..."
}

How to test?

uv run pytest tests/test_api_server_sql.py

• Call the list endpoint with include_sql=True to verify the SQL query is returned in the response
• Test with various filter queries and pagination tokens to ensure the SQL compilation works correctly
• Verify that when include_sql=False or is omitted, the sql field remains None

Why make this change?

• This enables debugging and monitoring of the actual SQL queries generated by the pipeline runs list endpoint.
• This is required by E2E testing (PR here) to be able to log and report what SQL statement was used per API request.

[yuechao-qin]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: Add include_sql to Search Pipeline Run API #135 👈 (View in Graphite)
• feat: Search pipeline run API uses pagination cursor #131
• feat: Search within date range in pipeline run API #130
• feat: Search pipeline name in pipeline run API #129
• refactor: Convert legacy filter to be filter_query #128
• feat: Search created by user in pipeline run API #108
• feat: Search annotations in pipeline run API #107
• feat: Create Pydantic models for Search Pipeline Run API #106
• refactor: Search pipeline run refactoring and baseline unit tests #105
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Ark-kun]

I'm not sure we should support this in public API.
This could also become problematic if we support alternative non-SQL search methods like ElasticSearch/OpenSearch.

It's pretty simple to locally compile arbitrary filter_query or page_token to arbitrary SQL dialect.

Let's discuss the primary use case and decide how to accomplish that goal without adding this debug information to the public API surface.

[yuechao-qin]

• add "_" in front of paramter "_include_sql"
• hide include_sql from openapi
• add this to oasis-backend not tangle (https://oasis.shopify.io/api/stats/quick?format=html)

[yuechao-qin]

Brainstorming ideas:

• add "_" in front of paramter "_include_sql"
• hide include_sql from openapi
• add this to oasis-backend not tangle (https://oasis.shopify.io/api/stats/quick?format=html)