How to cancel or stop an in-flight agent invocation and remove it from context

[rohan-kavari]

I am using ADK to handle user queries that can take a non-trivial amount of time to process.

Use case

1. A user submits a query.

2. The agent starts processing the query.

3. Before the agent finishes, the user realizes the query needs to be edited.

4. The user wants to:

   • Stop or cancel the current in-flight invocation.
   • Ensure the partially processed query does not continue running.
   • Ensure the cancelled query is not added to the agent’s context or memory.

5. The user then resubmits a modified query.

---

Questions

1. Does ADK support cancellation of an in-flight invocation (for example via request ID, session ID, or cancellation token)?

2. If using streamQuery, is there a supported way to terminate the stream such that the agent execution is also stopped on the backend?

3. If a client disconnects early, does ADK automatically stop execution or does the agent continue running?

4. What is the recommended pattern to prevent cancelled queries from being added to context?

   • Explicit context rollback?
   • Stateless queries?
   • Creating a new session per user edit?

---

Context

• ADK agent running via Agent Engine
• Using both query and streamQuery
• Frontend allows users to edit or cancel queries mid-execution

Any guidance on supported APIs, lifecycle hooks, or best practices for cancellation and context management would be very helpful.

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: ADK provides mechanisms to cancel in-flight requests and manage session context. For standard query invocations, you can set ctx.end_invocation = True. For streamQuery, you should close the request queue. To prevent a canceled query from affecting future context, you can use the session rewind feature.

---

Hello! Thank you for the detailed questions. Here are the recommended patterns for handling invocation cancellation and context management in ADK based on your use case.

1. Canceling a query Invocation

For a standard (non-streaming) invocation, you can gracefully stop the entire request-response cycle from within your agent, a callback, or a tool by setting ctx.end_invocation = True. When this flag is set, the agent will stop processing and yield its final event. [1]

2. Terminating a streamQuery Invocation

When using streamQuery (which corresponds to run_live on the backend), the client needs to signal its intent to terminate the session. The recommended way to do this is to have your application call live_request_queue.close(). This sends a graceful termination signal to the backend, allowing the agent to clean up resources and stop execution. You can also send a final LiveRequest with the close=True flag. [2]

3. Handling Early Client Disconnects

If a client disconnects abruptly (e.g., closes a browser tab), ADK does not automatically guarantee the termination of the backend process. A disconnect will raise a WebSocketDisconnect exception in your application. You should implement try...except...finally blocks to catch this exception and ensure live_request_queue.close() is called in the finally block. This ensures the backend process is properly terminated. [2]

It's important to distinguish this from involuntary network disconnects, where ADK's run_live is designed to automatically attempt reconnection for a period to resume the session. [3]

4. Preventing Canceled Queries from Being Added to Context

The recommended pattern for this is to use the session rewind feature. After a query is canceled, you can revert the session to the state it was in before the canceled request began. This effectively 'rolls back' the context, removing any events or state changes that occurred during the canceled invocation. [4]

For example, if you have a session history of Request A -> Request B (canceled) -> Request C, you can rewind the session to the state after Request A, and then proceed with Request C as if Request B never happened. This avoids the need for stateless queries or creating new sessions for every user edit.

---

Hopefully, these patterns provide a clear path forward for implementing your use case.

[1] ADK Context Documentation
[2] ADK Streaming Developer Guide, Part 2
[3] ADK Streaming Developer Guide, Part 4
[4] ADK Session Rewind Documentation