Skip to content

🤖 feat: add Auto agent with switch_agent tool for backend-side agent switching#2411

Open
ThomasK33 wants to merge 14 commits intomainfrom
agent-switching-tv9e
Open

🤖 feat: add Auto agent with switch_agent tool for backend-side agent switching#2411
ThomasK33 wants to merge 14 commits intomainfrom
agent-switching-tv9e

Conversation

@ThomasK33
Copy link
Member

Summary

Adds an Auto agent that acts as a routing layer — it analyzes the user's request and calls switch_agent to hand off to the best UI-selectable agent (plan, exec, explore, ask, etc.) without requiring any UI interaction. This enables headless/background workflows where no user click can complete agent handoffs.

Background

Currently, switching between agents (plan → exec, exec → explore, etc.) requires manual UI interaction via the AgentModePicker dropdown. This blocks automated/headless workflows and prevents agents from self-routing based on task analysis. The Auto agent + switch_agent tool solves this with a backend-side stream restart mechanism.

Implementation

Architecture: Signal tool + backend follow-up

  1. Model calls switch_agent({ agentId, followUp }) — a signal tool that returns { ok: true } immediately
  2. StreamManager detects the success and stops the autonomous loop (same pattern as agent_report)
  3. AgentSession inspects stream-end parts, finds the switch signal, and enqueues a synthetic follow-up sendMessage with the target agent's ID
  4. Next turn rebuilds system prompt + tools for the new agent — no tool list mutation mid-stream

Key components

Component What it does
src/node/builtinAgents/auto.md New Auto agent — routing-only, never does work itself
src/node/services/tools/switch_agent.ts Signal tool returning { ok, agentId, reason, followUp }
src/common/orpc/schemas/workspace.ts agentSwitchingEnabled metadata flag
src/node/services/agentDefinitions/resolveToolPolicy.ts Runtime policy: enable switch_agent only for Auto-started sessions, hard-deny for subagents
src/node/services/streamManager.ts Stop condition for successful switch_agent results
src/node/services/agentSession.ts Stream-end detection, target validation, synthetic follow-up dispatch, loop guardrail

Safety guardrails

  • Target validation: rejects hidden, disabled, and non-existent agents before dispatching
  • Loop prevention: max 3 consecutive agent switches per session (resets on real user messages)
  • Subagent hard-deny: child workspaces cannot use switch_agent
  • PTC exclusion: switch_agent cannot be called from inside QuickJS sandbox

Validation

  • make typecheck — passing
  • make lint — passing
  • make static-check — passing (including docs sync)
  • All related tests (47 tests across 4 files): passing
    • switch_agent.test.ts — tool execution tests
    • resolveToolPolicy.test.ts — policy gating tests (16 tests including new switch_agent coverage)
    • agentSession.switchAgent.test.ts — target validation + follow-up dispatch integration tests
    • streamManager.test.ts — stop condition tests

Risks

  • Low: Auto agent prompt quality — the routing instructions are straightforward but may need iteration based on real usage patterns
  • Low: The agentSwitchingEnabled flag is persisted as workspace metadata; once set, it stays enabled for the session lifetime (intentional — allows switching back and forth)

📋 Implementation Plan

Auto agent + switch_agent tool (backend stream restart)

Context / Why

We want an Auto agent that can switch the active agent (between agents that are selectable in the UI) without relying on the frontend. This is important for headless/background runs where no UI click can complete handoffs.

Key requirements:

  • Add a UI-selectable auto agent.
  • Add a switch_agent tool that lets Auto (and, conditionally, other UI agents) switch to another UI-selectable agent.
  • Option B semantics: switching should be handled backend-side by ending the current stream and starting a new one with the chosen agent.
  • The switch_agent tool should be available in sessions started from Auto, but not when starting directly in plan/exec/etc.

Evidence (why this is feasible)

  • Tool policies are assembled per-turn and support runtime overrides: src/node/services/agentDefinitions/resolveToolPolicy.ts.
  • Streams already support tool-driven loop termination via stopWhen (e.g. agent_report): src/node/services/streamManager.ts.
  • The backend can enqueue follow-up turns after a stream ends (used by compaction follow-ups and TaskService): src/node/services/agentSession.ts.
  • Backend already has canonical logic for “agents visible in UI” (uiSelectable) and enablement/disablement: src/node/orpc/router.ts (agents.list) + src/node/services/agentDefinitions/agentEnablement.ts.

Recommended approach (single approach)

Implement switch_agent as a signal tool plus a backend follow-up message:

  1. The model calls switch_agent({ agentId, ... }).
  2. StreamManager detects the successful tool result and stops the stream after the tool call completes.
  3. AgentSession inspects the stream-end parts, finds the switch signal, and enqueues a synthetic follow-up sendMessage in the same workspace, with options.agentId = <target>.

This keeps correctness properties we already rely on:

  • Tool list remains static within a single provider stream.
  • The next turn rebuilds system prompt + tools for the new agent.
  • It works even when no browser UI is attached.

Net LoC estimate (product code only): ~250–400 LoC


Implementation details

1) Add a new built-in agent: auto

File: src/node/builtinAgents/auto.md

  • base: exec (Auto should have full capabilities by default).
  • ui: visible/selectable.
  • tools.add: include switch_agent.
  • Instructions:
    • Auto is a router: it should choose the best UI agent and call switch_agent.
    • It should avoid producing a “real” answer in the same turn if it is switching (since the stream will stop immediately after the tool call).

Example frontmatter shape (exact schema may differ; confirm against existing built-ins):

---
name: Auto
base: exec
ui:
  hidden: false
subagent:
  runnable: true
tools:
  add:
    - switch_agent
---

<agent instructions...>

2) Define the new tool: switch_agent

Definition: src/common/utils/tools/toolDefinitions.ts

  • Add a new TOOL_DEFINITIONS.switch_agent entry.
  • Input schema should follow strict-mode conventions (.nullish() for optional fields).

Suggested schema shape:

switch_agent: {
  description: "Switch to a different UI-selectable agent and restart the stream.",
  inputSchema: z.object({
    agentId: AgentIdSchema,
    reason: z.string().max(512).nullish(),
    followUp: z.string().max(2000).nullish(),
  }),
}

Implementation: src/node/services/tools/switch_agent.ts

Responsibilities:

  • Validate agentId is:
    • resolvable,
    • not effectively disabled,
    • and UI-selectable for this workspace (matches AgentModePicker behavior; see section 6).
  • Return a structured success payload that downstream code can reliably detect.

Suggested output:

{ ok: true, agentId, reason, followUp }

Tool registry: add createSwitchAgentTool to the base tool set so policy filtering can enable/disable it:

  • src/common/utils/tools/tools.ts: import createSwitchAgentTool and add switch_agent: createSwitchAgentTool(config) to nonRuntimeTools.

PTC / code_execution compatibility: prevent nested switch_agent calls inside the QuickJS sandbox (keeps switching detectable + predictable):

  • src/node/services/ptc/toolBridge.ts: add "switch_agent" to EXCLUDED_TOOLS.

3) Gate tool availability to “Auto-started sessions”

We need switch_agent to be injected into other UI agents only if the session was started from Auto.

Recommended mechanism: persist a boolean on the workspace metadata, e.g. agentSwitchingEnabled.

  • Schema: add an optional boolean to WorkspaceMetadataSchema (src/common/orpc/schemas/workspace.ts).
  • Write-path: when the requested agentId for a turn is auto, set workspace.metadata.agentSwitchingEnabled = true (persist via the existing config/metadata save path).
    • Implementation detail: the cleanest place to do this is the same node-side path that already mutates/persists WorkspaceMetadata for a running workspace (likely WorkspaceService.sendMessage() or AgentSession.sendMessage()).
  • Threading: src/node/services/agentResolution.ts → pass enableAgentSwitchTool into resolveToolPolicyForAgent().
  • Read-path / policy: thread this flag into tool policy resolution:
    • Extend ResolveToolPolicyOptions in src/node/services/agentDefinitions/resolveToolPolicy.ts with enableAgentSwitchTool: boolean.
    • Append a runtime enable rule:
      if (enableAgentSwitchTool && !isSubagent) {
        runtimePolicy.push({ regex_match: "switch_agent", action: "enable" });
      }
    • Consider adding a subagent hard deny for switch_agent (match existing propose_plan/ask_user_question denies) to avoid child workspaces unexpectedly self-switching.
Alternative: derive “Auto session” from history

Instead of persisting metadata, compute enableAgentSwitchTool by scanning history for whether the earliest assistant message’s agentId is auto.

Pros: no schema changes.
Cons: requires parsing history at least once, and gets tricky for very fresh sessions (no assistant messages yet). Persisting a boolean is simpler and more robust.

Estimated LoC difference: -20 to +80 depending on caching.

4) Stop the current stream after switch_agent

File: src/node/services/streamManager.ts

  • Update createStopWhenCondition() to stop when a successful switch_agent tool result is observed.
  • Mirror the existing agent_report logic (stop autonomous tool loop cleanly; don’t abort).

Pseudo-shape:

if (part.type === "tool-result" && part.toolName === "switch_agent") {
  const out = part.output as unknown;
  if (isOkSwitchAgentResult(out)) return true;
}

5) Backend-side restart into the selected agent (Option B)

File: src/node/services/agentSession.ts

In the stream-end handler (within attachAiListeners):

  1. Scan event.parts for a switch_agent tool result.
  2. If present:
    • enqueue a follow-up message using existing queue mechanisms (or call sendMessage directly if safe)
    • set options.agentId = <target>
    • use internal flags similar to existing follow-up sends:
      • synthetic: true
      • skipAutoResumeReset: true

Follow-up message content:

  • Use followUp from tool input if provided; else default to something short and deterministic, e.g.:
    • "Continue." or
    • "Continue in the newly selected mode."

Defensive guardrail: add a per-session counter (similar to TaskService’s consecutiveAutoResumes) to prevent infinite ping-pong switching.

  • e.g. max 3 consecutive switch-triggered follow-ups.
  • reset on any real user message.

6) Validate targets against “agents visible in UI”

Validation should match AgentModePicker’s selectable list.

Reuse backend logic from agents.list (in src/node/orpc/router.ts) to determine:

  • uiSelectable (honors ui.hidden, legacy ui.selectable, and ui.requires: ["plan"])
  • isAgentEffectivelyDisabled (honors config overrides and disabled flags)

The switch_agent tool should reject:

  • hidden/internal agents (e.g. compact, explore, system1_bash)
  • disabled agents
  • agents requiring a plan when the plan file is empty

7) Tests

Add coverage for:

  • switch_agent rejects non-uiSelectable targets.
  • Tool policy gating: switch_agent only appears when agentSwitchingEnabled is true (and not for subagents).
  • End-to-end chaining behavior:
    • call switch_agent and confirm the session enqueues a follow-up stream with the new agentId
    • guardrail prevents infinite loops

(Place tests alongside existing node service tests; exact directory depends on current conventions.)


Validation checklist (when implementing)

  • make typecheck
  • make test
  • (If touched) make lint

Manual smoke tests

  1. Start a workspace in auto.
  2. Send a prompt that should route to plan.
  3. Confirm auto calls switch_agent(plan) and that the backend starts a new stream in plan without any UI interaction.
  4. Confirm a session started directly in plan does not have switch_agent available.

Generated with mux • Model: anthropic:claude-opus-4-6 • Thinking: xhigh • Cost: $2.83

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8ea3c4a05f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33
Copy link
Member Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 893b8e5b65

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33
Copy link
Member Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6551f1406a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33
Copy link
Member Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 45a26b6c0f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33
Copy link
Member Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 335cf982fc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33
Copy link
Member Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 82ac6dc4eb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33
Copy link
Member Author

@codex review

@chatgpt-codex-connector
Copy link

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant