Skip to content

Conversation

@amishne
Copy link
Contributor

@amishne amishne commented Dec 8, 2025

  1. Add "test" and "e2e" MCP tools, with their tests.
  2. Add an "all" experimental tool group so it's easy to enable them all at once.
  3. Make the default project use the actual project name instead of the "" string.
  4. Test cleanup, including centralizing test-only utilities into a new file.

In addition to the unit tests, the test tool was also tested manually installed into Gemini-CLI.

This adds new tools for running unit and end-to-end tests via the MCP server.
@amishne amishne requested review from clydin and dgp1130 December 8, 2025 23:05
@amishne amishne added area: @angular/cli action: review The PR is still awaiting reviews from at least one requested reviewer target: minor This PR is targeted for the next minor release labels Dec 8, 2025
@angular-robot angular-robot bot added the detected: feature PR contains a feature commit label Dec 8, 2025
Copy link
Collaborator

@dgp1130 dgp1130 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of random thoughts and code suggestions, but overall LGTM. Thanks for adding this!

return createStructuredContentOutput({
status: 'failure',
logs: [
`No e2e target is defined for project '${projectName ?? 'default'}'. Please setup e2e testing first.`,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: We should either link to documentation or call out direct usage of the ng e2e command as a mechanism to configure these tests.

Also typo: I think "setup" is a noun, "set up" would be using it as a verb.

Comment on lines +77 to +88
try {
logs = (await host.runCommand('ng', args)).logs;
} catch (e) {
status = 'failure';
if (e instanceof CommandError) {
logs = e.logs;
} else if (e instanceof Error) {
logs = [e.message];
} else {
logs = [String(e)];
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: Can this be refactored to a shared primitive somewhere? Seems like it would overlap pretty heavily with other executions of the CLI in the MCP server.

let logs: string[] = [];

try {
logs = (await host.runCommand('ng', args)).logs;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: I probably should have asked this on an earlier PR (maybe I did and forgot), but how is ng resolved? Is that looking for a global installation on the $PATH? Is it resolving within the project's node_modules somehow? I'm wondering if we want to call process.argv[0] to reuse the existing Angular CLI binary rather than attempting to find it again in the environment?

expect(mockHost.runCommand).not.toHaveBeenCalled();
});

it('should error if default project does not have e2e target and no project specified', async () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: A test which executes ng e2e successfully with the default project. I only see the error case here.

}

// This is ran by the agent so we want a non-watched, headless test.
args.push('--browsers', 'ChromeHeadless');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: What if someone doesn't have Chrome installed and instead uses Firefox or Safari?

@clydin is there a way to trigger "headless" without specifying a specific browser? The alternative I can think of is setting the CI=true environment variable.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: Add tests for these utilities directly. It can potentially reduce the need for some test coverage in the MCP tools as well.

Comment on lines +27 to +29
getAvailablePort: jasmine
.createSpy<Host['getAvailablePort']>('getAvailablePort')
.and.resolveTo(0),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: Could we put the auto-incrementing port number implementation here and potentially reuse it across all affected tests?

Comment on lines +24 to +28
const mock = createMockContext();
mockHost = mock.host;
mockContext = mock.context;
mockProjects = mock.projects;
mockWorkspace = mock.workspace;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: If we can make this mock setup easier (potentially with the fake implementation proposed in a separate comment) we can move this out of the beforeEach and into each test, so each one can get a unique environment for its needs rather than attempting to share and then retroactively configure a base environment.

* Creates a mock implementation of the Host interface for testing purposes.
* Each method is a Jasmine spy that can be configured.
*/
export function createMockHost(): MockHost {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider: Is there a path to a "fake" host instead of a "mock" host? Could we provide fake implementations for stat and existsSync if we accept a virtual file system (or create a real one in a temp directly from inputs)?

Faking implementation of runCommand and spawn is likely a bit harder, but we could either delegate the command to an input function or actually run a real subprocess to get more accurate error behavior. I'm thinking something along the lines of:

import * as fs from 'fs/promises';

class FakeHost implements Host {
  private nextPort = 0;

  private constructor(private readonly cmdImpls: Record<string, (...args: string[]) => CommandResult>, private readonly root: string) {}

  static async from(cmdImpls: Record<string, (...args: string[]) => CommandResult>, fs: Record<string, string | Buffer> = {}): Promise<FakeHost> {
    const tempDir = await fs.mkdtemp();

    for (const [path, content] of Object.entries(fs)) {
      await fs.writeFile(path, content);
    }

    return new FakeHost(cmdImpls, tempDir);
  }

  stat() { /* Check in `root` */ }
  existsSync() { /* Check in `root` */ }

  getAvailablePort(): number {
    return this.nextPort++;
  }

  runCommand(binary: string, args: string[]): /* ... */ {
    return this.cmdImpls[binary](args);
  }
  spawn() { /* Similar to `runCommand`... */ }

  await dispose(): Promise<void> {
    await fs.rmdir(this.root);
  }
}

I'm thinking this can avoid the need to define spies unique to each test and increase overall test confidence. If we can make this specific to each test as suggested in a separate comment, it can potentially remove the need to mutate the environment after the beforeEach within a test.

Downside is that we do need to explicitly call dispose to clean up the temp directory if we're using a real file system. If we can run these tests on Node 24, we can potentially get away with using and Symbol.dispose which would help.

it('does a thing', () => {
  await using host = await FakeHost.from(...);
  await runE2e(host, ...);
  // Host implicitly disposed at end of scope.
});

Comment on lines 80 to 85
it('should proceed if no workspace context is available (fallback)', async () => {
// If context.workspace is undefined, it should try to run ng e2e (which might fail or prompt, but tool runs it)
const noWorkspaceContext = {} as McpToolContext;
await runE2e({}, mockHost, noWorkspaceContext);
expect(mockHost.runCommand).toHaveBeenCalledWith('ng', ['e2e']);
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: This is the case where we run the MCP server outside of a specific Angular CLI project right? I wonder if we should just fail in that case rather than attempting to run an ng e2e command which will almost certainly fail.

Thoughts @clydin?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

action: review The PR is still awaiting reviews from at least one requested reviewer area: @angular/cli detected: feature PR contains a feature commit target: minor This PR is targeted for the next minor release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants