Mutation Testing and AI Test Generation: Killing Surviving Mutants with Stryker

The 95% Code Coverage Illusion

When I landed my first serious software engineering job after a year of self-learning, our repository had a strict rule: every pull request required at least 90% code coverage. If your tests dropped coverage to 89%, GitHub Actions blocked your merge.

For the first two months, I thought high coverage meant high quality software. Then, on a Friday afternoon, a critical pricing calculation error slipped straight through our CI pipeline into production. Our automated test suite passed with 94% coverage. How was that possible?

Because standard line coverage only measures which lines of code were executed by the runtime. It tells you nothing about whether your tests actually check the output. Look at this function and its test:

// fee-calculator.ts
export function calculateTieredFee(amount: number, isVip: boolean): number {
  if (amount <= 0) return 0;
  let rate = 0.03;
  if (isVip || amount >= 10000) {
    rate = 0.015;
  }
  return amount * rate;
}

// fee-calculator.test.ts (100% line coverage, 0 verification value)
test('calculates fee', () => {
  calculateTieredFee(500, false);
  calculateTieredFee(20000, true);
  calculateTieredFee(-10, false);
  // Notice: Zero assertions! Yet Istanbul and C8 report 100% line coverage.
});

If someone accidentally changes amount >= 10000 to amount > 10000, or breaks the calculation formula completely, the tests still pass. Fake coverage gives engineering teams false confidence. To catch real bugs, you need mutation testing.

How Mutation Testing Works

Mutation testing turns standard unit testing upside down. Instead of testing your code, it tests the strength of your test suite. A mutation runner like Stryker JS systematically alters your source code in memory, creating small intentional bugs called mutants:

• Equality Mutator: Flips === to !==.
• Boundary Mutator: Modifies amount >= 10000 to amount > 10000.
• Arithmetic Mutator: Swaps amount * rate with amount / rate.
• Block Removal: Empties the body of your if statements.

After injecting each mutant, Stryker runs your test suite:

[Original Code] ──► Stryker injects mutation (>= becomes >)
                         │
                         ▼
                 [Run Test Suite]
                    │        │
        Tests Fail  │        │  Tests Still Pass!
                    ▼        ▼
            MUTANT KILLED    MUTANT SURVIVED
            (Strong Tests)   (Blind Spot in Test Suite!)

If your tests fail, congratulations: the mutant was killed. If your tests pass despite broken application logic, the mutant survived. Surviving mutants show you the exact line where your tests failed to verify behavior.

Setting Up Stryker JS in a TypeScript Project

You can add Stryker to any modern TypeScript project using Vitest or Jest in a few minutes:

# Install Stryker core and Vitest runner as dev dependencies
npm install -D @stryker-mutator/core @stryker-mutator/vitest-runner vitest typescript

Add a clean stryker.config.json file at your project root:

{
  "$schema": "https://raw.githubusercontent.com/stryker-mutator/stryker-js/master/packages/api/schema/stryker-schema.json",
  "packageManager": "npm",
  "reporters": ["html", "clear-text", "progress", "json"],
  "testRunner": "vitest",
  "coverageAnalysis": "perTest",
  "mutate": [
    "src/**/*.ts",
    "!src/**/*.test.ts",
    "!src/**/*.d.ts"
  ],
  "thresholds": {
    "high": 80,
    "low": 60,
    "break": 70
  }
}

Running Stryker and Finding Surviving Mutants

Run Stryker directly from your terminal:

npx stryker run

Stryker executes every mutation and produces a clear summary in your console:

# Mutation Testing Summary
Total Mutants: 14
Killed: 9
Survived: 5
Mutation Score: 64.28%

Survived Mutants Details:
File: src/fee-calculator.ts
Line 5: if (isVip || amount >= 10000)
  Mutator: EqualityOperator
  Mutated: if (isVip || amount > 10000)
  Status: Survived (No test verified the exact boundary of 10,000)

Using AI to Write Targeted Mutant Killers

Instead of manually staring at surviving mutant diffs for hours, you can feed Stryker's JSON report into an automated script that writes exact assertions to kill each mutant.

Here is a Node.js script that reads reports/mutation/mutation.json and generates targeted Vitest tests using the AI SDK:

// scripts/generate-mutant-killers.ts
import * as fs from 'node:fs/promises';
import { createAnthropic } from '@ai-sdk/anthropic';
import { generateText } from 'ai';

const anthropic = createAnthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

interface StrykerReport {
  files: Record<string, {
    source: string;
    mutants: Array<{
      id: string;
      mutatorName: string;
      status: string;
      replacement: string;
      location: { start: { line: number; column: number }; end: { line: number; column: number } };
    }>;
  }>;
}

async function generateKillerTests(): Promise<void> {
  const reportRaw = await fs.readFile('reports/mutation/mutation.json', 'utf-8');
  const report: StrykerReport = JSON.parse(reportRaw);

  for (const [filePath, fileData] of Object.entries(report.files)) {
    const survivingMutants = fileData.mutants.filter((m) => m.status === 'Survived');
    if (survivingMutants.length === 0) continue;

    console.log(`Analyzing ${survivingMutants.length} surviving mutants in ${filePath}...`);

    const prompt = `You are a Senior QA Automation Engineer.
Below is TypeScript code and a list of surviving mutants from Stryker JS.
Write a clean Vitest test suite with explicit assertions that kill every surviving mutant.

### Source Code (${filePath}):
\`\`\`typescript
${fileData.source}
\`\`\`

### Surviving Mutants:
${survivingMutants.map((m) => `- Line ${m.location.start.line}: ${m.mutatorName} replaced with '${m.replacement}'`).join('\n')}

Output only valid Vitest test code inside a markdown block.`;

    const { text } = await generateText({
      model: anthropic('claude-3-5-sonnet-20241022'),
      prompt,
    });

    const outPath = filePath.replace('.ts', '.mutant-killer.test.ts');
    const cleanedCode = text.replace(/```typescript/g, '').replace(/```/g, '').trim();
    await fs.writeFile(outPath, cleanedCode, 'utf-8');
    console.log(`Saved killer tests: ${outPath}`);
  }
}

generateKillerTests().catch(console.error);

If you want to review foundational debugging techniques before diving into mutation testing, read our guide on debugging basics for beginners.

The AI-Generated Killer Test Suite

Here is what the resulting killer test suite looks like. It explicitly tests the exact edge conditions that your original tests missed:

// src/fee-calculator.mutant-killer.test.ts
import { describe, it, expect } from 'vitest';
import { calculateTieredFee } from './fee-calculator.js';

describe('calculateTieredFee Mutant Killers', () => {
  it('verifies exact boundary condition at 10,000 for non-VIP users', () => {
    // Kills mutant: amount >= 10000 mutated to amount > 10000
    const feeAtBoundary = calculateTieredFee(10000, false);
    expect(feeAtBoundary).toBe(150); // 10000 * 0.015

    const feeBelowBoundary = calculateTieredFee(9999.99, false);
    expect(feeBelowBoundary).toBeCloseTo(299.9997, 2); // 9999.99 * 0.03
  });

  it('verifies zero fee for zero and negative inputs', () => {
    // Kills mutant: amount <= 0 mutated to amount < 0
    expect(calculateTieredFee(0, false)).toBe(0);
    expect(calculateTieredFee(-100, false)).toBe(0);
  });
});

Run npx stryker run again. Every mutant dies, and your mutation score jumps from 64% to 100%.

How to Run Mutation Testing in CI Without Breaking Build Times

Mutation testing evaluates dozens of code permutations, which can take several minutes on large repositories. Here is how to keep it fast in production:

• Turn on Incremental Mode: Set "incremental": true in your config so Stryker only re-evaluates files touched in the latest git commits.
• Scope to Pull Requests: Run mutation testing only on pull request branches targeting main, passing only the changed TypeScript files.
• Focus on Critical Logic: Prioritize auth checks, payment calculations, and core data transformers where silent errors create real financial disasters.

Check our post on acquiring job-ready coding skills to see why understanding test rigor is what distinguishes senior engineers from beginner tutorial copiers.

The Takeaway

Do not chase 100% vanity coverage metrics to impress a dashboard. Set up Stryker JS, let it intentionally break your code, and use targeted AI prompts to plug the leaks. That is how real engineering teams ship with confidence.