Engineering Interactive AI Coding Tutors: Progressive Challenges, AST Linting, and Socratic Hints

The Failure of the Chatbot as a Teacher

Most AI coding educational tools are fundamentally broken. A student gets stuck on a recursive array flattener, pastes the problem into a chat box, and the AI outputs the working solution in two seconds. The student pastes the code into their editor, watches the unit test turn green, and experiences a momentary dopamine hit.

No learning occurred. The friction of debugging was short-circuited. When that student faces a real production incident with unformatted stack traces and conflicting dependencies, they freeze. True engineering proficiency requires grappling with abstract constraints, isolating edge cases, and tracing state transitions mentally.

To teach effectively, an AI coding tutor must refuse to write code for the student. It must behave like a senior software engineer conducting a pair-programming session: analyzing the student's abstract syntax tree (AST), identifying logical misconceptions, and delivering tiered Socratic guidance.

System Architecture: The Socratic Tutor Engine

An effective tutoring architecture decouples code analysis from language generation:

Student Code Submission
         │
         ▼
┌──────────────────────────────────────────────┐
│ Phase 1: AST Structural Analysis             │  ──[Banned API / Syntax Error]──> Fast Local Feedback
│ (TypeScript Compiler API / Babel Traversal)   │
└──────────────────────────────────────────────┘
         │ (Valid AST Structure)
         ▼
┌──────────────────────────────────────────────┐
│ Phase 2: Sandboxed Vitest Test Runner        │  ──> Capture failing assertion & inputs
└──────────────────────────────────────────────┘
         │
         ▼
┌──────────────────────────────────────────────┐
│ Phase 3: Socratic Hint Synthesizer           │  ──> Tier 1: Conceptual Thought Question
│ (Structured LLM Prompt with Strict Zod Schema)│  ──> Tier 2: Edge Case Warning
└──────────────────────────────────────────────┘  ──> Tier 3: High-Level Pseudocode Hint
         │
         ▼
Progressive Guidance Delivered to Student IDE

Step 1: Inspecting Student Submissions with AST Traversal

When asking a student to implement an algorithm (e.g. manual array sorting or building a linked list), they must not bypass the exercise by calling built-in convenience methods like Array.prototype.sort or Array.prototype.flat.

We use the TypeScript Compiler API to analyze the student's submission without running arbitrary code:

// src/tutor/ast-inspector.ts
import * as ts from 'typescript';

export interface AstAuditResult {
  hasSyntaxErrors: boolean;
  bannedCallDetected: boolean;
  flaggedMethodName?: string;
  syntaxErrorMessages: string[];
}

const BANNED_APIS = new Set(['sort', 'reverse', 'flat', 'flatMap']);

export function auditStudentSubmission(sourceCode: string): AstAuditResult {
  const sourceFile = ts.createSourceFile(
    'student_submission.ts',
    sourceCode,
    ts.ScriptTarget.ES2022,
    true
  );

  const diagnostics = (sourceFile as any).parseDiagnostics || [];
  if (diagnostics.length > 0) {
    return {
      hasSyntaxErrors: true,
      bannedCallDetected: false,
      syntaxErrorMessages: diagnostics.map((d: ts.Diagnostic) => 
        typeof d.messageText === 'string' ? d.messageText : d.messageText.messageText
      )
    };
  }

  let bannedCall: string | undefined;

  function visitor(node: ts.Node) {
    if (ts.isCallExpression(node)) {
      const expression = node.expression;
      if (ts.isPropertyAccessExpression(expression)) {
        const methodName = expression.name.text;
        if (BANNED_APIS.has(methodName)) {
          bannedCall = methodName;
        }
      }
    }
    ts.forEachChild(node, visitor);
  }

  visitor(sourceFile);

  return {
    hasSyntaxErrors: false,
    bannedCallDetected: !!bannedCall,
    flaggedMethodName: bannedCall,
    syntaxErrorMessages: []
  };
}

If the student invokes arr.sort(), the AST inspector intercepts the payload immediately: "Exercise constraint violated: Manual sorting required. Built-in sort() is disallowed."

Step 2: Tiered Socratic Prompt Architecture

When test assertions fail, pass the failing test inputs and the student's code to an LLM with rigid system constraints. Enforce the response shape with Zod:

// src/tutor/hint-generator.ts
import { z } from 'zod';
import { ChatOpenAI } from '@langchain/openai';

export const SocraticHintSchema = z.object({
  identifiedConcept: z.string().describe('The core data structure or logic concept involved'),
  tier1_question: z.string().describe('A thought-provoking question pointing out the bug without syntax'),
  tier2_edgeCase: z.string().describe('A specific failing input example that breaks their mental model'),
  tier3_pseudocode: z.string().describe('High-level step-by-step logic. ABSOLUTELY NO WORKING CODE')
});

export type SocraticHint = z.infer<typeof SocraticHintSchema>;

const TUTOR_SYSTEM_PROMPT = `You are a world-class computer science professor teaching algorithmic reasoning.

CRITICAL CONSTRAINTS:
1. NEVER output working JavaScript or TypeScript code.
2. NEVER write variable declarations or function definitions.
3. Your goal is to guide the student to discover their own logical bug.
4. Tier 1 must be a single reflective question.
5. Tier 2 highlights an input where their logic breaks (e.g. empty array, duplicate keys).
6. Tier 3 outlines algorithmic steps in plain English sentences.`;

export async function generateSocraticHints(
  challengeTitle: string,
  studentCode: string,
  failingTestCase: { input: any; expected: any; actual: any }
): Promise<SocraticHint> {
  const model = new ChatOpenAI({ modelName: 'gpt-4o-mini', temperature: 0.2 });
  const structuredModel = model.withStructuredOutput(SocraticHintSchema);

  const prompt = `
Challenge: ${challengeTitle}
Failing Test: Input: ${JSON.stringify(failingTestCase.input)} | Expected: ${JSON.stringify(failingTestCase.expected)} | Actual: ${JSON.stringify(failingTestCase.actual)}

Student Implementation:
\`\`\`typescript
${studentCode}
\`\`\`
Analyze the bug and return structured Socratic guidance.`;

  return await structuredModel.invoke([
    { role: 'system', content: TUTOR_SYSTEM_PROMPT },
    { role: 'user', content: prompt }
  ]);
}

Step 3: Managing Progressive Hint Exposure in the UI

Do not dump all three hint tiers onto the student at once. Deliver guidance progressively as the student continues to struggle:

1. First Submission Failure: Display only tier1_question: "Consider what happens inside your while-loop when the left pointer and right pointer meet at the exact same index. Does your condition allow the loop to run one final comparison?"
2. Second Consecutive Failure: Unlock tier2_edgeCase: "Test your logic manually with input [2, 2, 2]. Does your pivot partitioning handle elements equal to the pivot value without entering an infinite loop?"
3. Third Consecutive Failure: Unlock tier3_pseudocode: "1. Initialize two pointers at start and end. 2. Advance the left pointer until finding an element greater than pivot. 3. Decrement right pointer until finding an element smaller than pivot. 4. Swap elements and repeat."

Step 4: Dynamic Challenge Generation Pipeline

Once a student passes a level, generate the next incremental challenge based on the exact concepts they struggled with:

Pedagogical Outcomes

By enforcing AST validation, prohibiting copy-paste solutions, and gating hints behind Socratic interrogation, you build an educational platform that develops genuine software engineers rather than prompt-dependent script copiers.

To explore AST-based code analysis in depth, check our guide on inside AI code explainers: AST parsing and tokenization. For broader learning perspectives, see our analysis on how self-taught coders learn faster with LLMs and master production prompts in modern AI coding workflows.