Building an Automated Prompt Generator: AST Parsers, Context Windows, and Few-Shot Sampling

Most tutorials on prompt generation show you a basic string template: you take a hardcoded instruction, insert a variable with template literals, and send it to an OpenAI endpoint. That approach works fine for a weekend demo. But when you build production AI features (automated code documentation, test case generators, or intelligent code review bots), static string templates fail immediately.

Real production files exceed context window limits. Irrelevant code pollutes the model attention mechanism. When you paste 2,000 lines of spaghetti JavaScript into a prompt, the model hallucinates or truncates because you failed to prune the context. An automated prompt generator is not a Mad Libs text generator. It is an ingestion compiler: it parses source code into structured syntax trees, calculates token weights, injects dynamic few-shot exemplars, and outputs deterministic payloads.

Here is how to design and build an automated prompt generation engine in TypeScript.

1. The Four-Stage Prompt Compilation Architecture

Treat prompt creation like a compiler pipeline. Raw source documents go in; an optimized, token-bounded prompt emerges:

1. Static Analysis and Parsing: Parse input source code into an Abstract Syntax Tree (AST). Extract only the relevant symbols (function signatures, type interfaces, exported methods) while discarding internal implementation bloat.
2. Token Budgeting: Calculate the maximum token allotment for the system prompt, dynamic context, exemplars, and the expected output completion.
3. Dynamic Exemplar Retrieval: Query an in-memory vector store or SQLite index for two high-quality examples matching the current programming language and pattern.
4. Schema Binding: Wrap instructions with JSON schema constraints so the model output can be validated deterministically.

2. Parsing Code Context with Abstract Syntax Trees (AST)

If you want an LLM to generate unit tests for a 500-line TypeScript service, sending the entire 500 lines wastes 80% of your token budget on internal utility calculations. The model only needs to understand the public interface: method names, parameter types, and return signatures.

Here is a lightweight context extractor using the TypeScript compiler API:

import * as ts from 'typescript';

interface MethodSummary {
  name: string;
  parameters: string[];
  returnType: string;
}

export function extractMethodSignatures(sourceCode: string): MethodSummary[] {
  const sourceFile = ts.createSourceFile(
    'temp.ts',
    sourceCode,
    ts.ScriptTarget.Latest,
    true
  );

  const methods: MethodSummary[] = [];

  function visit(node: ts.Node) {
    if (ts.isMethodDeclaration(node) && node.name) {
      const name = node.name.getText(sourceFile);
      const parameters = node.parameters.map(p => p.getText(sourceFile));
      const returnType = node.type ? node.type.getText(sourceFile) : 'void';

      methods.push({ name, parameters, returnType });
    }
    ts.forEachChild(node, visit);
  }

  visit(sourceFile);
  return methods;
}

Extracting method signatures reduces a 5,000-token class into a 150-token interface summary. This leaves ample room in the context window for high-value reasoning and prevents context dilution.

3. Dynamic Token Budgeting

Every language model has a hard context limit (e.g., 8,192 or 32,768 tokens). If your generated prompt plus the expected completion exceeds this boundary, the API throws an HTTP 400 error: context_length_exceeded.

A production prompt generator enforces strict token compartmentalization:

export class TokenBudgetManager {
  private maxContext: number;
  private reservedOutputTokens: number;

  constructor(maxContext = 8192, reservedOutput = 1500) {
    this.maxContext = maxContext;
    this.reservedOutputTokens = reservedOutput;
  }

  public calculateAvailableContext(systemPromptTokens: number, exemplarTokens: number): number {
    const available = this.maxContext - this.reservedOutputTokens - systemPromptTokens - exemplarTokens;
    if (available <= 0) {
      throw new Error('Token budget overflow: system prompt and exemplars exceed total window.');
    }
    return available;
  }
}

If the user input code exceeds the available budget, your compiler slices the least relevant AST branches instead of crashing the API call.

4. Dynamic Few-Shot Exemplar Selection

Zero-shot prompts (giving the model instructions without examples) produce inconsistent results. Fixed few-shot prompts (hardcoding the same two examples into every prompt) waste tokens on irrelevant patterns.

The solution is dynamic exemplar sampling. Store thirty verified input-output pairs in a lightweight vector index. Before generating the final prompt, compute the cosine similarity between the current task and your exemplar database:

interface Exemplar {
  taskType: 'UNIT_TEST' | 'DOCUMENTATION' | 'REFACTOR';
  language: string;
  input: string;
  expectedOutput: string;
}

export function selectBestExemplar(task: 'UNIT_TEST', language: string, pool: Exemplar[]): Exemplar | null {
  // Match language and task precisely
  const matches = pool.filter(e => e.taskType === task && e.language === language);
  return matches.length > 0 ? matches[0] : null;
}

Injecting a single language-specific exemplar improves output formatting accuracy from 65% to over 95% without complex prompt engineering tricks.

5. Enforcing Output Schemas Without Begging

Do not write prompts ending with: "Please please return ONLY JSON and do not include any markdown formatting." Models routinely ignore negative constraints under temperature variance.

Instead, use structured JSON schema enforcement via OpenAI tool calling or Anthropic tool use:

export const testGeneratorTool = {
  type: 'function',
  function: {
    name: 'save_generated_tests',
    description: 'Stores generated Jest unit test cases',
    parameters: {
      type: 'object',
      properties: {
        testSuiteName: { type: 'string' },
        importsRequired: { type: 'array', items: { type: 'string' } },
        testCases: {
          type: 'array',
          items: {
            type: 'object',
            properties: {
              description: { type: 'string' },
              testCode: { type: 'string' },
            },
            required: ['description', 'testCode'],
          },
        },
      },
      required: ['testSuiteName', 'importsRequired', 'testCases'],
    },
  },
};

When you bind your automated prompt generator to explicit tool definitions, the output is guaranteed to parse via JSON.parse() every single time.

Summary Pipeline Checklist

• Parse raw code into ASTs to eliminate whitespace and internal implementation bloat.
• Calculate strict token budgets before making API calls to prevent 400 context errors.
• Retrieve dynamic few-shot exemplars tailored to the specific language and task.
• Use tool calling and JSON schemas instead of conversational begging to guarantee parseable output.