Building a Markdown Content Pipeline: Node.js, Remark, Rehype, and Unified AST

Why Engineering Blogs Abandon Monolithic CMS Platforms

Monolithic content management systems introduce massive operational overhead. Managing PHP runtimes, patching plugin security vulnerabilities, and paying for database hosting simply to deliver static text articles wastes engineering time. Modern developer publications build on Git-backed content pipelines. Writers compose in clean Markdown or MDX, and a build script transforms the raw files into pre-rendered static HTML and typed JSON manifests.

The standard foundation for building content generation tooling in the JavaScript ecosystem is Unified.js. Rather than converting Markdown with fragile regular expressions, Unified treats content as an Abstract Syntax Tree (AST), allowing developers to transform syntax, validate frontmatter, generate tables of contents, and highlight code blocks deterministically.

The Unified Transformation Pipeline

The Unified pipeline operates as a pipeline of three modular stages:

[Raw Markdown with YAML Frontmatter]
                 │
                 ▼
[1. remark-parse] ──> Converts Markdown string into Markdown AST (mdast)
                 │
                 ▼
[2. remark-rehype] ──> Bridges Markdown AST into HTML AST (hast)
                 │
                 ▼
[3. rehype-stringify] ──> Serializes HTML AST into sanitized HTML string

Because each step manipulates an explicit AST node hierarchy, writing custom plugins (such as auto-linking headings, embedding YouTube containers, or rewriting asset URLs) requires only a few lines of tree-visitor code.

Production TypeScript: Automated Content Build Engine

Below is an end-to-end Node.js pipeline. It reads Markdown files, validates frontmatter using Zod, extracts estimated reading time, generates an interactive table of contents, and exports production JSON artifacts:

// scripts/build-content.ts
import fs from 'node:fs/promises';
import path from 'node:path';
import matter from 'gray-matter';
import { z } from 'zod';
import { unified } from 'unified';
import remarkParse from 'remark-parse';
import remarkGfm from 'remark-gfm';
import remarkRehype from 'remark-rehype';
import rehypeSlug from 'rehype-slug';
import rehypeAutolinkHeadings from 'rehype-autolink-headings';
import rehypeStringify from 'rehype-stringify';

// 1. Define Strict Article Frontmatter Schema
const FrontmatterSchema = z.object({
  title: z.string().min(5),
  date: z.string().datetime(),
  author: z.string().default('Ankur Ishwar'),
  categories: z.array(z.string()).min(1),
  tags: z.array(z.string()).default([]),
  cover: z.string().url().nullable().optional(),
  draft: z.boolean().default(false),
});

export interface ArticlePayload {
  slug: string;
  frontmatter: z.infer<typeof FrontmatterSchema>;
  wordCount: number;
  readingTimeMin: number;
  bodyHtml: string;
}

export async function processMarkdownFile(filePath: string): Promise<ArticlePayload> {
  const rawSource = await fs.readFile(filePath, 'utf-8');
  const { data, content } = matter(rawSource);

  // Validate metadata schema
  const frontmatter = FrontmatterSchema.parse(data);
  const slug = path.basename(filePath, path.extname(filePath));

  // Calculate word count & reading duration
  const words = content.trim().split(/\s+/).filter(Boolean).length;
  const readingTimeMin = Math.ceil(words / 200);

  // Compile Markdown AST to HTML
  const file = await unified()
    .use(remarkParse)               // Parse markdown to mdast
    .use(remarkGfm)                 // Tables, strikethrough, tasklists
    .use(remarkRehype)              // Transform mdast to hast
    .use(rehypeSlug)                // Add id attributes to h1-h6
    .use(rehypeAutolinkHeadings, {  // Add anchor links to headings
      behavior: 'wrap',
    })
    .use(rehypeStringify)           // Serialize hast to HTML string
    .process(content);

  return {
    slug,
    frontmatter,
    wordCount: words,
    readingTimeMin,
    bodyHtml: String(file),
  };
}

// Batch Execution Pipeline
export async function buildCatalog(contentDir: string, outputDir: string): Promise<void> {
  const entries = await fs.readdir(contentDir);
  const mdFiles = entries.filter((f) => f.endsWith('.md'));
  const articles: ArticlePayload[] = [];

  for (const file of mdFiles) {
    const fullPath = path.join(contentDir, file);
    const article = await processMarkdownFile(fullPath);
    if (!article.frontmatter.draft) {
      articles.push(article);
    }
  }

  // Write single master index manifest and individual post JSONs
  await fs.mkdir(outputDir, { recursive: true });
  for (const art of articles) {
    await fs.writeFile(
      path.join(outputDir, `${art.slug}.json`),
      JSON.stringify(art, null, 2)
    );
  }

  console.log(`Successfully compiled ${articles.length} articles to ${outputDir}`);
}

Writing Custom Rehype Plugins

One major advantage of Unified is creating custom AST visitors. For instance, to automatically transform external links to open in a new tab with secure rel="noopener noreferrer" attributes:

// scripts/plugins/rehype-secure-links.ts
import { visit } from 'unist-util-visit';
import type { Root, Element } from 'hast';

export function rehypeSecureLinks() {
  return (tree: Root) => {
    visit(tree, 'element', (node: Element) => {
      if (node.tagName === 'a' && typeof node.properties?.href === 'string') {
        const href = node.properties.href;
        // Match external protocols
        if (href.startsWith('http://') || href.startsWith('https://')) {
          node.properties.target = '_blank';
          node.properties.rel = 'noopener noreferrer';
        }
      }
    });
  };
}

Static Pipeline vs. Monolithic CMS

Summary: The Power of AST-Driven Content

Building your own content generation pipeline using Node.js, Remark, and Rehype liberates your engineering workflow from clumsy dashboard editors. You gain deterministic type validation, instant builds, and the ability to treat documentation and editorial articles as version-controlled code.