Engineering Reference.

Quick Start

Three commands to go from repo to a live docs site with an optional AI chatbot.

# Python 3.10+ required
pip install deepdoc

# With chatbot dependencies (FAISS, FastAPI, fastembed)
pip install "deepdoc[chatbot]"

# cd into your target codebase first
cd /path/to/your-repo

# Initialize — writes .deepdoc.yaml with sensible defaults
deepdoc init --provider anthropic

# Add --with-chatbot to enable the /ask Q&A interface
deepdoc init --provider openai --with-chatbot

# Set your API key (matches llm.api_key_env in .deepdoc.yaml)
export ANTHROPIC_API_KEY=sk-ant-...

# Full pipeline — scan, plan, generate, build
deepdoc generate

✓ Scan complete        — 312 files · 47 endpoints · 8 integrations
✓ Plan complete        — 34 buckets across 6 sections
✓ Generate complete    — 34 pages · 2 MDX fixes applied
✓ Chatbot index built  — 1,247 chunks · 4.2 MB FAISS index
✓ Site scaffold written → site/

  Phase timings:  scan 4s · plan 18s · generate 72s · build 3s

# Preview locally (Node 18+ required, deps auto-installed)
deepdoc serve

  Docs   → http://localhost:3000
  Chat   → http://localhost:3000/ask

Architecture

DeepDoc is a stateful, five-phase pipeline. Each phase has a single responsibility and a well-defined output contract. Nothing is hand-edited inside docs/, site/, or .deepdoc/ — all fixes go into generators, builders, or prompts.

Five-Phase Pipeline

Each phase runs sequentially. The output of one is the sole input to the next — there are no cross-phase side effects.

Core Data Models

The entire pipeline communicates through two dataclasses defined in deepdoc/v2_models.py.

@dataclass
class DocBucket:
    # Identity
    bucket_type: str          # system | feature | endpoint | integration | database
    title: str                # page title shown in nav
    slug: str                 # URL slug, e.g. "checkout-flow"
    section: str              # nav section grouping
    description: str          # one-line description for planning context

    # Ownership — what evidence goes into this page
    owned_files: list[str]    # file paths relative to repo root
    owned_symbols: list[str]  # "module.Class.method" qualified names
    artifact_refs: list[str]  # config keys, env vars, OpenAPI ops

    # Generation instructions
    generation_hints: dict    # prompt_style, is_introduction_page, is_endpoint_ref
    required_sections: list[str]
    required_diagrams: list[str]

    # Topology
    depends_on: list[str]     # slugs this bucket depends on
    parent_slug: str | None

    # Metadata
    priority: int             # generation order within section
    publication_tier: str     # core | advanced | deprecated
    source_kind_summary: str  # summary of what kind of code is in owned_files

@dataclass
class DocPlan:
    buckets: list[DocBucket]         # ordered list — generation order
    nav_structure: dict[str, list]   # section → [slug, ...] for sidebar nav
    skipped_files: list[str]         # intentionally excluded (tests, fixtures…)
    orphaned_files: list[str]        # files not assigned to any bucket
    classification: str              # repo type: "api_service" | "monolith" | …
    integration_candidates: list[str]

    # Shim — adapts DocBucket to legacy DocPage interface
    @property
    def pages(self) -> list[_BucketAsPage]: ...

Configuration

.deepdoc.yaml lives in your repo root. deepdoc init writes sensible defaults; you only need to set llm.provider and your API key env var.

project_name: My API
description: "Internal API service for orders and fulfilment"

llm:
  provider: anthropic           # required
  model: claude-opus-4-7        # optional — uses provider default if omitted
  api_key_env: ANTHROPIC_API_KEY

generation_mode: feature_buckets  # v2 pipeline (default)
max_parallel_workers: 6
output_dir: docs
site_dir: site

# Optional chatbot
chatbot:
  enabled: true
  embeddings:
    provider: fastembed           # 100% local — no API key needed
    model: nomic-ai/nomic-embed-text-v1.5

CLI Reference

All commands accept --help for the full flag list. Run python -m deepdoc.cli --help to verify installation.

Bucket Types

Every documentation page belongs to a bucket. The planner picks the right type based on the repo classification and scanned signals. Each type maps to a different generation prompt and page template.

Supported Languages

DeepDoc parses source files natively (no subprocess or tree-sitter compilation required for most languages). Route detection is framework-aware.

Chatbot

An opt-in RAG chatbot that answers questions about your codebase. Embeddings run 100% locally via fastembed — no extra API key or paid embedding service required.

# Enable during init
deepdoc init --provider anthropic --with-chatbot

# Or add to .deepdoc.yaml manually
chatbot:
  enabled: true

# The chatbot model can differ from the doc generation model
chatbot:
  enabled: true
  answer:
    provider: openai
    model: gpt-4o
  embeddings:
    provider: fastembed
    model: nomic-ai/nomic-embed-text-v1.5  # downloaded once, ~270 MB

Query modes

Retrieval architecture

Query
  │
  ├─ FAISS vector search    (nomic-embed-text-v1.5, 768-dim)
  │    └─ top-K semantic candidates
  │
  ├─ SQLite FTS5 search     (BM25 lexical)
  │    └─ top-K keyword candidates
  │
  ├─ Graph neighbour expansion
  │    └─ chunks adjacent to strong hits
  │
  ├─ Optional: LLM rerank    (cross-encoder scoring)
  │
  └─ Hit selection          → evidence[] + references[]
       │
       └─ Answer generation (answer_mixin.py)
            ├─ evidence-first prompt
            ├─ exact file:line citations
            └─ SSE streaming response

Incremental Updates

After the first full generate, use deepdoc update instead of regenerating everything. It only spends LLM tokens on buckets affected by your changes.

# Diff against last sync, regenerate affected buckets only
deepdoc update

✓ Diff: 8 files changed since 3f2a91b
✓ Affected buckets: checkout-flow, order-api, payment-integration
✓ Replan triggered: new route detected in orders/views.py
✓ Regenerated 4 pages in 18s (saved ~54s vs full run)

# Force a full replan even on small changes
deepdoc update --replan

# Compare against a specific git ref
deepdoc update --since main

Deployment

The generated site/ directory is a standard Next.js project. Deploy it anywhere that runs Next.js, or use deepdoc deploy for one-command GitHub Pages deployment.

# Builds Next.js static export and pushes to gh-pages branch
deepdoc deploy

# .deepdoc.yaml — GitHub Pages config block
github_pages:
  base_path: /your-repo-name   # required if repo is not at root
  branch: gh-pages

# The site/ directory is a standard Next.js app
cd site
npm install
npm run build          # or: next build
npm run start          # production server

# Or point Vercel/Netlify at the site/ subdirectory
# Build command: next build
# Output directory: site/.next
# Root directory: site

name: Regenerate docs
on:
  push:
    branches: [main]

jobs:
  docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: {fetch-depth: 0}

      - uses: actions/setup-python@v5
        with: {python-version: "3.11"}

      - uses: actions/setup-node@v4
        with: {node-version: "20"}

      - run: pip install deepdoc

      - run: deepdoc update         # incremental — only changed buckets
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

      - run: deepdoc deploy
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Module Reference

All modules follow the rule: extend _v2 variants, never hand-edit generated output, route fixes go in repo_resolver.py.