Internal Documentation That Doesn't Rot: Docusaurus, Auto-Generated Docs, and OAuth2 Proxy

Every engineering org has the same dirty secret: the documentation is wrong. Not missing — that would be easy to diagnose. It’s there, spread across Confluence spaces nobody visits, Google Docs with ambiguous ownership, README files that described the architecture two refactors ago, and a wiki that someone set up in 2019 and forgot about.

The problem isn’t that people don’t write docs. It’s that docs live outside the development workflow. They rot because updating them is a separate task, in a separate tool, with no review process and no automation.

Here’s how we fix it: treat documentation as code, auto-generate what you can, and protect it behind your existing identity provider.

graph LR
    R1[Repo 1 - README] --> CI[GitLab CI]
    R2[Repo 2 - README] --> CI
    R3[Terraform Modules] --> CI
    CI -->|Build| DOC[Docusaurus]
    DOC --> OAP[OAuth2 Proxy]
    OAP --> KC[Keycloak]
    KC -->|Authenticated| USER[Employee]

Docusaurus as Your Internal Docs Portal

Docusaurus is a static site generator built for documentation. It takes Markdown and MDX files, generates a fast, searchable site, and deploys as static files anywhere. The reasons it wins for internal docs:

• Git-native. Docs live in a repo. Changes go through pull requests. You get review, history, and blame for free.
• Sidebar auto-generation. Drop files into a directory structure and Docusaurus builds the navigation automatically.
• Search. Built-in local search plugin or Algolia DocSearch. Your team can actually find things.
• Versioning. Tag docs to releases when you need to.
• MDX. Embed React components in Markdown — interactive diagrams, live config examples, tabbed code blocks.

But the real power comes when you stop writing docs by hand and start generating them.

Auto-Generating Docs From Your Repositories

The idea is simple: a CI pipeline clones your repositories, extracts documentation artifacts, copies them into the Docusaurus source tree, and builds the site. Every merge to main in any repo triggers a rebuild. The docs are always current because they come directly from the source.

Here’s a GitLab CI pipeline that does exactly this:

# .gitlab-ci.yml in your docusaurus repo
stages:
  - collect
  - build
  - deploy

variables:
  DOCS_DIR: "docs/projects"

collect_docs:
  stage: collect
  image: alpine/git:latest
  script:
    - mkdir -p ${DOCS_DIR}
    - |
      REPOS=(
        "group/infra-terraform"
        "group/ansible-roles"
        "group/api-gateway"
        "group/frontend-app"
      )
      for REPO in "${REPOS[@]}"; do
        REPO_NAME=$(basename "$REPO")
        git clone --depth 1 "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.example.com/${REPO}.git" "/tmp/${REPO_NAME}"

        # Copy README and any docs/ directory
        mkdir -p "${DOCS_DIR}/${REPO_NAME}"
        cp "/tmp/${REPO_NAME}/README.md" "${DOCS_DIR}/${REPO_NAME}/index.md" 2>/dev/null || true
        cp -r "/tmp/${REPO_NAME}/docs/." "${DOCS_DIR}/${REPO_NAME}/" 2>/dev/null || true
      done
    # Generate Terraform module docs
    - |
      apk add --no-cache terraform-docs
      for dir in /tmp/infra-terraform/modules/*/; do
        MODULE=$(basename "$dir")
        terraform-docs markdown table "$dir" > "${DOCS_DIR}/infra-terraform/modules-${MODULE}.md"
      done
  artifacts:
    paths:
      - ${DOCS_DIR}

build_site:
  stage: build
  image: node:20-alpine
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - build/

deploy:
  stage: deploy
  image: alpine:latest
  script:
    - # Deploy static files to your hosting (S3, Nginx, Kubernetes, etc.)
    - echo "Deploying to internal docs server"

You can extend this pattern to any tool that generates Markdown:

• terraform-docs for Terraform module references — inputs, outputs, providers, all auto-generated.
• Ansible role README templates generated from meta/main.yml and defaults.
• OpenAPI/Swagger rendered to Markdown with tools like widdershins or Docusaurus’s own OpenAPI plugin.
• Helm chart docs from helm-docs.

Set up pipeline triggers so that a merge in any source repo kicks off the docs pipeline. The documentation portal rebuilds itself.

Protecting Docs With OAuth2 Proxy and Keycloak

Internal docs shouldn’t be public. But they also shouldn’t require yet another login. If you’ve already set up Keycloak as your identity provider (and if you haven’t, read our Keycloak SSO post), you can gate access to your documentation portal with OAuth2 Proxy in about 20 minutes.

The architecture is straightforward:

Browser → Ingress (Nginx/Traefik) → OAuth2 Proxy → Docusaurus static files
                                         ↓
                                    Keycloak OIDC

OAuth2 Proxy sits in front of the static files. Unauthenticated requests get redirected to Keycloak. After login, the user gets a session cookie and sees the docs. No changes to Docusaurus itself.

OAuth2 Proxy Configuration

# oauth2-proxy.cfg (or equivalent env vars / Helm values)
provider = "keycloak-oidc"
client_id = "internal-docs"
client_secret = "your-client-secret"
redirect_url = "https://docs.internal.example.com/oauth2/callback"
oidc_issuer_url = "https://keycloak.example.com/realms/company"

# Cookie settings
cookie_secret = "a-random-32-byte-base64-string"
cookie_secure = true
cookie_domains = [".internal.example.com"]

# Allow all authenticated users from this realm
email_domains = ["*"]
allowed_groups = ["/engineering"]

# Upstream: serve static files
upstreams = ["file:///var/www/docusaurus/"]

Create a client in Keycloak with access type “confidential”, set the valid redirect URI to https://docs.internal.example.com/oauth2/callback, and you’re set. Use the allowed_groups claim to restrict access to specific Keycloak groups if needed.

Kubernetes Ingress

If you’re running on Kubernetes, the cleanest approach uses ingress annotations to wire oauth2-proxy as an auth middleware:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: internal-docs
  annotations:
    nginx.ingress.kubernetes.io/auth-url: "https://oauth2.internal.example.com/oauth2/auth"
    nginx.ingress.kubernetes.io/auth-signin: "https://oauth2.internal.example.com/oauth2/start?rd=$scheme://$host$request_uri"
    nginx.ingress.kubernetes.io/auth-response-headers: "X-Auth-Request-User,X-Auth-Request-Email"
spec:
  tls:
    - hosts:
        - docs.internal.example.com
      secretName: docs-tls
  rules:
    - host: docs.internal.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: docusaurus
                port:
                  number: 80

Deploy oauth2-proxy as a separate service (there are solid Helm charts for it), point the ingress annotations at it, and every request to your docs site is authenticated against Keycloak. Your team uses the same credentials they use for GitLab, Grafana, and everything else in your internal developer platform.

What You Get

With this setup in place:

• Single source of truth. Documentation lives in the repos it describes, or is generated from them. There’s one place to look.
• Always current. Every merge triggers a rebuild. If the code changed, the docs changed.
• Searchable. Full-text search across all projects, all modules, all runbooks. No more grepping Confluence.
• Access-controlled. Only authenticated employees see it. No separate accounts. No shared passwords. Just Keycloak.
• Reviewable. Documentation changes go through the same PR process as code. Somebody has to approve them.

The total infrastructure cost is one static site, one OAuth2 Proxy instance, and a CI pipeline. You likely already have everything else.

Documentation that rots is worse than no documentation — it actively misleads. Documentation that regenerates itself from the source code is the only kind worth maintaining.

---

At robto, we build internal platforms and developer tooling that engineering teams actually use. If your documentation is scattered, outdated, or locked behind tools nobody opens — let’s fix that.