Compile-time Markdown in Phoenix, and why I reversed it

There is a version of this problem everyone solves badly at least once: you want a blog, you have Markdown files, and you reach for a database. Then you have a cache. Then you have a cache invalidation bug at eleven at night.

The alternative is older and duller. Read the files while compiling, turn them into structs, let the compiler hold them. A request becomes a lookup in a module attribute.

I built exactly that, and published a version of this post arguing for it. Three weeks later I tore it out. This is the honest version.

The original argument#

It went like this, and I still think it was sound given its premises:

• A request is a module-attribute lookup — no parse cost, no query.
• No database is needed for content at all.
• There is no cache, so there are no cache-invalidation bugs.
• A malformed post fails the build rather than a request.

The accepted cost: publishing a post requires a deploy. For a personal site, I argued, that is a feature — the writing lives in git, every revision is recoverable, there is no admin panel to keep patched.

Both premises were false#

"No database is needed." By the time the site was real it had committed to Postgres for moderated comments, a notes feed, post visibility, and authentication for an admin. The database arrived anyway. The saving never materialised — I was paying the deploy-to-publish cost to avoid a dependency I had already taken on.

"Publishing by deploy is acceptable." It is not. Rebuilding and redeploying a container image to publish a paragraph is over-engineering wearing the costume of simplicity. A full docker compose build takes minutes, mostly compiling Rust NIFs that have nothing to do with the paragraph I changed.

And an inconsistency I should have caught much earlier: the notes feed publishes instantly from the database, while a blog post needed a full image rebuild. Two content types, wildly different publishing stories, for a reason no reader could possibly perceive.

What actually replaces it#

Markdown files are still the source of truth and still live in git. They are read at boot, parsed once, and held in :persistent_term. A reload rescans, validates, and swaps the whole set atomically.

defmodule Tb.Content.Store do
  @key :tb_content_set

  @doc "The configured content root. CONTENT_PATH wins, so a container can be pointed at a mount."
  def root do
    System.get_env("CONTENT_PATH") ||
      Application.get_env(:tb, :content_path) ||
      Path.join(:code.priv_dir(:tb), "content")
  end

  def reload do
    case Loader.load(root()) do
      {:ok, set} ->
        :persistent_term.put(@key, set)
        {:ok, summarise(set)}

      {:error, problems} ->
        Logger.warning("content reload REFUSED, keeping previous set:\n" <> Enum.join(problems, "\n"))
        {:error, problems}
    end
  end
end

:persistent_term is the right structure here for a reason worth stating explicitly. Reads are a term lookup with no copying and no process in the path — so serving a page costs about what a compiled module attribute cost, which was the one real advantage of parsing at compile time. Writes trigger a global GC scan, which is expensive; but writes happen when I publish, and reads happen on every request. That is exactly the trade you want.

The property I nearly lost#

Compile-time parsing gave one genuine guarantee: a broken post cannot reach production, because it fails the build.

Moving to runtime could easily have thrown that away. It does not, because the loader validates the entire tree before anything is swapped in:

case Loader.load(root()) do
  {:ok, set}       -> :persistent_term.put(@key, set)   # atomic swap, all or nothing
  {:error, probs}  -> {:error, probs}                    # previous set stays live
end

A reload never half-applies. If anything fails validation the previous content carries on being served and the caller gets the list of problems. A malformed file cannot take the site down — it can only fail to appear.

That promoted validation from a nice-to-have into the thing standing between a bad file and a broken site, so it is not shy about it: duplicate slugs naming both files, unparseable dates, missing front matter naming the file and the field, series parts that are not contiguous, a series folder with no _series.md, an <h1> in a body where only front matter should supply one, and descriptions under 60 characters — because that field is the meta description and the LinkedIn preview, and a short one is a silent SEO failure.

At boot the rules differ, deliberately:

def load! do
  case Loader.load(root()) do
    {:ok, set} -> :persistent_term.put(@key, set)
    {:error, problems} -> raise "content failed to load:\n" <> format(problems)
  end
end

Raising is right here. At startup there is no previous good content to fall back to, so refusing to start beats serving an empty site.

Keeping the score honestly#

The third row is the interesting one. The usual argument against a cache is invalidation, and it is a real argument — but it applies to caches with per-item lifetimes. Replacing the entire set atomically has no such failure mode. There is no entry that can be stale relative to another.

What publishing looks like now#

The content directory is a read-only bind mount, so git pull on the host is visible to the running container immediately:

    volumes:
      # Read-only on purpose: the application never writes content. Posts arrive by
      # git pull on the host, and the container only reads what appears.
      - ./priv/content:/content:ro
    environment:
      CONTENT_PATH: /content

And publishing is two commands that take seconds, with no rebuild and no restart:

git pull
docker compose exec web /app/bin/tb rpc 'Tb.Content.Store.reload()'

Existence and visibility stay separate, which is the part I would keep even if everything else changed. A reload makes a post known; a second, explicit act makes it public. So a finished post can sit on the server for a week until I want it out, and reloading content never accidentally publishes a draft or unpublishes something live.

What I would tell myself three weeks earlier#

Compile-time parsing is a good answer to "how do I serve Markdown fast without a database". It is a bad answer to "how do I publish", and I conflated the two because the first version of this site genuinely had no database.

The tell was there the whole time: I had two content types with different publishing stories and no reader-visible reason for it. When part of your system can do something instantly and another part needs a deploy to do the same thing, the inconsistency is usually the bug — not the part that is fast.