Then I ran docker compose up -d… and the runner immediately face-planted with:

ERROR: Failed to load config stat /etc/gitlab-runner/config.toml: no such file or directory
builds=0 max_builds=1

The container was up. The file was gone. My confidence? Also gone.

Two hours later, I learned (again) that Docker Compose will happily create a different volume than the one you think you’re using — and won’t warn you. Classic.

What was happening and why it mattered

GitLab Runner stores its registration and config in:

•             /etc/gitlab-runner/config.toml (inside the container)
  

So you typically mount a volume there so the runner keeps its config across restarts. My workflow was:

1. Register the runner using docker run ... register

2. Start the runner using docker compose up -d

Sounds normal, right?

Except I accidentally created two different volumes.

• docker run used: gitlab-runner-config

• docker compose created/used: gitlabrunner_gitlab-runner-config

So:

• Registration wrote config.toml into Volume A

• Runner started with Volume B

• Runner: “config file doesn’t exist”

• Me: “but I literally saw it earlier???” 🤡

Looking for trouble with a slight chance of survival? Follow me

Step 1 — Reproduce the “it registered but can’t start” symptom

You register like this:

docker run --rm -it \
  -v gitlab-runner-config:/etc/gitlab-runner \
  gitlab/gitlab-runner:latest register

This writes config.toml into the named Docker volume gitlab-runner-config.

Then you start via Compose:

docker compose up -d

Runner container starts… but can’t find /etc/gitlab-runner/config.toml.

Step 2 — The key debugging move: compare what volume each container actually mounted

Check volumes:

docker volume ls | grep gitlab
docker inspect <runner_container_name> --format '{{ json .Mounts }}' | jq

Or without jq:

docker inspect <runner_container_name> | grep -A5 Mounts

This is where the “oh no” moment happens:

• your manual register container wrote to gitlab-runner-config

• your compose runner container mounted <project>_gitlab-runner-config

Compose prefixes volume names with the project name unless you tell it not to. That’s the “name mangling” you hit.

Step 3 — Understand why Compose does that

Docker Compose has this concept of a “project”. By default, the project name is derived from the directory name (or overridden with -p / COMPOSE_PROJECT_NAME).

When you define a named volume in docker-compose.yml like:

volumes:
  gitlab-runner-config:

Compose will create it as:

• <project>_gitlab-runner-config

So your folder name gitlabrunner/ becomes:

• gitlabrunner_gitlab-runner-config

This is expected behavior… and also a great way to lose two hours of your life.

Step 4 — Fix it properly (the clean way)

You have two solid paths.

Multiple approaches (pros/cons + what I’d pick)

Approach A — Make Compose use the exact same existing volume (recommended)

Mark the volume as external and explicitly name it:

services:
  gitlab-runner:
    image: gitlab/gitlab-runner:latest
    restart: always
    volumes:
      - gitlab-runner-config:/etc/gitlab-runner
      - /var/run/docker.sock:/var/run/docker.sock

volumes:
  gitlab-runner-config:
    external: true
    name: gitlab-runner-config

Pros

• No surprises.

• Same volume used by docker run and Compose.

• Works even if the project name changes.

Cons

• You must create the volume first (or your register command already created it).

✅ This is my final choice. It’s explicit. DevOps loves explicit.

Approach B — Register the runner using Compose so everything lives in the Compose world

Instead of docker run ... register, do:

docker compose run --rm gitlab-runner register

Now the registration happens inside the same Compose project context, so it writes to the same prefixed volume.

Pros

• One tool (Compose) owns the workflow.

• Less naming confusion.

Cons

• Slightly more “Compose-y”.

• If you later run with a different project name, you can still get split-brain volumes.

This is great if you want “everything is Compose, always”.

Approach C — Set a fixed Compose project name (band-aid, but useful)

In .env:

COMPOSE_PROJECT_NAME=gitlabrunner

Or run:

docker compose -p gitlabrunner up -d

Pros

• Predictable prefix names.

Cons

• Still not as explicit as external: true.

• Someone else running it differently can recreate the problem.

Common mistakes / gotchas

• Assuming gitlab-runner-config means the same thing across docker run and Compose.

• Forgetting Compose adds a project prefix to volumes by default.

• Registering with docker run, running with docker compose up, then wondering why the config “disappears”.

• Not checking what is actually mounted via docker inspect.

• Having multiple folders with different Compose project names = multiple “almost identical” volumes.

• Typo trap: error mentions config.toml, but your brain reads it as config.yml because YAML is everywhere ^=^

Quick checklist

• Confirm the runner expects /etc/gitlab-runner/config.toml

• List volumes: docker volume ls

• Inspect mounts: docker inspect <container> | grep -A5 Mounts

• Ensure registration and runner use the same named volume

• Prefer external: true + explicit name: in Compose for shared volumes

• Or register via docker compose run --rm gitlab-runner register

Conclusion

One takeaway: Docker Compose didn’t lose your config — you put it in a different universe.

Volumes are state. State needs boring, explicit naming. Your future self will thank you.

If you’ve had your own “container is running but reality isn’t” moment, I’d genuinely love to hear it.

Misery loves company. Especially in DevOps.

TL;DR

docker run and docker compose can end up using different named volumes.

Compose prefixes volume names with the project name unless you force it not to.

Fix it with external: true + name: gitlab-runner-config, or register using docker compose run