homelab/docs/forgejo-proxmox.md

Bringing up Forgejo on Proxmox

A run-once bootstrap, same shape as postgres-proxmox.md and deliberately downstream of it — Forgejo's database lives on the shared Postgres that runbook creates, so none of this works until that one is finished.

The end state: an LXC on the AppData pool of turtle-proxmox-02 (VMID 160, 192.168.50.52), running Forgejo natively under systemd, with its database on the shared Postgres LXC at 192.168.50.54 and a nightly pg_dump landing on its own disk in time for the node's 02:00 vzdump.

Why turtle-proxmox-02

Nothing forces it. Postgres is on -01, so putting Forgejo on -02 splits the two guests across the cluster: a node going down takes one of them with it rather than both. Note what that does not buy — Forgejo with its database unreachable is not a working forge, so this is about not losing both filesystems at once, not about staying up.

It is set as a literal node_name on the forgejo module in build/config/terraform/main.tf, not from var.proxmox_node. Change it only before the container exists: the provider treats node_name as a replacement, and prevent_destroy in src/forgejo/terraform/main.tf turns that into a failed plan. Moving a live Forgejo is a Proxmox migration (pct migrate) followed by editing the literal to match, not a terraform apply.

1. Secrets

homelab/forgejo needs five keys. Four of them may already exist from the Unraid stack; LFS_JWT_SECRET almost certainly does not — the Compose deployment let Forgejo generate it, and the native install renders app.ini in full with INSTALL_LOCK = true, so there is no first boot for Forgejo to invent one on.

vault kv get kv/homelab/forgejo

If LFS_JWT_SECRET is missing, generate one and patch it in without disturbing the others:

vault kv patch kv/homelab/forgejo \
  LFS_JWT_SECRET="$(docker run --rm codeberg.org/forgejo/forgejo:10 forgejo generate secret)"

Full key table in vault-secrets.md. install.yml asserts on all five before it touches the container, so a missing one fails on the first task with a message naming it rather than three tasks later on an undefined variable.

2. The container template

The template must be present on turtle-proxmox-02, not just on -01 where Postgres was built. If StorageOne is not shared across the cluster this is a separate download:

ssh root@turtle-proxmox-02 'pveam list StorageOne'
# if the ubuntu-26.04 template is absent:
ssh root@turtle-proxmox-02 'pveam update && pveam download StorageOne ubuntu-26.04-standard_26.04-1_amd64.tar.zst'

Whatever is there has to match lxc_template_file_id exactly — the filename moves with each point release, which is why that variable has no default.

3. Terraform

State is on the CNPG cluster, so the Pis have to be up (see postgres-proxmox.md § "Why this order").

cd build/config/terraform
terraform init -backend-config=...        # as per postgres-proxmox.md
terraform plan -out=deploy.plan -target=module.forgejo
terraform apply deploy.plan

Read the plan before applying. It should be one resource to add and nothing to change or destroy; anything touching module.postgres — in particular a replacement — means stop and work out why, because that guest is live.

Note deploy.plan embeds the state and every input variable, sensitive ones included, so it holds the Proxmox API token in the clear. .gitignore covers *.plan; delete it once applied rather than leaving it in the tree.

Confirm the guest exists and Ansible can see it:

ssh root@turtle-proxmox-02 'pct list | grep 160'
cd ../ansible && ansible-inventory --list --yaml proxmox_guests | grep -A2 forgejo

If it is absent from the inventory but present in pct list, the terraform tag is missing — that filter is what tells this repo's guests from hand-made ones, and a guest without it is silently never deployed to.

4. Install Forgejo into it

cd build/config/ansible
ansible-playbook playbooks/proxmox.yml -e only_stacks=forgejo

This provisions the forgejo database and role on 192.168.50.54, then installs the binary, app.ini, the systemd unit and the dump timer. The database step runs from your controller, not from inside the container (delegate_to: localhost), so 192.168.50.54:5432 has to be reachable from wherever you run this.

Check:

ssh root@192.168.50.52 'systemctl is-active forgejo && systemctl list-timers forgejo-dbdump.timer'
curl -sI http://192.168.50.52:3000/ | head -1

4b. The first admin user

A fresh install has no way in: INSTALL_LOCK = true skips the setup wizard, which is where the first admin would normally be created, and DISABLE_REGISTRATION = true means you cannot self-register either (both in src/forgejo/ansible/proxmox/templates/app.ini.j2). Nothing in install.yml creates a user, so the account has to be made from the CLI inside the container:

ssh root@192.168.50.52

sudo -u git forgejo admin user create   --admin   --username <you>   --email <you>@turtlesystems.co.uk   --password '<strong-password>'   --config /etc/forgejo/app.ini   --work-path /var/lib/forgejo

Three things that are easy to get wrong:

• sudo -u git, not root. RUN_USER is git; running the CLI as root leaves root-owned files under /var/lib/forgejo that Forgejo then cannot write. app.ini is 0640 root:git, so the git user can read it.
• --config is not optional. Forgejo looks for custom/conf/app.ini under the work path by default; this install puts it at /etc/forgejo/app.ini (FORGEJO_CONFIG_DIR in vars.yml). Without the flag the CLI reads a config that isn't there and never reaches the database.
• Recent versions default --must-change-password to true, so expect a forced change on first web login. Pass --must-change-password=false to skip it. Confirm the flags for the pinned FORGEJO_RELEASE with forgejo admin user create --help rather than assuming.

Then log in at http://192.168.50.52:3000/user/login — by IP, because ROOT_URL is https://git.turtlesystems.uk and nothing resolves there until the cutover in step 7. The login form works over the IP; some links and redirects Forgejo renders will point at the not-yet-live hostname.

Skip this step if you are restoring a database in step 6. Users live in the database, so a dump brings its own admin back and an account created here is overwritten by the restore. Create one only if this is genuinely an empty forge — or after the restore, if the dump turns out to have no usable admin.

5. Node backups

VMID 160 is on -02, so it is -02's vzdump job that covers it. pve_backup_all: true in group_vars/proxmox_nodes.yml means no edit is needed — but the job has to actually exist on that node:

ansible-playbook playbooks/pve_host.yml --limit turtle-proxmox-02
ssh root@turtle-proxmox-02 'cat /etc/pve/jobs.cfg'

The 01:30 dump and the 02:00 vzdump are a pair. Move one, move the other — see src/forgejo/ansible/proxmox/README.md.

6. Repository data

Ansible does not move any of this. A fresh install is a working, empty forge; the repositories are a separate restore into /var/lib/forgejo/data/forgejo-repositories.

The original instructions here were an rsync off nas2, which no longer exists as a machine to read from — whatever repository data survives has to come from wherever it went when that host was retired. After restoring:

ssh root@192.168.50.52 'chown -R git:git /var/lib/forgejo/data && systemctl restart forgejo'

Restore the database the same way, from whatever dump you have, before starting Forgejo against it — an empty database with populated repositories gives back every file and no issues, pull requests, users or permissions.

7. Cutover

Last, once the above is verified:

• Point git.turtlesystems.uk at whatever terminates HTTPS for it — the Caddy reverse proxy at 192.168.50.51, which proxies through to this container's FORGEJO_HTTP_PORT.

• Point src.turtlesystems.uk at this container (192.168.50.52). This is the name in FORGEJO_SSH_DOMAIN, and the one SSH clone URLs use.

• SSH clone URLs move from port 2222 to 22, and onto their own hostname. The Unraid stack published 2222 to dodge the host's own sshd; this container has its own IP and uses its own sshd, with Forgejo managing the git user's authorized_keys. Every existing remote needs editing — there is no redirect for this.

  The two hostnames are not redundancy. An SSH client sends a version banner and nothing identifying the host it meant — there is no equivalent of TLS SNI — so a reverse proxy cannot route SSH by name the way it routes HTTP. The web name therefore has to point at the proxy and the SSH name at Forgejo. Collapsing them into one would mean putting a layer-4 TCP proxy on the proxy's port 22 (Caddy can, via the third-party caddy-l4 module, at the cost of a custom binary and of losing client IPs on SSH), not configuring the hostname more cleverly.

• The Forgejo Actions runner registration is tied to the instance. Re-register it against the new host, or CI stops running (.forgejo/workflows/).

8. Sign-in through Authentik (optional)

Forgejo supports OIDC, and app.ini is already set up for it — but only the policy half. The provider itself cannot be configured from app.ini: Forgejo keeps authentication sources in its database. That makes this the one part of Forgejo's configuration Ansible does not own, and the reason it is a runbook step rather than a task in install.yml. It is run once and then carried forward by the nightly pg_dump, the same as every other row in that database.

Do this after step 7, not before. The redirect URI has to match ROOT_URL (https://git.turtlesystems.uk), so DNS and TLS must already be live. Configuring it against http://192.168.50.52:3000 means doing it twice.

On the Authentik side

Create an OAuth2/OIDC Provider plus an Application for it, as for any other app on auth.turtlesystems.uk. The redirect URI is:

https://git.turtlesystems.uk/user/oauth2/authentik/callback

The last path segment is the name of the auth source in Forgejo, not a fixed string — it has to match the --name below. Store the generated client secret in Vault (vault-secrets.md → OIDC_CLIENT_SECRET).

On the Forgejo side

ssh root@192.168.50.52

sudo -u git forgejo admin auth add-oauth   --name authentik   --provider openidConnect   --key <client-id>   --secret <client-secret>   --auto-discover-url https://auth.turtlesystems.uk/application/o/forgejo/.well-known/openid-configuration   --config /etc/forgejo/app.ini   --work-path /var/lib/forgejo

Same sudo -u git and --config requirements as step 4b, for the same reasons. Verify the flag names against the pinned FORGEJO_RELEASE with forgejo admin auth add-oauth --help before running — and list what exists afterwards with forgejo admin auth list, which is also how you find the id for update-oauth if the secret is ever rotated.

No restart is needed: the source is a database row, not a file Forgejo reads at boot.

What the app.ini side already does

Set in src/forgejo/ansible/proxmox/templates/app.ini.j2, so a redeploy keeps them — the comments there carry the detail:

Setting	Effect
DISABLE_REGISTRATION = false	It blocks OIDC auto-registration too, not just the local signup form. Left true, Authentik logins authenticate and are then refused an account.
ALLOW_ONLY_EXTERNAL_REGISTRATION = true	Restores "no self-service signup" without blocking Authentik.
ENABLE_AUTO_REGISTRATION = true	Creates the Forgejo account on first successful SSO login.
ACCOUNT_LINKING = auto	Attaches the OIDC identity to an existing local account with the same email.

ACCOUNT_LINKING is the one that matters on the first login: if the break-glass admin from step 4b uses the same email address as your Authentik account, signing in through Authentik lands you in that admin account. With linking off you would arrive as a second, unprivileged user and have to promote it from the CLI.

Keep the local admin

Do not delete the step 4b account or convert it to SSO-only. The login path is now Forgejo → Authentik → CloudNativePG → a healthy k3s cluster, and any of those failing takes SSO down with it — including the cases you would most want to log into the forge to investigate. This is the same reasoning roles/k3s_traefik uses for keeping the port-forward-only dashboard route alive alongside the Authentik-published one.

Git over HTTPS

Users who arrive through Authentik have no Forgejo password, so HTTPS clones need a personal access token. SSH keys are unaffected — and since step 7 put git-over-SSH on port 22 with Forgejo managing the git user's authorized_keys, SSH is the smoother default to point people at.

Rolling back

There is no state: absent for lxc_app — removal would need a per-app uninstall.yml and Forgejo has none. To back out: stop the service (systemctl stop forgejo), point DNS back at whatever was serving before, and leave the container in place. Destroying it needs the prevent_destroy block removed by hand first, which is deliberate.