Self-Hosted VPN with Headscale: One Mesh Network for Your Entire Homelab

(Part 1/2)

Andrea Lucarelli

~4 min read · February 19, 2026 (Updated: February 19, 2026) · Free: Yes

(Part 1/2)

How to run your own Tailscale-compatible coordination server on a $5 VPS — so every device, VM, and container you own lives on a single private network, accessible from anywhere.

The problem with homelabs is that they're local. You can run 20 services on 4 Proxmox nodes, but the moment you leave the house, everything disappears behind your router's NAT. You either expose ports to the internet (bad idea for internal services), use a self-hosted VPN (painful to manage), or pay for a commercial tunnel service.

I went down a different path. I run Headscale — the open-source, self-hosted implementation of the Tailscale control plane — on a small cloud VPS. Every device I own connects to it: Proxmox nodes, Synology NAS, Mac laptops, VMs, LXC containers. They all get a stable 100.64.x.x IP. They can reach each other from anywhere, without opening a single firewall port.

This article walks through the full setup: Headscale on a VPS with Traefik for HTTPS, Tailscale clients on all homelab devices, MagicDNS with custom records, and a custom domain to wrap it all together.

Why Headscale Instead of Running a WireGuard Server?

Standard WireGuard requires you to manually manage peer configs — add a device, edit every other peer's config, restart the daemon. It's fine for two or three devices. It doesn't scale.

Tailscale solves the peer management problem with a coordination server that handles key exchange, NAT traversal, and device discovery. The catch: it's a commercial service with free tier limits and phone-home behavior.

Headscale implements the same protocol — fully compatible with the standard Tailscale client — but you run the control plane yourself. You're not limited to 3 users or 100 devices. No telemetry. No dependency on Tailscale's infrastructure.

The VPN traffic itself still flows peer-to-peer via WireGuard. The VPS only handles signaling. So a $5/month VPS with 1 vCPU and 1 GB RAM is more than enough.

VPS Setup

I use a cheap and simple hosting plan (~$5/month, shared) running Ubuntu 24.04 LTS. Any VPS with a public IP works. Requirements:

Docker CE installed
Port 80 and 443 open
Port 41641/UDP open (Tailscale DERP/WireGuard fallback — optional but useful)

The full stack on the VPS runs in Docker Compose:

Traefik — reverse proxy with Let's Encrypt for all HTTPS
Headscale — the control plane
Headscale Admin — web GUI (protected by Basic Auth)
Other services — I also run n8n and Vaultwarden on the same host

Deploying Headscale with Traefik

You can use docker container starting from the following yaml file and customize it for your needs.

docker-compose.yml (relevant excerpt):

services:
  traefik:
    image: traefik:3.6
    command:
      - --providers.docker=true
      - --providers.docker.exposedByDefault=false
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --certificatesresolvers.le.acme.httpchallenge=true
      - --certificatesresolvers.le.acme.httpchallenge.entrypoint=web
      - --certificatesresolvers.le.acme.email=you@yourdomain.com
      - --certificatesresolvers.le.acme.storage=/letsencrypt/acme.json
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - letsencrypt:/letsencrypt

  headscale:
    image: headscale/headscale:0.28
    volumes:
      - ./headscale/config:/etc/headscale
      - ./headscale/data:/var/lib/headscale
    command: serve
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.headscale.rule=Host(`headscale.yourdomain.com`)"
      - "traefik.http.routers.headscale.tls.certresolver=le"
      - "traefik.http.services.headscale.loadbalancer.server.port=8080"
      # WebSocket support for Tailscale noise protocol
      - "traefik.http.middlewares.headscale-cors.headers.accesscontrolalloworiginlist=https://headscale.yourdomain.com"

  headscale-admin:
    image: goodieshq/headscale-admin:latest
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.hs-admin.rule=Host(`headscale.yourdomain.com`) && PathPrefix(`/admin`)"
      - "traefik.http.routers.hs-admin.tls.certresolver=le"
      - "traefik.http.routers.hs-admin.middlewares=hs-auth"
      - "traefik.http.middlewares.hs-auth.basicauth.users=youruser:$$apr1$$..."

For Headscale's config.yaml, the critical settings:

server_url: https://headscale.yourdomain.com

listen_addr: 0.0.0.0:8080
metrics_listen_addr: 0.0.0.0:9090
grpc_listen_addr: 0.0.0.0:50443

base_domain: ts.yourdomain.com   # NOT the same domain as server_url

dns:
  magic_dns: true
  override_local_dns: true
  base_domain: ts.yourdomain.com
  nameservers:
    global:
      - 1.1.1.1

# Custom DNS records for private services — visible only inside the tailnet
extra_records:
  - name: "proxmox.yourdomain.com"
    type: A
    value: "100.64.0.2"
  - name: "nas.yourdomain.com"
    type: A
    value: "100.64.0.3"
  - name: "akira.yourdomain.com"
    type: A
    value: "100.64.0.9"

Important: base_domain cannot be the parent domain of server_url. If your server is at headscale.yourdomain.com, use ts.yourdomain.com or vpn.yourdomain.com as the base domain. Headscale rejects configurations where server_url is a subdomain of base_domain.

Deploy:

docker compose up -d
curl https://headscale.yourdomain.com/health
# → {"status":"ok"}

Creating a User and Pre-Auth Keys

Everything in Headscale is under a "user" (think: namespace). All my devices live under one user:

docker exec headscale headscale users create yourname
docker exec headscale headscale preauthkeys create --user yourname --reusable --expiration 90d
# → Save this key

The reusable pre-auth key means you can register new devices without running the approve command for each one. Convenient for adding VMs.

Connecting Devices

What follows depends strictly on what are your current devices and needs, but I provide here a list of useful examples that you can customize for your own implementation.

Linux Nodes (Proxmox, Ubuntu VMs, Debian VMs)

curl -fsSL https://tailscale.com/install.sh | sh
systemctl enable --now tailscaled
tailscale up --login-server https://headscale.yourdomain.com --authkey <your-preauthkey>

# Confirm registration
docker exec headscale headscale nodes list

Synology NAS

Synology's built-in Tailscale package connects to official Tailscale only. You need a manually installed SPK from pkgs.tailscale.com. Check your NAS architecture first:

# On the NAS via SSH:
uname -m   # armv7l, x86_64, etc.

Download the correct SPK for your Tailscale version, install via DSM Package Manager → Manual Install, then:

# SSH into NAS
tailscale up --login-server https://headscale.yourdomain.com --authkey <preauthkey>

Note: If your NAS reports HTTP 308 redirect errors, you're running a Tailscale version too old to handle Headscale's redirect behavior. Upgrade to at least v1.90+.

Unprivileged LXC Containers (Portainer, etc.)

LXC containers without a TUN device need userspace networking mode:

# Install Tailscale inside the container (standard install script works)
# Then configure userspace mode:
mkdir -p /etc/systemd/system/tailscaled.service.d
cat > /etc/systemd/system/tailscaled.service.d/override.conf << 'EOF'
[Service]
ExecStart=
ExecStart=/usr/sbin/tailscaled --tun=userspace-networking --socket=/run/tailscale/tailscaled.sock
EOF
systemctl daemon-reload
systemctl restart tailscaled
tailscale up --login-server https://headscale.yourdomain.com --authkey <preauthkey>

Connectivity works normally. The performance penalty of userspace mode is negligible for control traffic.

macOS

Download Tailscale standalone (not the App Store version — the CLI is required):

/Applications/Tailscale.app/Contents/MacOS/Tailscale login \
  --login-server https://headscale.yourdomain.com

A browser window opens. Copy the registration command that appears, run it in your Headscale container. Done.

Continue to Part 2

#vpn #headscale #tailscale #wireguard #homelab

Self-Hosted VPN with Headscale: One Mesh Network for Your Entire Homelab

(Part 1/2)

(Part 1/2)

Why Headscale Instead of Running a WireGuard Server?

VPS Setup

Deploying Headscale with Traefik

Creating a User and Pre-Auth Keys

Connecting Devices

Linux Nodes (Proxmox, Ubuntu VMs, Debian VMs)

Synology NAS

Unprivileged LXC Containers (Portainer, etc.)

macOS

Reporting a Problem