U4 Compute

Containers: packaging services you can rebuild in minutes

Containers make services small, fast and disposable. The services are disposable; the data is not. Get that distinction right and containers become the easiest part of your lab to maintain.

At a glance

  • Decide per application. Default to a container when the application supports it; use a VM when it needs its own kernel, strong isolation or a different operating system.
  • Keep configuration in compose files under version control and data in known locations. Back up the data and the compose files, not the containers.
  • Prefer official images or images you build yourself. Every third-party image maintainer becomes part of your trust chain.
  • Pin versions and update deliberately, one stack at a time.
  • Containers are a packaging format, not a security boundary.

What a container is

A container is a group of processes isolated by features of the host's Linux kernel. Namespaces give it its own view of processes, network interfaces and filesystems; control groups limit its CPU and memory. Unlike a VM, a container has no kernel of its own. It starts in seconds and uses only the memory its processes need, which is why a small host that struggles with two VMs can run dozens of containers.

There are two broad kinds:

  • Application containers run one service from a layered, read-only image plus writable volumes. They are defined in files and replaced rather than repaired.
  • System containers behave like lightweight VMs, with an init system, user logins and updates inside. They are stateful and must be backed up like VMs.

Container or virtual machine?

  1. Let the hardware decide first. On a host with 4 to 8 GB of memory, containers are often the only practical choice.
  2. Check that the application supports containers. Some applications assume fixed paths, a full init system or kernel modules. Forcing them into a container produces workarounds that break on every update.
  3. Use a VM for other operating systems, firewalls and network appliances, anything that loads kernel modules, and services that need strong isolation from their neighbors.
  4. Otherwise, default to a container for its small footprint and easy rebuilds.

Most labs end up hybrid, and a very common pattern is containers running inside a dedicated VM. The VM gets backed up and snapshotted as a unit, and the container host can be rebuilt without touching the hypervisor.

A sensible container host

  • Dedicate a VM to running containers, sized for what you run, and back it up nightly.
  • Install the container engine from its official package repository, not from a universal package format, which can lag behind and confine the engine in surprising ways.
  • Keep one directory per application, for example /srv/notes, holding the compose file, an environment file and the data directories.
  • Keep compose files in a git repository, with secrets in environment files that are excluded from the repository.

A compose file describes a stack in one readable place. This example runs a notes application with its own database, publishes the web port only on localhost for a reverse proxy, and keeps all data under one directory:

services:
  app:
    image: example/notes-app:2.4.1
    restart: unless-stopped
    env_file: .env
    depends_on:
      - db
    volumes:
      - /srv/notes/data:/data
    ports:
      - "127.0.0.1:8080:8080"
  db:
    image: example/sql-database:16
    restart: unless-stopped
    env_file: .env
    volumes:
      - /srv/notes/db:/var/lib/database

The image names are placeholders. Use the official images and paths documented by the application you run.

Where your data lives

People lose data in containers for one reason more than any other: they never knew where the data was stored. If an application writes inside the container instead of a volume, the data disappears the next time the container is recreated.

  • Use bind mounts to known paths, such as /srv/app/data, or named volumes whose location you have checked. Bind mounts make backups obvious.
  • Consider NFS-backed storage on a NAS for important application data, so the storage layer's snapshots and replication protect it. Shared Storage for Containers covers mounts, permissions and the traps.
  • Give each application its own database container and volume, rather than one shared database server for everything. Backups, restores and upgrades become independent.
  • Dump databases to files on a schedule and back up the dumps, rather than relying on copies of live database files.
  • Test a rebuild. Delete a stack, recreate it from the compose file and restored data, and confirm it comes back. Do this before you need to.

Trusting images

Public registries host a very large number of images, and some are abandoned, outdated or deliberately malicious. Projects that were popular and then abandoned are attractive targets for takeover. Pulling an image means running someone else's software with access to whatever you mount into it.

  • Prefer images published by the project itself.
  • Where none exists, choose maintainers with a public build process and regular rebuilds, or build the image yourself from a published recipe.
  • Read the recipe. It shows what the image downloads and runs during the build.
  • Be wary of tutorials that pull an image from an unknown account or pipe a script from the internet straight into a shell.
  • Remove stacks you no longer use, so their images stop accumulating vulnerabilities.

Updating without surprises

Containers make updates easy, which is exactly why they can go wrong quickly. A habit that works:

  1. Pin a version, at least the major version, instead of a moving latest tag.
  2. Read the release notes for breaking changes and required migration steps.
  3. Snapshot the container host VM or back up the application's data directory.
  4. Update one stack, then check that it starts and works.
  5. Run any database migration command the release notes call for. A service that fails after an update often only needs a one-time upgrade step.

Tools that update containers automatically are convenient for low-stakes services. For anything the household relies on, use them in notify-only mode and update deliberately.

Networking pitfalls

  • Published ports work like port forwarding. Several containers can listen on port 80 internally while being published on different host ports.
  • Published ports can bypass the host firewall. On common container engines, publishing a port inserts rules ahead of the host's own firewall rules, so a host firewall may not block it. Bind published ports to localhost or a specific address, as in the example above.
  • Use a reverse proxy to give each web application a proper name and certificate. See DNS, Domains and Certificates.
  • Giving containers their own addresses on the LAN is possible but confusing, and the host often cannot reach those containers directly. Use it only when an application truly needs it.

Security basics

Containers share the host kernel. A kernel vulnerability or a badly configured container can reach the host. Treat containers as a convenient way to package and run software, and keep real isolation at the VM and network level.

  • Do not run containers in privileged mode unless the application genuinely requires it.
  • Do not mount the container engine's control socket into a container unless you understand that this grants it control of the host.
  • Consider rootless container engines, which reduce what a compromised container can do.
  • Place internet-facing stacks on a separate container host or VM from private ones.

Management interfaces and orchestration

A web interface for your container host makes volumes, networks and stacks visible, which is excellent for learning. Keep your compose files in git anyway, so the definitions survive if the interface does not.

Container orchestrators that schedule workloads across many machines are important in industry and well worth learning for a career. They need at least three control nodes for resilience and add considerable complexity. For the services a household depends on, a single well-backed-up container host is simpler, more reliable and easier to recover. Learn orchestration in a separate cluster, perhaps on a few single-board computers or small VMs, where breaking it costs nothing.