Two hours Project

Sharing one NAS with many containers

Sooner or later several services need the same files: the media server, the photo library, the backup job. The clean answer is to keep the data on the NAS, mount it once on the container host and hand it to each container. The details that trip people up are permissions, user IDs and what happens when the NAS is not there.

  • IntermediateDifficulty
  • 2 hoursTime
  • NAS + hostExisting hardware
  • $0Typical cost

The pattern

Each kind of data lives in its own dataset or share on the NAS, where snapshots, scrubs and replication protect it. The container host mounts those shares once, at fixed paths. Each container then receives only the paths it needs through bind mounts. Nothing is copied, every service sees the same files, and rebuilding a container never touches the data.

For services, NFS is usually the better protocol between Linux machines: simple, fast and well suited to many small reads. SMB is the right choice for desktops and laptops. Many NAS setups export the same dataset both ways, to different clients.

Step 1: prepare the NAS

  1. Create a dataset per purpose, such as media, photos and app-data. Each gets its own snapshot schedule and quota. See ZFS and Filesystems.
  2. Create a service account on the NAS with a fixed numeric user and group ID, for example 2000, and make it the owner of those datasets. Services will write as this ID.
  3. Export over NFS to specific hosts only. Allow the container host's storage-network address, never a whole household subnet. In our example network the NAS has 10.42.70.10 on the non-routed storage VLAN and the container host has 10.42.70.30.
  4. Export read-only where you can. A media server only needs to read the library; only the job that adds files needs write access.

Step 2: mount it on the container host

First, give the container host a second network interface on the storage VLAN, with an address such as 10.42.70.30 and no gateway. On a hypervisor, that means adding a second virtual network card on VLAN 70 to the container host VM. Storage traffic then stays on the switch and never passes through the firewall. See VLANs and Segmentation.

Next, install the NFS client package, create the mount points and add entries that mount on demand. The automount options below make the system mount the share when it is first used rather than at the exact moment of boot, which avoids failures when the NAS starts more slowly than the host:

10.42.70.10:/tank/media    /mnt/media    nfs  defaults,_netdev,noatime,x-systemd.automount  0  0
10.42.70.10:/tank/photos   /mnt/photos   nfs  defaults,_netdev,noatime,x-systemd.automount  0  0

Reload the service manager's configuration, then access each path once and confirm the files appear. Reboot the host and check again; mounts that only work until the next reboot are a classic trap.

Step 3: hand it to containers

Give every container the same internal path for the same data. If the media server, the library manager and the import job all see the library at /media, files can be moved between folders instantly and links work, because everything is on one filesystem. In a compose file it looks like this:

services:
  media:
    image: example/media-server:12.1
    user: "2000:2000"
    volumes:
      - /srv/media-server/config:/config
      - /mnt/media:/media:ro

Configuration stays on the container host under /srv, where it is small and backed up nightly. The library stays on the NAS, mounted read-only. The image name is a placeholder; use the official image of the server you choose.

System containers and user ID mapping

If you run services as system containers on a hypervisor platform rather than application containers in a VM, two details matter:

  • Unprivileged system containers cannot mount NFS themselves. Mount the share on the hypervisor host, then pass the directory into the container as a bind mount point.
  • User IDs are shifted. Inside an unprivileged container, user 0 corresponds to a high number on the host, commonly 100000, and every other ID shifts by the same amount. A service running as 2000 inside the container appears as 102000 to the host and the NAS. Either make the NAS permissions match the shifted ID or configure an explicit ID mapping for that container.

Caution

Do not fix permission errors by making shares writable by everyone or by running containers as root. It works, and it means any compromised container can rewrite every file on the share. Fix the ownership and IDs instead.

Failure modes to plan for

What happensWhyPrevention
A service writes gigabytes to the host's own diskThe share was not mounted, so the empty mount point was written to directlyAutomount options, and an uptime check that verifies a known file exists on each mount
Containers hang after the NAS rebootsStale NFS handlesRestart affected containers after NAS maintenance; schedule NAS updates when the household is asleep
Permission denied on new filesContainer user ID does not match the NAS owner, or ID mapping shifted itFixed service IDs, documented in your service catalog
Everything stops when the NAS is downOne storage box serves every serviceAccept it consciously, keep the NAS simple and dedicated, and back it up

Backups

Shared storage concentrates data, which makes it easy to protect and costly to lose. Snapshot the NAS datasets on schedules that match how often they change, back up the container host's /srv directory nightly, and keep the irreplaceable datasets in your off-site copy. See Backup Strategy.

Verify

  • Reboot the container host and confirm every mount and service returns without intervention.
  • Reboot the NAS and confirm services recover, or note which ones need a restart.
  • Delete a container and recreate it from its compose file; its data should be untouched.
  • Create a file from inside each container and check its ownership on the NAS.

This pattern underpins the Home Media Stack project and both larger reference builds.