FreeBSD Jails and Container Management Tools

FreeBSD [Link] and the BSD family of UNIX systems are known for their exceptional security, reliability, and performance (especially in networking).

For greater context, check out the other posts: Virtualization with Bhyve on FreeBSD [Link] and Introduction to UNIX for Linux Users [Link]

Jails and containers in FreeBSD are not virtual machines, even though they look and feel like one [Link]. A jail is nothing more than an isolated process confined to a limited environment using the chroot concept.

Compared to Linux Containers and Application Containers, Jails first appeared in the year 2000, while LXC and Docker followed in 2008 and 2013, respectively.

MANAGEMENT TOOLS

Jails can be managed with different tools and methods.

Native jails
- The native low-level tools that rely on manually editing jail.conf with a declarative format.
BastilleBSD [Link]
- A higher-level tool known for being the simplest and most widely recommended choice.
AppJail [Link]
- The closest equivalent to a modern container platform, with reusable application definitions.
Pot [Link]
- Feels remarkably like a Dockerfile when creating a container build definition called a Potfile.
iocage
- Worth mentioning but out of scope of this post, as it is maintained by the TrueNAS team, which no longer runs on FreeBSD. Development has stalled and most users have moved on.

FETCH OFFICIAL RELEASE TARBALLS

While some tools can automatically fetch and manage release base images, here is how to do it manually, as required for native jails.

Create a dedicated ZFS dataset for jails and mount it.

zfs create zroot/jails
zfs set mountpoint=/usr/jails zroot/jails
zfs create zroot/jails/releases
zfs create zroot/jails/releases/15.1-RELEASE
zfs create zroot/jails/containers

Then fetch the base image.

mkdir -p /usr/jails/releases/15.1-RELEASE
cd /usr/jails/releases/15.1-RELEASE
fetch https://download.freebsd.org/releases/amd64/15.1-RELEASE/base.txz
tar -xpf base.txz && rm base.txz
echo "nameserver 1.1.1.1" > /usr/jails/releases/15.1-RELEASE/etc/resolv.conf

Create a snapshot and clone it as many times as needed (never use the template directly).

zfs snapshot zroot/jails/releases/15.1-RELEASE@base
zfs clone zroot/jails/releases/15.1-RELEASE@base zroot/jails/containers/proxy
zfs clone zroot/jails/releases/15.1-RELEASE@base zroot/jails/containers/app
zfs clone zroot/jails/releases/15.1-RELEASE@base zroot/jails/containers/db

NATIVE JAILS

Unlike containers on Linux, Jails are part of the FreeBSD system and require no additional packages. They just need to be enabled at boot.

Check if it is enabled.

sysrc jail_enable

If not, enable it.

sysrc jail_enable=YES

Service basic operations.

service jail start
service jail restart
service jail stop
service jail status

Configure the containers.

nano /etc/jail.conf

Basic example.

path = "/usr/jails/containers/$name";
mount.devfs;

exec.start = "/bin/sh /etc/rc";
exec.stop  = "/bin/sh /etc/rc.shutdown";

proxy {
    host.hostname = proxy.local;
    interface = "vtnet0";
    ip4.addr = 10.1.1.10/24;
}

app {
    host.hostname = app.local;
    interface = "vtnet0";
    ip4.addr = 10.1.1.11/24;
}

db {
    host.hostname = db.local;
    interface = "vtnet0";
    ip4.addr = 10.1.1.12/24;
}

Note: the root file system must be present at the indicated path.

Managing workloads.

service jail start
service jail status proxy
service jail stop db

Basic management commands.

jls
- List configured containers.
jexec proxy ls
- Execute a command inside a jail, or get a shell on it.
freebsd-update -b /usr/jails/releases/15.1-RELEASE fetch install
- Updates the template, so all new clones benefit from it.
- Can also be applied to a stopped container, or to a running one that is then restarted to load the new files.

BASTILLE

Install and configure Bastille.

pkg install -y bastille
zfs create zroot/bastille
nano /usr/local/etc/bastille/bastille.conf

bastille_prefix="/usr/local/bastille"
bastille_zfs_enable="YES"
bastille_zfs_zpool="zroot"

Pull the base image of a specific release.

bastille bootstrap 15.1-RELEASE
bastille list releases

Deploy a jail from a defined release (or template).

bastille create web 15.1-RELEASE 10.1.1.20
bastille list

For reference, bastille stores jail configurations at /usr/local/bastille/jails/<JAIL_NAME>/, where:

root/
- The base file system of the jail.
jail.conf
- The native jail configuration generated by Bastille.
fstab
- Mounting and mapping configurations.
bastille.conf
- Bastille-specific configuration (e.g., ZFS, networking).

Templates (releases) are stored in /usr/local/bastille/releases/.

Basic commands.

bastille bootstrap
- Pull images.
bastille create
- Deploy a jail.
bastille list
- List jails.
bastille start
- Start a jail.
bastille stop
- Stop a jail.
bastille restart
- Restart a jail.
bastille console
- Get a shell in a jail.
bastille cmd
- Execute a command inside a jail.
bastille pkg
- Manage packages in a jail.
bastille cp
- Copy files between host and guest.
bastille destroy
- Delete a jail.

APPJAIL

pkg install -y appjail
sysrc appjail_enable=YES
nano /usr/local/etc/appjail/appjail.conf

Configure.

EXT_IF=vtnet0
ON_IF=vtnet0
FREEBSD_VERSION=15.1-RELEASE
FREEBSD_ARCH=amd64
IMAGE_ARCH=amd64
SHORTEN_DOMAIN_NAMES=1

Fetch the FreeBSD release.

appjail fetch
appjail fetch list

Deploy a jail.

appjail quick web
appjail jail list

Manage.

appjail start web
appjail login web
appjail cmd web ls
appjail stop web
appjail jail destroy -f web

Create a Makejail file (the closest equivalent to a Dockerfile).

PKG nginx
SYSRC nginx_enable=YES
SERVICE nginx start

Build the jail.

appjail makejail -f Makejail -j nginx

There is also Director, which works with AppJail to provide something closer to Docker Compose, and Overlord, which is the closest equivalent to an orchestrator like Kubernetes or Docker Swarm.

POT

Install Pot.

pkg install -y pot
pot version

Make sure Jails is enabled.

zfs list
sysrc jail_enable=YES

Set the interface Pot shall use with jails.

ifconfig
nano /usr/local/etc/pot/pot.conf

POT_EXTIF=vtnet0

Initialize Pot.

pot init
pot show

Fetch the base image of your choice.

pot fetch -r 15.1
pot list -a

Deploy a jail from the fetched base image.

pot create -p web -t single -b 15.1 -N alias -i 10.7.1.20
pot info -p web

Start and inspect the new jail.

pot start -p web
pot show -r
pot list -v
pot top -p web

Turn it into a template.

pot stop -p web
pot snapshot -p web
zfs list -t snapshot

Create a clone from the template (parent pot).

pot clone -p web-clone -P web -i 10.7.1.21
pot start -p web-clone
pot term -p web-clone

Create a Flavors (Potfiles) file (the closest equivalent to a Dockerfile).

pkg nginx
sysrc nginx_enable=YES
copy nginx.conf /usr/local/etc/nginx/nginx.conf

Potluck [Link] is a repository of Flavors, similar to Docker Hub. See Nomad [Link] for container orchestration.

BASIC COMPARISON

The following table lists some of the most basic features expected in container environments.

Feature	Bastille	AppJail	Pot
ZFS	Supported	Supported	Required
Automation	Templates	Makejail, Initscripts, Images	Flavours, Images
Jail types	clone, thin, thick, vnet, bridged vnet, Linux, empty	clone, copy, tiny, thin, thick, empty, linux+debootstrap	thick
VNET	Supported	Supported	Supported
Resource control	Supported	Supported	Limited to CPU and memory
IPv6 support	Yes	Yes	Yes
Linux containers	Yes	Yes
OCI support		Yes
Network management		Virtual networks	Subnet, requires sysutils/potnet
Healthcheckers		Yes
Log management	Yes	Yes
Parse jail.conf for syntax errors		Yes	Yes
Volume management	Basic	Yes	Basic, only with the fscomp feature
Parallel startup	Yes	Yes
Bridges	Yes	Yes	Yes
VLAN	Yes	Yes
Startup order control	Yes	Yes	Yes
X11 support		Yes

REFLECTIONS

I have only scratched the surface here, particularly with the more modern AppJail and Pot, which bring contemporary concepts to container management.

The way I see FreeBSD Jails, they are closer to Linux Containers (LXC/Incus) in the sense that they are full systems (pets), not ephemeral workloads (cattle) as treated by Kubernetes and Docker.

That does not mean Jails cannot work as application-specific containers, but that was not their original intent. I am curious to keep exploring what is possible in that direction.