Although I am a Debian Developer (not very active, BTW) I am using Ubuntu LTS (right now version 24.04.1) on my main machine; it is my work laptop and I was told to keep using Ubuntu on it when it was assigned to me, although I don’t believe it is really necessary or justified (I don’t need support, I don’t provide support to others and I usually test my shell scripts on multiple systems if needed anyway).

Initially I kept using Debian Sid on my personal laptop, but I gave it to my oldest son as the one he was using (an old Dell XPS 13) was stolen from him a year ago.

I am still using Debian stable on my servers (one at home that also runs LXC containers and another one on an OVH VPS), but I don’t have a Debian Sid machine anymore and while I could reinstall my work machine, I’ve decided I’m going to try to use a system container to run Debian Sid on it.

As I want to use a container instead of a VM I’ve narrowed my options to lxc or systemd-nspawn (I have docker and podman installed, but I don’t believe they are good options for running system containers).

As I will want to take snapshots of the container filesystem I’ve decided to try incus instead of systemd-nspawn (I already have experience with it and while it works well it has less features than incus).

Installing incus

As this is a personal system where I want to try things, instead of using the packages included with Ubuntu I’ve decided to install the ones from the zabbly incus stable repository.

To do it I’ve executed the following as root:

# Get the zabbly repository GPG key
curl -fsSL https://pkgs.zabbly.com/key.asc -o /etc/apt/keyrings/zabbly.asc
# Create the zabbly-incus-stable.sources file
sh -c 'cat <<EOF > /etc/apt/sources.list.d/zabbly-incus-stable.sources
Enabled: yes
Types: deb
URIs: https://pkgs.zabbly.com/incus/stable
Suites: $(. /etc/os-release && echo ${VERSION_CODENAME})
Components: main
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/zabbly.asc
EOF'

Initially I only plan to use the command line tools, so I’ve installed the incus and the incus-extra packages, but once things work I’ll probably install the incus-ui-canonical package too, at least for testing it:

apt update
apt install incus incus-extra

Adding my personal user to the incus-admin group

To be able to run incus commands as my personal user I’ve added it to the incus-admin group:

sudo adduser "$(id -un)" incus-admin

And I’ve logged out and in again of my desktop session to make the changes effective.

Initializing the incus environment

To configure the incus environment I’ve executed the incus admin init command and accepted the defaults for all the questions, as they are good enough for my current use case.

Creating a Debian container

To create a Debian container I’ve used the default debian/trixie image:

incus launch images:debian/trixie debian

This command downloads the image and creates a container named debian using the default profile.

The exec command can be used to run a root login shell inside the container:

incus exec debian -- su -l

Instead of exec we can use the shell alias:

incus shell debian

which does the same as the previous command.

Inside that shell we can try to update the machine to sid changing the /etc/apt/sources.list file and using apt:

root@debian:~# echo "deb http://deb.debian.org/debian sid main contrib non-free" \
  >/etc/apt/sources.list
root@debian:~# apt update
root@debian:~# apt dist-upgrade

As my machine has docker installed the apt update command fails because the network does not work, to fix it I’ve executed the commands of the following section and re-run the apt update and apt dist-upgrade commands.

Making the incusbr0 bridge work with Docker

To avoid problems with docker networking we have to add rules for the incusbr0 bridge to the DOCKER-USER chain as follows:

sudo iptables -I DOCKER-USER -i incusbr0 -j ACCEPT
sudo iptables -I DOCKER-USER -o incusbr0 -m conntrack \
  --ctstate RELATED,ESTABLISHED -j ACCEPT

That makes things work now, but to make things persistent across reboots we need to add them each time the machine boots.

As suggested by the incus documentation I’ve installed the iptables-persistent package (my command also purges the ufw package, as I was not using it) and saved the current rules when installing:

sudo apt install iptables-persistent --purge

Integrating the DNS resolution of the container with the host

To make DNS resolution for the ictus containers work from the host I’ve followed the incus documentation.

To set up things manually I’ve run the following:

br="incusbr0";
br_ipv4="$(incus network get "$br" ipv4.address)";
br_domain="$(incus network get "$br" dns.domain)";
dns_address="${br_ipv4%/*}";
dns_domain="${br_domain:=incus}";
resolvectl dns "$br" "${dns_address}";
resolvectl domain "$br" "~${dns_domain}";
resolvectl dnssec "$br" off;
resolvectl dnsovertls "$br" off;

And to make the changes persistent across reboots I’ve created the following service file:

sh -c "cat <<EOF | sudo tee /etc/systemd/system/incus-dns-${br}.service
[Unit]
Description=Incus per-link DNS configuration for ${br}
BindsTo=sys-subsystem-net-devices-${br}.device
After=sys-subsystem-net-devices-${br}.device

[Service]
Type=oneshot
ExecStart=/usr/bin/resolvectl dns ${br} ${dns_address}
ExecStart=/usr/bin/resolvectl domain ${br} ~${dns_domain}
ExecStart=/usr/bin/resolvectl dnssec ${br} off
ExecStart=/usr/bin/resolvectl dnsovertls ${br} off
ExecStopPost=/usr/bin/resolvectl revert ${br}
RemainAfterExit=yes

[Install]
WantedBy=sys-subsystem-net-devices-${br}.device
EOF"

And enabled it:

sudo systemctl daemon-reload
sudo systemctl enable --now incus-dns-${br}.service

If all goes well the DNS resolution works from the host:

$ host debian.incus
debian.incus has address 10.149.225.121
debian.incus has IPv6 address fd42:1178:afd8:cc2c:216:3eff:fe2b:5cea

Using my host user and home dir inside the container

To use my host user and home directory inside the container I need to add the user and group to the container.

First I’ve added my user group with the same GID used on the host:

incus exec debian -- addgroup --gid "$(id --group)" --allow-bad-names \
  "$(id --group --name)"

Once I have the group I’ve added the user with the same UID and GID as on the host, without defining a password for it:

incus exec debian -- adduser --uid "$(id --user)" --gid "$(id --group)" \
  --comment "$(getent passwd "$(id --user -name)" | cut -d ':' -f 5)" \
  --no-create-home --disabled-password --allow-bad-names \
  "$(id --user --name)"

Once the user is created we can mount the home directory on the container (we add the shift option to make the container use the same UID and GID as we do on the host):

incus config device add debian home disk source=$HOME path=$HOME shift=true

We have the shell alias to log with the root account, now we can add another one to log into the container using the newly created user:

incus alias add ush "exec @ARGS@ -- su -l $(id --user --name)"

To log into the container as our user now we just need to run:

incus ush debian

To be able to use sudo inside the container we could add our user to the sudo group:

incus exec debian -- adduser "$(id --user --name)" "sudo"

But that requires a password and we don’t have one, so instead we are going to add a file to the /etc/sudoers.d directory to allow our user to run sudo without a password:

incus exec debian -- \
  sh -c "echo '$(id --user --name) ALL = NOPASSWD: ALL' /etc/sudoers.d/user"

Accessing the container using ssh

To use the container as a real machine and log into it as I do on remote machines I’ve installed the openssh-server and authorized my laptop public key to log into my laptop (as we are mounting the home directory from the host that allows us to log in without password from the local machine).

Also, to be able to run X11 applications from the container I’ve adusted the $HOME/.ssh/config file to always forward X11 (option ForwardX11 yes for Host debian.incus) and installed the xauth package.

After that I can log into the container running the command ssh debian.incus and start using it after installing other interesting tools like neovim, rsync, tmux, etc.

Taking snapshots of the container

As this is a system container we can take snapshots of it using the incus snapshot command; that can be specially useful to take snapshots before doing a dist-upgrade so we can rollback if something goes wrong.

To work with container snapshots we run use the incus snapshot command, i.e. to create a snapshot we use de create subcommand:

incus snapshot create debian

The snapshot sub commands include options to list the available snapshots, restore a snapshot, delete a snapshot, etc.

Conclusion

Since last week I have a terminal running a tmux session on the Debian Sid container with multiple zsh windows open (I’ve changed the prompt to be able to notice easily where I am) and it is working as expected.

My plan now is to add some packages and use the container for personal projects so I can work on a Debian Sid system without having to reinstall my work machine.

I’ll probably write more about it in the future, but for now, I’m happy with the results.