website/content/posts/abusing-systemd-nspawn-with-nested-containers.md
Anthony Wang 224f2b7132
All checks were successful
ci/woodpecker/push/<no value> Pipeline was successful
Publish abusing systemd-nspawn with nested containers post
2023-07-07 14:55:42 +00:00

3.6 KiB

title date description type tags
Abusing systemd-nspawn With Nested Containers 2023-03-01T04:03:13Z Pushing systemd-nspawn and my laptop to their limits with indefinitely nested containers post
linux
containers
systemd-nspawn
fun

systemd-nspawn is some pretty insane stuff that you probably already have installed on your computer. Today's mission to wreck havoc will be to replicate an old project I did a few years ago, Arch All the Way Down, but using some real power tools. I was able to achieve 4 nested containers with Docker and my old laptop. It shouldn't be too hard to improve that!

There's a really cool trick you can do with systemd-nspawn where you can almost instantly create a container based on your existing root directory:

$ sudo systemd-nspawn --directory=/ --volatile=yes --set-credential=passwd.plaintext-password.root:"" --set-credential=firstboot.locale:C.UTF-8 -b

This is all nice and great, and I was easily able to create containers nested 10 layers down, but it's annoying to have to log in to each one and run the command again. Let's automate this.

The first thing we can do is auto login. Cool, now our cursed setup looks like this:

$ cat autologin.conf
[Service]
ExecStart=
ExecStart=-/sbin/agetty -o '-p -f -- \\u' --noclear --keep-baud --autologin root - 115200,38400,9600 $TERM
$ sudo systemd-nspawn --directory=/ --volatile=yes --set-credential=passwd.plaintext-password.root:"" --set-credential=firstboot.locale:C.UTF-8 -b --bind=autologin.conf:/etc/systemd/system/console-getty.service.d/autologin.conf

Now for some more fun: we will make the container auto-run this command by adding the thing above to its /root/.bash_profile. Don't try this at home.

$ cat bash_profile
systemd-nspawn --directory=/ --volatile=yes --set-credential=passwd.plaintext-password.root:"" --set-credential=firstboot.locale:C.UTF-8 -b --bind=/etc/systemd/system/console-getty.service.d/autologin.conf:/etc/systemd/system/console-getty.service.d/autologin.conf --bind=/root/.bash_profile:/root/.bash_profile
$ sudo systemd-nspawn --directory=/ --volatile=yes --set-credential=passwd.plaintext-password.root:"" --set-credential=firstboot.locale:C.UTF-8 -b --bind=autologin.conf:/etc/systemd/system/console-getty.service.d/autologin.conf  --bind=bash_profile:/root/.bash_profile

You might be a bit worried at this point (for many reasons), but one concern is that this spawns nested containers infinitely. Fortunately, it takes around a second to spawn a container so we won't DOS ourselves instantly, and you can easily hit Ctrl+] 3 times to terminate all containers.

As for measuring the nesting depth, container processes are visible on the host, so ps aux | grep systemd-logind or something like that will do the trick.

Now let's have some real fun! My laptop has 16GB of RAM, so I wonder how many nested containers it can handle. A few hundred maybe?

Nope, only 33. I still have plenty of RAM left over, but I'm getting a Failed to fork inner child: No space left on device error. Aw man.

Unfortunately, I spent way too much time trying to figure out this error, but it doesn't seem to be something you can search up online, for obvious reasons. This is the line in the nspawn source code that raises the error, but it's not really helpful. If anyone has any ideas, I'd love to hear it!