dmo.ca/ blog/ VServer and 127.0.0.1

I recently moved our automated testing systems from a KVM+Qemu setup to Linux-VServer. We didn't need the overhead of a full virtual environment, even one as lightweight as KVM. However, there was a little snag -- 127.0.0.1 didn't work correctly.

Previous versions of vserver had issues with 127.0.0.1 -- you could choose to have no loopback at all, or have the loopback address rewritten to the vserver's primary IP address. If you want to differentiate between connections to 127.0.0.1 and your actual IP, well, you couldn't.

However, this is somewhat ingeniously fixed in newer development versions of vserver. Under the hood, the kernel will create a 127.x.y.1 address for each vserver guest (with x.y based on the 16-bit context ID of that vserver), and remap 127.0.0.0/8 transparently, giving you a private 127.0.0.1 for each guest. Unfortunately, that didn't work out-of-the box. Instead, I got surprising old-style behaviour, with 127.0.0.1 remapped to the primary IP. Running a tcpdump on the host showed that yes, everything was rewritten to the primary. However, the kernel was creating a local lback address for me -- it just wasn't getting used.

After much fighting with under-documented options, to no avail, I was convinced that there had to be something wrong with Debian's backport of the vserver patch, and built a bleeding-edge kernel with this morning's vserver pre4 patch. Still no luck -- it wasn't Debian's fault.

Then, while trying to silence what I thought was a completely unrelated warning:

Warning: Executing wildcard deletion to stay compatible with old scripts.
     Explicitly specify the prefix length (192.168.10.60/32) to avoid this warning.
     This special behaviour is likely to disappear in further releases,
     fix your scripts!

I set the network prefix for my guest eth0 device with:

echo "24" > /etc/vservers/<yourname>/interfaces/0/prefix

This isn't set up by default by 'vserver build' for some reason, so I thought it unnecessary. Guess what, it's not. On the next vserver restart, the private 127.0.0.1 worked as specified, getting mapped to 127.x.y.1 and the packets zipping along over the host's lo interface.

I have no idea why this works, but perhaps someone on #vserver will be able to help me out with an explanation.