Offshore Outsourcing will help you to reduce cost and enhance your productivity

Home About Us Services Partners Articles Classifieds Directory Contact Us
   
Offshoring
Outsourcing
BPO
Computers Networks
Internet
Operating Systems
Data Storage
Telecommunications
Programming
Software Engineering
Information Technology
Online Rights - Law
Business
E-Commerce
IT Outsourcing
Business Consulting
Finance & Accounting
Graphic Design
Web Services
Search Engine Optimization
Open Source
Hardware
Security
Others
Games

Making OpenBSD Binary Patches With Chroot

by Lawrence Teo March 26, 2007
This article is brought to you by Calyptix Security, maker of the OpenBSD-powered AccessEnforcer all-in-one Internet security appliance.

Unlike other operating systems, patches for the OpenBSD base system are distributed as source code patches. These patches are usually applied by compiling and installing them onto the target system. While that upgrade procedure is well-documented, it is not always suitable for certain systems that do not have the OpenBSD compiler set installed for various reasons such as disk space constraints. To fill this gap, open source projects like binpatch were started to allow administrators to create binary patches using the BSD make system. This article proposes an alternative method to build binary patches using a chroot environment in an attempt to more closely mirror the instructions given in the OpenBSD patch files.

New OpenBSD administrators often find that OpenBSD is different from other operating systems in terms of its upgrade process. Linux systems, for example, usually distribute all components, including the kernel and all userspace software, as packages; to upgrade the packages, the administrator would use the distribution-specific upgrade tool to upgrade those packages (like apt-get for Debian systems and upgradepkg for Slackware).

OpenBSD uses a similar tool called pkg_add(1) to upgrade third-party software (called "ports" in OpenBSD nomenclature). However, the OpenBSD base system has a different upgrade process. The base system here refers to software that are part of stock OpenBSD, such as the tools in /bin and /usr/bin which are not third-party software. Patches to the base system are distributed as source code patches on http://www.openbsd.org/errata.html, which have to be hand-compiled and installed on the target system.

This means that the OpenBSD compiler distribution set (compXX.tgz) has to be installed on the target system in order to build and apply those patches. However, this is not always desirable for a few reasons:

  • the compiler set tends to be very large (it is a 75MB compressed tarball on OpenBSD 4.0/i386), so it may not fit on a target system with disk space constraints;
  • the target system may have very low memory that prohibits compilation;
  • the target system may have a very slow CPU that would take a very long time to compile large programs;
  • the target system may be a virtual machine with the above limitations.

In addition, an OpenBSD administrator might maintain a large number of such systems, making it inconvenient to compile and install patches by hand on each individual system.

Fortunately, excellent projects like binpatch (http://openbsdbinpatch.sourceforge.net/) exist to fill this void. Binpatch uses the BSD make system to build binary patches. It employs a custom Makefile in order to create binary patches based on the instructions given in the OpenBSD patch files from http://www.openbsd.org/errata.html.

A binary patch is simply a tarball (a .tar.gz or .tgz file) containing the pre-compiled files that are the result of applying the source code patch. To illustrate, suppose OpenBSD distributes a source code patch for the Apache webserver, which is part of the base system. The corresponding binary patch would be a tarball consisting of the new Apache programs, modules, manpages, and other files that are created as a result of patching and re-compiling Apache. The binary patch is created on a system with the full compiler set, and distributed to the target system where the compiler set is not available.

In this article, I will describe a technique that uses a chroot environment to create binary patches. This technique is in no way more superior compared to binpatch's BSD make system technique; it is merely an alternative method that attempts to more closely mirror the original instructions in the OpenBSD patch files by using a chroot environment. Please note that this proposed method is highly experimental and we do not recommend it for production use -- unless you really know what you are doing! I also assume you have some moderate experience and skills with OpenBSD administration.

Let's Begin!

You will first need access to the OpenBSD distribution files like baseXX.tgz, etcXX.tgz, and compXX.tgz, as well as the source code tarballs (src.tar.gz and sys.tar.gz). An easy way to have convenient access to these files is to order an OpenBSD CD from the OpenBSD store. To make this article easier to follow, let's suppose we are attempting to build a binary patch in OpenBSD 4.0 on the i386 platform. This process involves two separate machines: we will build the binary patch on a machine which we will call a build system, and we will deploy the binary patch on a target system.

First, we need some space to emulate a full OpenBSD system that we can use as a chroot environment. Let's create the following hierarchy in a new directory called /var/chbinpatch:

mkdir -p /var/chbinpatch/work/usr/src  

Then, extract the OpenBSD distribution files ({base,etc,comp,man,misc}40.tgz) to /var/chbinpatch/work:

for i in base etc comp man misc; do tar zxpf /path/to/${i}40.tgz -C /var/chbinpatch/work; done  

Note that the above command is meant for OpenBSD 4.0; for another version, such as OpenBSD 4.1, replace "40" with "41".

Next, extract the source code tarballs src.tar.gz and sys.tar.gz to /var/chbinpatch/work/usr/src:

tar zxpf /path/to/src.tar.gz -C /var/chbinpatch/work/usr/src  tar zxpf /path/to/sys.tar.gz -C /var/chbinpatch/work/usr/src  

We now have an almost-complete OpenBSD "system" in /var/chbinpatch/work. However, one thing is missing -- to be able to compile in a chroot environment, gcc will need the necessary devices in /dev. However, directories like /var and /usr are mounted with the nodev option by default, thus it is not possible to use devices in /var/chbinpatch/work/dev. To work around this, we backup the existing /dev that we just extracted and create a temporary memory filesystem to act as /dev in the chroot environment.

mv /var/chbinpatch/work/dev /var/chbinpatch/work/dev-orig  mkdir /var/chbinpatch/work/dev  mount_mfs -o nosuid -s 8192 /dev/wd0b /var/chbinpatch/work/dev  

If your swap device is not /dev/wd0b, change the mount_mfs(8) command accordingly.

gcc needs the standard devices to be present in /dev, so let's copy over the MAKEDEV script from the original /dev and execute it with the "std" option:

cd /var/chbinpatch/work/dev  cp -p ../dev-orig/MAKEDEV .  ./MAKEDEV std  

We now have a complete OpenBSD "system." Let's proceed by applying a patch from the OpenBSD errata page.

Building a Binary Patch

Suppose we would like to apply OpenBSD 4.0 errata #1 from http://www.openbsd.org/errata40.html. Download the patch file to /var/chbinpatch/work/usr/src:

cd /var/chbinpatch/work/usr/src  ftp ftp://ftp.openbsd.org/pub/OpenBSD/patches/4.0/common/001_httpd.patch  

Let's enter the chroot environment:

/usr/sbin/chroot /var/chbinpatch/work /bin/ksh  

For those unfamiliar with chroot(8), the above command will treat /var/chbinpatch/work as the root directory (/), and run /bin/ksh as the shell.

Now, to build a binary patch, we must figure out the files that will change after applying the patch, and isolate those files to be part of a tarball that can be distributed to a target system. This is where the chroot environment truly shines. In an actual non-chroot OpenBSD system, it would be difficult (though not impossible) to isolate those files because the system is actually running, and background processes might make changes to the filesystem. In contrast, inside a chroot environment, the system is totally "clean"; no processes are running that will make any change to the chrooted filesystem.

To actually be able to track file changes after applying a patch, we will need to create two "cookie" files, which are really just dummy marker files that are meant to facilitate the find(1) command to, well, find the files that have changed after the compile/install process. One cookie file is created before the build process (let's call it the "pre-installation cookie"), and the other after the build process (the "post-installation cookie"). The idea is to use find(1) to generate a list of files with timestamps later than the pre-installation cookie but earlier than the post-installation cookie. This will become clearer later in the article. Readers familiar with OpenBSD ports will notice that this cookie technique is borrowed from the make system in the OpenBSD ports tree.

Let's create the pre-installation cookie first:

touch /001_httpd.pre  

Remember, we are in a chroot environment, so / is actually /var/binpatch/work. I would not recommend creating these dummy files if this is the regular / directory on an actual system! Also, take note to name your cookie according to the patch file (in this case, 001_httpd) so that you will be able to keep track of which cookies are used for which patch.

Now, proceed to build the patch according to the patch instructions. In this case, we are following the instructions in the OpenBSD 4.0 001_httpd.patch file:

cd /usr/src  patch -p0 < 001_httpd.patch  cd usr.sbin/httpd  make -f Makefile.bsd-wrapper obj  make -f Makefile.bsd-wrapper cleandir  make -f Makefile.bsd-wrapper depend  make -f Makefile.bsd-wrapper  make -f Makefile.bsd-wrapper install  

After installation, create the post-installation cookie.

touch /001_httpd.post  

The pre-installation and post-installation cookies are very important -- if they are not present, you will not be able to generate the binary patch accurately. So, always remember to create them!

Now, use the find(1) command to generate a list of files in /usr, /sbin, and /bin that were created by the "make install" process:

export P="001_httpd"  find /usr /sbin /bin -path /usr/src -prune -or -path /usr/obj -prune -or \    -newer /${P}.pre -a ! -newer /${P}.post -type f | \    grep -v '/usr/src' | grep -v '/usr/obj' | sed 's/^/./' >/${P}.list  

Recall that the find(1) command is used to find files later than the pre-installation cookie but earlier than the post-installation cookie. Also, note that the above command intentionally excludes the /usr/src and /usr/obj directories, and pipes its output into a file called /001_httpd.list.

Now you can use that 001_httpd.list to create a tarball of the relevant files that are part of the patch:

cd /  cat 001_httpd.list | xargs tar cvzpf errata-4.0-001-i386.tar.gz  

Your binary patch, errata-4.0-001-i386.tar.gz, is now complete! Exit the chroot environment:

exit  

To apply the binary patch, you will first need to use scp(1) to transfer it to the target OpenBSD system:

scp -p /var/chbinpatch/work/errata-4.0-001-i386.tar.gz user@target:  

After the transfer, log in to the target system and extract it using tar(1):

ssh user@target  su -  cd /  tar zxvpf ~user/errata-4.0-001-i386.tar.gz  

Your binary patch is now applied on the target system. Remember to follow any further instructions from the patch file if needed. For example, for this patch file you would have to restart Apache using the apachectl(8) command.