|
|
The UML test suite
The UML test suite is a set of Perl scripts which I use as a
regression test on new UML patches. It comprises a test harness,
which organizes and runs the suite, and the suite itself, which is a
directory hierarchy which groups similar tests.
The tests to be done during a particular test run are specified by the
parent directory, if you want all of the tests in that directory to
run, an individual test, or any combination of directories and files.
The tests are able to specify the properties of the UML in which they
are to run. The test harness compiles and runs UMLs according to the
test requirements, trying to minimize the number of builds and boots
required to satisfy all of the tests' requirements.
The tests vary widely in what they test and how they work. The
simplest ones run a command (which they may have compiled themselves)
and check the return value. Others run gdb and make it step through a
test program to check that breakpoints work correctly. These use an
already-booted UML, and don't need any special setup of the UML.
In contrast, there are tests which have requirements of the UML. For
example, the boot_filesystems test checks that a user-specified set of
filesystems boots correctly. So, it requests an unbooted UML for its
use, but puts no more requirements on it. Others test various
configurable options, and request a UML with certain options enabled,
which they boot.
Finally, there are tests which stress the kernel generally in order to
make sure that it is stable under a hard workload. The kbuild test,
which runs a kernel build, is an example of this.
The main reason would be to do a sanity-check on UML when compiled
locally, with whatever unusual config options you favor, and run on
your hardware. UML is somewhat sensitive to toolchain changes, and,
to a less extent, to the host hardware. Users regularly shake out
bugs that don't appear during development, and these are often exposed
by different toolchains and hardware.
Secondly, if UML fails mysteriously somehow, a test may exercise the
same bug. In this case, the test is likely to be much more specific
than your workload, and the search for the bug much more constrained.
It would also be a help to the UML project in general to have the
tests run as often as possible, and on as a great a variety of hosts
as possible. This will expose bugs, and get them fixed, more quickly.
First, download the latest version from the test suite section of
the UML download page.
host%
wget http://www.user-mode-linux.org/mirror/uml_tests_20040726.tar.bz2
Uncompress and extract it
host% bunzip2 uml_tests_20040726.tar.bz2
host% tar xf uml_tests_20040726.tar
Set up a ~/.umltest, which is the config file needed to tell the
harness about the external resources, such as filesystems, and other
information it will need in order to run. The file contains a perl
hash, with different types of information under different keys. If
you don't know perl, don't panic, because the syntax is simple
enough. Mine looks like this:
{
kernel_pool => "/home/jdike/linux/2.4/um",
uml => {
ubd => { 0 => "/home/jdike/roots/debian_22" },
mem => "64M",
cow_ubds => 1,
prompts => { "/home/jdike/roots/debian_22" => "usermode:.*#" },
},
tests => { "filesystems/boot_filesystems" =>
{ filesystems => [ "/home/jdike/roots/debian_22" ],
time => 0
},
"stress/kbuild" =>
{ kernel_pool => "/home/jdike/roots/kernel",
time => 0 },
},
}
The kernel_pool tells the harness the location of a UML source pool.
This will be where it builds whatever UMLs it needs.
The uml section specifies the default UML command. This will be used
for any tests which don't have special requirements. The ubd hash
within that is a mapping of device numbers to files, and this one will
turn into "ubd0=/home/jdike/roots/debian_22" on the command line. The
prompts hash specifies a regular expression which matches the shell
prompts for the filesystems that will be booted.
The tests section provides test-specific information. The keys here
are the filenames of the tests under the tests/ directory, and without
the ".pl" extension. They can ask for arbitrary information here. I
specify one filesystem for the boot_filesystems test to boot, and an
image of a filesystem containing a kernel source tree for the kernel
build test.
Now, the tests can be run as follows
host% perl test.pl
This will run the full test suite, reporting an overall status when
it's done. A full log of the run will appear in tests.log. This will
contain all output from the UMLs, including boot logs and anything the
tests do.
Specific tests can be run by adding them to the command line
perl test.pl stress/kbuild
Adding a directory to the command line will cause all of the tests
within the that directory to be run. So,
perl test.pl functional
will run all the tests under functional/, but nothing else.
The tests are all under the "tests" directory, each in a separate perl
file. Here is a very simple test, commented:
# Virtually all tests will need these two modules. The first defines
# the Test object, an instance of which is returned from here. The
# second is a library of useful utilities which essentially all of the
# tests use.
use UML::Test;
use UML::Testlib;
# strict is good
use strict;
# This function is the actual test. It will be run inside a booted
# UML, at the shell. run_exit_0 is from TestLib. It runs the dd
# command within the UML object given to the test, returning if the
# command had an exit status of 0, and dying if not. In this case,
# the harness will trap that and report the failure.
sub run {
my $test = shift;
my $uml = shift;
run_exit_0($uml, "dd if=/dev/mem of=/dev/null");
}
# This constructs the Test object and returns it. This specifies that
# this test be run in a booted UML (state => "up") and that the test
# is contained in the 'run' function.
UML::Test->new(state => "up", run => \&run);
This illustrates the two essential features of a test
-
The function which executes the actual test
-
The construction of the test object, which specifies the test function
and any requirements that the test makes on the UML.
The test can also request an unbooted UML with 'state => "up"', if the
purpose of the test is to check that UML boots. It can also request a
UML with special configuration options. The initrd test does this
like so:
UML::Test->new(state => "up",
run => \&run,
kconfig => { MMAPPER => "y" },
config => { "uml/iomem" => "mmapper,$file" } );
The kconfig option requests that it be given a UML with CONFIG_MMAPPER
set to "y", and the config option adds 'iomem=mmapper,$file' to the
UML command line.
|