Running the Regression Script

There are a number of different tests that may be run, such as: the standard set that uses disk Volumes and runs under any userid; a small set of tests that write to tape; another set of tests where you must be root to run them. Normally, I run all my tests as non-root and very rarely run the root tests. The tests vary in length, and running the full tests including disk based testing, tape based testing, autochanger based testing, and multiple drive autochanger based testing can take 3 or 4 hours.

Setting the Configuration Parameters

There is nothing you need to change in the source directory.

To begin:

cd bacula/regress

The very first time you are going to run the regression scripts, you will need to create a custom config file for your system. We suggest that you start by:

cp prototype.conf config

Then you can edit the config file directly.

# Where to get the source to be tested

# Where to send email   !!!!! Change me !!!!!!!

# Full "default" path where to find sqlite (no quotes!)

# if you don't have an autochanger set AUTOCHANGER to /dev/null
# For two drive tests -- set to /dev/null if you do not have it 

# This must be the path to the autochanger including its name

# Set what backend to use "postresql" "mysql" or "sqlite3"

# Set your database here

# Set this to "--with-tcp-wrappers" or "--without-tcp-wrappers"

# Set this to "" to disable OpenSSL support, "--with-openssl=yes"
# to enable it, or provide the path to the OpenSSL installation,
# eg "--with-openssl=/usr/local"

# You may put your real host name here, but localhost or
# is valid also and it has the advantage that it works on a 
# non-networked machine

Building the Test Bacula

Once the above variables are set, you can build the setup by entering:

make setup

This will setup the regression testing and you should not need to do this again unless you want to change the database or other regression configuration parameters.

Setting up your SQL engine

If you are using SQLite or SQLite3, there is nothing more to do; you can simply run the tests as described in the next section.

If you are using MySQL or PostgreSQL, you will need to establish an account with your database engine for the user name regress and you will need to manually create a database named regress that can be used by user name regress, which means you will have to give the user regress sufficient permissions to use the database named regress. There is no password on the regress account.

You have probably already done this procedure for the user name and database named bacula. If not, the manual describes roughly how to do it, and the scripts in bacula/regress/build/src/cats named create_mysql_database, create_postgresql_database, grant_mysql_privileges, and grant_postgresql_privileges may be of a help to you.

Generally, to do the above, you will need to run under root to be able to create databases and modify permissions within MySQL and PostgreSQL.

It is possible to configure MySQL access for database accounts that require a password to be supplied. This can be done by creating a /.my.cnf file which supplies the credentials by default to the MySQL commandline utilities.

host     = localhost
user     = regress
password = asecret

A similar technique can be used PostgreSQL regression testing where the database is configured to require a password. The /.pgpass file should contain a line with the database connection properties.


Running the Disk Only Regression

The simplest way to copy the source code, configure it, compile it, link it, and run the tests is to use a helper script:


This will run the base set of tests using disk Volumes. If you are testing on a non-Linux machine several of the of the tests may not be run. In any case, as we add new tests, the number will vary. It will take about 1 hour and you don't need to be root to run these tests (I run under my regular userid). The result should be something similar to:

Test results
  ===== auto-label-test OK 12:31:33 =====
  ===== backup-bacula-test OK 12:32:32 =====
  ===== bextract-test OK 12:33:27 =====
  ===== bscan-test OK 12:34:47 =====
  ===== bsr-opt-test OK 12:35:46 =====
  ===== compressed-test OK 12:36:52 =====
  ===== compressed-encrypt-test OK 12:38:18 =====
  ===== concurrent-jobs-test OK 12:39:49 =====
  ===== data-encrypt-test OK 12:41:11 =====
  ===== encrypt-bug-test OK 12:42:00 =====
  ===== fifo-test OK 12:43:46 =====
  ===== backup-bacula-fifo OK 12:44:54 =====
  ===== differential-test OK 12:45:36 =====
  ===== four-concurrent-jobs-test OK 12:47:39 =====
  ===== four-jobs-test OK 12:49:22 =====
  ===== incremental-test OK 12:50:38 =====
  ===== query-test OK 12:51:37 =====
  ===== recycle-test OK 12:53:52 =====
  ===== restore2-by-file-test OK 12:54:53 =====
  ===== restore-by-file-test OK 12:55:40 =====
  ===== restore-disk-seek-test OK 12:56:29 =====
  ===== six-vol-test OK 12:57:44 =====
  ===== span-vol-test OK 12:58:52 =====
  ===== sparse-compressed-test OK 13:00:00 =====
  ===== sparse-test OK 13:01:04 =====
  ===== two-jobs-test OK 13:02:39 =====
  ===== two-vol-test OK 13:03:49 =====
  ===== verify-vol-test OK 13:04:56 =====
  ===== weird-files2-test OK 13:05:47 =====
  ===== weird-files-test OK 13:06:33 =====
  ===== migration-job-test OK 13:08:15 =====
  ===== migration-jobspan-test OK 13:09:33 =====
  ===== migration-volume-test OK 13:10:48 =====
  ===== migration-time-test OK 13:12:59 =====
  ===== hardlink-test OK 13:13:50 =====
  ===== two-pool-test OK 13:18:17 =====
  ===== fast-two-pool-test OK 13:24:02 =====
  ===== two-volume-test OK 13:25:06 =====
  ===== incremental-2disk OK 13:25:57 =====
  ===== 2drive-incremental-2disk OK 13:26:53 =====
  ===== scratch-pool-test OK 13:28:01 =====
Total time = 0:57:55 or 3475 secs

and the working tape tests are run with

make full_test

Test results
  ===== Bacula tape test OK =====
  ===== Small File Size test OK =====
  ===== restore-by-file-tape test OK =====
  ===== incremental-tape test OK =====
  ===== four-concurrent-jobs-tape OK =====
  ===== four-jobs-tape OK =====

Each separate test is self contained in that it initializes to run Bacula from scratch (i.e. newly created database). It will also kill any Bacula session that is currently running. In addition, it uses ports 8101, 8102, and 8103 so that it does not intefere with a production system.

Alternatively, you can do the ./do_disk work by hand with:

make setup

The above will then copy the source code within the regression tree (in directory regress/build), configure it, and build it. There should be no errors. If there are, please correct them before continuing. From this point on, as long as you don't change the Bacula source code, you should not need to repeat any of the above steps. If you pull down a new version of the source code, simply run make setup again.

Once Bacula is built, you can run the basic disk only non-root regression test by entering:

make test

Other Tests

There are a number of other tests that can be run as well. All the tests are a simply shell script keep in the regress directory. For example the ''make test`` simply executes ./all-non-root-tests. The other tests, which are invoked by directly running the script are:

All non-tape tests not requiring root. This is the standard set of tests, that in general, backup some data, then restore it, and finally compares the restored data with the original data.

All non-tape tests requiring root permission. These are a relatively small number of tests that require running as root. The amount of data backed up can be quite large. For example, one test backs up /usr, another backs up /etc. One or more of these tests reports an error - I'll fix it one day.

All tape test not requiring root. There are currently three tests, all run without being root, and backup to a tape. The first two tests use one volume, and the third test requires an autochanger, and uses two volumes. If you don't have an autochanger, then this script will probably produce an error.

All tape and file tests not requiring root. This includes just about everything, and I don't run it very often.

If a Test Fails

If you one or more tests fail, the line output will be similar to:

  !!!!! concurrent-jobs-test failed!!! !!!!!

If you want to determine why the test failed, you will need to rerun the script with the debug output turned on. You do so by defining the environment variable REGRESS_DEBUG with commands such as:


Then from the "regress" directory (all regression scripts assume that you have "regress" as the current directory), enter:


where test-name should be the name of a test script - for example: tests/backup-bacula-test.

Kern Sibbald 2013-08-18