Subsections

New Features in 3.1.4 (Development Version

This chapter presents the new features that are currently under development in the 3.1.x versions to be released as Bacula version 3.2.0 sometime in late 2009 or early 2010.

Truncate volume after purge

The Pool directive ActionOnPurge=Truncate instructs Bacula to truncate the volume when it is purged. It is useful to prevent disk based volumes from consuming too much space.

Pool {
  Name = Default
  Action On Purge = Truncate
  ...
}

Maximum Concurrent Jobs for Devices

Maximum Concurrent Jobs is a new Device directive in the Storage Daemon configuration permits setting the maximum number of Jobs that can run concurrently on a specified Device. Using this directive, it is possible to have different Jobs using multiple drives, because when the Maximum Concurrent Jobs limit is reached, the Storage Daemon will start new Jobs on any other available compatible drive. This facilitates writing to multiple drives with multiple Jobs that all use the same Pool.

Restore from Multiple Storage Daemons

Previously, you were able to restore from multiple devices in a single Storage Daemon. Now, Bacula is able to restore from multiple Storage Daemons. For example, if your full backup runs on a Storage Daemon with an autochanger, and your incremental jobs use another Storage Daemon with lots of disks, Bacula will switch automatically from one Storage Daemon to an other within the same Restore job.

You must upgrade your File Daemon to version 3.1.3 or greater to use this feature.

This project was funded by Bacula Systems with the help of Equiinet.

File Deduplication using Base Jobs

A base job is sort of like a Full save except that you will want the FileSet to contain only files that are unlikely to change in the future (i.e. a snapshot of most of your system after installing it). After the base job has been run, when you are doing a Full save, you specify one or more Base jobs to be used. All files that have been backed up in the Base job/jobs but not modified will then be excluded from the backup. During a restore, the Base jobs will be automatically pulled in where necessary.

This is something none of the competition does, as far as we know (except perhaps BackupPC, which is a Perl program that saves to disk only). It is big win for the user, it makes Bacula stand out as offering a unique optimization that immediately saves time and money. Basically, imagine that you have 100 nearly identical Windows or Linux machine containing the OS and user files. Now for the OS part, a Base job will be backed up once, and rather than making 100 copies of the OS, there will be only one. If one or more of the systems have some files updated, no problem, they will be automatically restored.

A new Job directive Base=Jobx, Joby... permits to specify the list of files that will be used during Full backup as base.

Job {
   Name = BackupLinux
   Level= Base
   ...
}

Job {
   Name = BackupZog4
   Base = BackupZog4, BackupLinux
   Accurate = yes
   ...
}

In this example, the job BackupZog4 will use the most recent version of all files contained in BackupZog4 and BackupLinux jobs. Base jobs should have run with level=Base to be used.

By default, Bacula will compare permissions bits, user and group fields, modification time, size and the checksum of the file to choose between the current backup and the BaseJob file list. You can change this behavior with the BaseJob FileSet option. This option works like the verify= one, that is described in the FileSetFileSetResource chapter.

FileSet {
  Name = Full
  Include = {
    Options {
       BaseJob  = pmugcs5
       Accurate = mcs5
       Verify   = pin5
    }
    File = /
  }
}

This project was funded by Bacula Systems.

AllowCompression = yesno

This new directive may be added to Storage resource within the Director's configuration to allow users to selectively disable the client compression for any job which writes to this storage resource.

For example:

Storage {
  Name = UltriumTape
  Address = ultrium-tape
  Password = storage_password # Password for Storage Daemon
  Device = Ultrium
  Media Type = LTO 3
  AllowCompression = No # Tape drive has hardware compression
}

The above example would cause any jobs running with the UltriumTape storage resource to run without compression from the client file daemons. This effectively overrides any compression settings defined at the FileSet level.

This feature is probably most useful if you have a tape drive which supports hardware compression. By setting the AllowCompression = No directive for your tape drive storage resource, you can avoid additional load on the file daemon and possibly speed up tape backups.

This project was funded by Collaborative Fusion, Inc.

Accurate Fileset Options

In previous versions, the accurate code used the file creation and modification times to determine if a file was modified or not. Now you can specify which attributes to use (time, size, checksum, permission, owner, group, ...), similar to the Verify options.

FileSet {
  Name = Full
  Include = {
    Options {
       Accurate = mcs5
       Verify   = pin5
    }
    File = /
  }
}

: i compare the inodes
: p compare the permission bits
: n compare the number of links
: u compare the user id
: g compare the group id
: s compare the size
: a compare the access time
: m compare the modification time (st_mtime)
: c compare the change time (st_ctime)
: d report file size decreases
: 5 compare the MD5 signature
: 1 compare the SHA1 signature

Important note: If you decide to use checksum in Accurate jobs, the File Daemon will have to read all files even if they normally would not be saved. This increases the I/O load, but also the accuracy of the deduplication. By default, Bacula will check modification/creation time and size.

Tab-completion for Bconsole

If you build bconsole with readline support, you will be able to use the new auto-completion mode. This mode supports all commands, gives help inside command, and lists resources when required. It works also in the restore mode.

To use this feature, you should have readline development package loaded on your system, and use the following option in configure.

./configure --with-readline=/usr/include/readline --disable-conio ...

The new bconsole won't be able to tab-complete with older directors.

Bvfs API

To help developers of restore GUI interfaces, we have added new dot commands that permit browsing the catalog in a very simple way.

.bvfs_update [jobid=x,y,z] This command is required to update the Bvfs cache in the catalog. You need to run it before any access to the Bvfs layer.
.bvfs_lsdirs jobid=x,y,z path=/path | pathid=101 This command will list all directories in the specified path or pathid. Using pathid avoids problems with character encoding of path/filenames.
.bvfs_lsfiles jobid=x,y,z path=/path | pathid=101 This command will list all files in the specified path or pathid. Using pathid avoids problems with character encoding.

You can use limit=xxx and offset=yyy to limit the amount of data that will be displayed.

* .bvfs_update jobid=1,2
* .bvfs_update
* .bvfs_lsdir path=/ jobid=1,2

Testing your Tape Drive

To determine the best configuration of your tape drive, you can run the new speed command available in the btape program.

This command can have the following arguments:

file_size=n: Specify the Maximum File Size for this test (between 1 and 5GB). This counter is in GB.
nb_file=n: Specify the number of file to be written. The amount of data should be greater than your memory ( $file\_size*nb\_file$ ).
skip_zero: This flag permits to skip tests with constant data.
skip_random: This flag permits to skip tests with random data.
skip_raw: This flag permits to skip tests with raw access.
skip_block: This flag permits to skip tests with Bacula block access.

*speed file_size=3 skip_raw
btape.c:1078 Test with zero data and bacula block structure.
btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes.
++++++++++++++++++++++++++++++++++++++++++
btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0)
btape.c:406 Volume bytes=3.221 GB. Write rate = 44.128 MB/s
...
btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 43.531 MB/s

btape.c:1090 Test with random data, should give the minimum throughput.
btape.c:956 Begin writing 3 files of 3.221 GB with blocks of 129024 bytes.
+++++++++++++++++++++++++++++++++++++++++++
btape.c:604 Wrote 1 EOF to "Drive-0" (/dev/nst0)
btape.c:406 Volume bytes=3.221 GB. Write rate = 7.271 MB/s
+++++++++++++++++++++++++++++++++++++++++++
...
btape.c:383 Total Volume bytes=9.664 GB. Total Write rate = 7.365 MB/s

When using compression, the random test will give your the minimum throughput of your drive . The test using constant string will give you the maximum speed of your hardware chain. (cpu, memory, scsi card, cable, drive, tape).

You can change the block size in the Storage Daemon configuration file.

New Block Checksum Device Directive

You may now turn off the Block Checksum (CRC32) code that Bacula uses when writing blocks to a Volume. This is done by adding:

Block Checksum = no

doing so can reduce the Storage daemon CPU usage slightly. It will also permit Bacula to read a Volume that has corrupted data.

The default is yes - i.e. the checksum is computed on write and checked on read.

We do not recommend to turn this off particularly on older tape drives or for disk Volumes where doing so may allow corrupted data to go undetected.

New Bat Features

Media List View

By clicking on ``Media'', you can see the list of all your volumes. You will be able to filter by Pool, Media Type, Location,...And sort the result directly in the table. The old ``Media'' view is now known as ``Pool''.

$\includegraphics[width=13cm]{bat-mediaview.eps}$

Media Information View

By double-clicking on a volume (on the Media list, in the Autochanger content or in the Job information panel), you can access a detailed overview of your Volume. (cf 2.1.)

**Figure 2.1:** Media information
$\includegraphics[width=13cm]{bat11.eps}$

Job Information View

By double-clicking on a Job record (on the Job run list or in the Media information panel), you can access a detailed overview of your Job. (cf 2.2.)

**Figure 2.2:** Job information
$\includegraphics[width=13cm]{bat12.eps}$

Autochanger Content View

By double-clicking on a Storage record (on the Storage list panel), you can access a detailed overview of your Autochanger. (cf 2.2.)

**Figure 2.3:** Autochanger content
$\includegraphics[width=13cm]{bat13.eps}$

Console Timeout Option

You can now use the -u option of bconsole to set a timeout for each command.

Kern Sibbald 2012-06-02