Subsections


Installing and Configuring PostgreSQL

If you are considering using PostreSQL, you should be aware of their philosophy of upgrades, which could be destabilizing for a production shop. Basically at every major version upgrade, you are required to dump your database in an ASCII format, do the upgrade, and then reload your database (or databases). This is because they frequently update the "data format" from version to version, and they supply no tools to automatically do the conversion. If you forget to do the ASCII dump, your database may become totally useless because none of the new tools can access it due to the format change, and the PostgreSQL server will not be able to start.

If you are building PostgreSQL from source, please be sure to add the --enable-thread-safety option when doing the ./configure for PostgreSQL.

Installing PostgreSQL

If you use the ./configure --with-postgresql=PostgreSQL-Directory statement for configuring Bacula, you will need PostgreSQL version 7.4 or later installed. NOTE! PostgreSQL versions earlier than 7.4 do not work with Bacula. If PostgreSQL is installed in the standard system location, you need only enter --with-postgresql since the configure program will search all the standard locations. If you install PostgreSQL in your home directory or some other non-standard directory, you will need to provide the full path with the --with-postgresql option.

Installing and configuring PostgreSQL is not difficult but can be confusing the first time. If you prefer, you may want to use a package provided by your chosen operating system. Binary packages are available on most PostgreSQL mirrors.

If you prefer to install from source, we recommend following the instructions found in the PostgreSQL documentation.

If you are using FreeBSD, this FreeBSD Diary article will be useful. Even if you are not using FreeBSD, the article will contain useful configuration and setup information.

If you configure the Batch Insert code in Bacula (attribute inserts are 10 times faster), you must be using a PostgreSQL that was built with the --enable-thread-safety option, otherwise you will get data corruption. Most major Linux distros have thread safety turned on, but it is better to check. One way is to see if the PostgreSQL library that Bacula will be linked against references pthreads. This can be done with a command such as:

  nm /usr/lib/libpq.a | grep pthread_mutex_lock

The above command should print a line that looks like:

         U pthread_mutex_lock

if does, then everything is OK. If it prints nothing, do not enable batch inserts when building Bacula.

After installing PostgreSQL, you should return to completing the installation of Bacula. Later, after Bacula is installed, come back to this chapter to complete the installation. Please note, the installation files used in the second phase of the PostgreSQL installation are created during the Bacula Installation. You must still come back to complete the second phase of the PostgreSQL installation even if you installed binaries (e.g. rpm, deb, ...).

Configuring PostgreSQL

At this point, you should have built and installed PostgreSQL, or already have a running PostgreSQL, and you should have configured, built and installed Bacula. If not, please complete these items before proceeding.

Please note that the ./configure used to build Bacula will need to include --with-postgresql=PostgreSQL-directory, where PostgreSQL-directory is the directory name that you specified on the ./configure command for configuring PostgreSQL (if you didn't specify a directory or PostgreSQL is installed in a default location, you do not need to specify the directory). This is needed so that Bacula can find the necessary include headers and library files for interfacing to PostgreSQL.

Bacula will install scripts for manipulating the database (create, delete, make tables etc) into the main installation directory. These files will be of the form *_bacula_* (e.g. create_bacula_database). These files are also available in the <bacula-src>/src/cats directory after running ./configure. If you inspect create_bacula_database, you will see that it calls create_postgresql_database. The *_bacula_* files are provided for convenience. It doesn't matter what database you have chosen; create_bacula_database will always create your database.

Now you will create the Bacula PostgreSQL database and the tables that Bacula uses. These instructions assume that you already have PostgreSQL running. You will need to perform these steps as a user that is able to create new databases. This can be the PostgreSQL user (on most systems, this is the pgsql user).

  1. cd <install-directory>

    This directory contains the Bacula catalog interface routines.

  2. ./create_bacula_database

    This script creates the PostgreSQL bacula database. Before running this command, you should carefully think about what encoding sequence you want for the text fields (paths, files, ...). Ideally, the encoding should be set to UTF8. However, many Unix systems have filenames that are not encoded in UTF8, either because you have not set UTF8 as your default character set or because you have imported files from elsewhere (e.g. MacOS X). For this reason, Bacula uses SQL_ASCII as the default encoding. If you want to change this, please modify the script before running it.

    If running the script fails, it is probably because the database is owned by a user other than yourself. On many systems, the database owner is pgsql and on others such as Red Hat and Fedora it is postgres. You can find out which it is by examining your /etc/passwd file. To create a new user under either your name or with say the name bacula, you can do the following:

       su
       (enter root password)
       su pgsql (or postgres)
       createuser kern (or perhaps bacula)
       Shall the new user be allowed to create databases? (y/n) y
       Shall the new user be allowed to create more new users? (y/n) (choose
             what you want)
       exit
    

    At this point, you should be able to execute the ./create_bacula_database command.

  3. ./make_bacula_tables

    This script creates the PostgreSQL tables used by Bacula.

  4. ./grant_bacula_privileges

    This script creates the database user bacula with restricted access rights. You may want to modify it to suit your situation. Please note that this database is not password protected.

Each of the three scripts (create_bacula_database, make_bacula_tables, and grant_bacula_privileges) allows the addition of a command line argument. This can be useful for specifying the user name. For example, you might need to add -h hostname to the command line to specify a remote database server.

To take a closer look at the access privileges that you have setup with the above, you can do:

PostgreSQL-directory/bin/psql --command \\dp bacula

Also, I had an authorization problem with the password. In the end, I had to modify my pg_hba.conf file (in /var/lib/pgsql/data on my machine) from:

  local   all    all        ident  sameuser
to
  local   all    all        trust  sameuser

This solved the problem for me, but it is not always a good thing to do from a security standpoint. However, it allowed me to run my regression scripts without having a password.

A more secure way to perform database authentication is with md5 password hashes. Begin by editing the pg_hba.conf file, and just prior the the existing ``local'' and ``host'' lines, add the line:

  local bacula bacula md5

and restart the Postgres database server (frequently, this can be done using "/etc/init.d/postgresql restart" or "service postgresql restart") to put this new authentication rule into effect.

Next, become the Postgres administrator, postgres, either by logging on as the postgres user, or by using su to become root and then using su - postgres to become postgres. Add a password to the bacula database for the bacula user using:

  \$ psql bacula
  bacula=# alter user bacula with password 'secret';
  ALTER USER
  bacula=# \\q

You'll have to add this password to two locations in the bacula-dir.conf file: once to the Catalog resource and once to the RunBeforeJob entry in the BackupCatalog Job resource. With the password in place, these two lines should look something like:

  dbname = bacula; user = bacula; password = "secret"
    ... and ...
  RunBeforeJob = "/etc/make_catalog_backup bacula bacula secret"

Naturally, you should choose your own significantly more random password, and ensure that the bacula-dir.conf file containing this password is readable only by the root.

Even with the files containing the database password properly restricted, there is still a security problem with this approach: on some platforms, the environment variable that is used to supply the password to Postgres is available to all users of the local system. To eliminate this problem, the Postgres team have deprecated the use of the environment variable password-passing mechanism and recommend the use of a .pgpass file instead. To use this mechanism, create a file named .pgpass containing the single line:

  localhost:5432:bacula:bacula:secret

This file should be copied into the home directory of all accounts that will need to gain access to the database: typically, root, bacula, and any users who will make use of any of the console programs. The files must then have the owner and group set to match the user (so root:root for the copy in root, and so on), and the mode set to 600, limiting access to the owner of the file.

Re-initializing the Catalog Database

After you have done some initial testing with Bacula, you will probably want to re-initialize the catalog database and throw away all the test Jobs that you ran. To do so, you can do the following:

  cd <install-directory>
  ./drop_bacula_tables
  ./make_bacula_tables
  ./grant_bacula_privileges

Please note that all information in the database will be lost and you will be starting from scratch. If you have written on any Volumes, you must write an end of file mark on the volume so that Bacula can reuse it. Do so with:

   (stop Bacula or unmount the drive)
   mt -f /dev/nst0 rewind
   mt -f /dev/nst0 weof

Where you should replace /dev/nst0 with the appropriate tape drive device name for your machine.

Installing PostgreSQL from RPMs

If you are installing PostgreSQL from RPMs, you will need to install both the PostgreSQL binaries and the client libraries. The client libraries are usually found in a devel package, so you must install:

  postgresql
  postgresql-devel
  postgresql-server
  postgresql-libs

These will be similar with most other package managers too. After installing from rpms, you will still need to run the scripts that set up the database and create the tables as described above.

Converting from MySQL to PostgreSQL

The conversion procedure presented here was worked out by Norm Dressler <ndressler at dinmar dot com>

This process was tested using the following software versions:

WARNING: Always as a precaution, take a complete backup of your databases before proceeding with this process!

  1. Shutdown bacula (cd /etc/bacula;./bacula stop)
  2. Run the following command to dump your Mysql database:

           mysqldump -f -t -n >bacula-backup.dmp
    

  3. Make a backup of your /etc/bacula directory (but leave the original in place).
  4. Go to your Bacula source directory and rebuild it to include PostgreSQL support rather then Mysql support. Check the config.log file for your original configure command and replace enable-mysql with enable-postgresql.
  5. Recompile Bacula with a make and if everything compiles completely, perform a make install.
  6. Shutdown Mysql.
  7. Start PostgreSQL on your system.
  8. Create a bacula user in Postgres with the createuser command. Depending on your Postgres install, you may have to SU to the user who has privileges to create a user.
  9. Verify your pg_hba.conf file contains sufficient permissions to allow bacula to access the server. Mine has the following since it's on a secure network:

    local all all trust
                    
    host all all 127.0.0.1 255.255.255.255 trust
                    
    NOTE: you should restart your postgres server if you
          made changes
    

  10. Change into the /etc/bacula directory and prepare the database and tables with the following commands:

    ./create_postgresql_database
                                    
    ./make_postgresql_tables
                                    
    ./grant_postgresql_privileges
    

  11. Verify you have access to the database:

      
    psql -Ubacula bacula
    

    You should not get any errors.

  12. Load your database from the Mysql database dump with:

    psql -Ubacula bacula <bacula-backup.dmp>
    

  13. Resequence your tables with the following commands:

    psql -Ubacula bacula
                    
    SELECT SETVAL('basefiles_baseid_seq', (SELECT
    MAX(baseid) FROM basefiles));
    SELECT SETVAL('client_clientid_seq', (SELECT
    MAX(clientid) FROM client));
    SELECT SETVAL('file_fileid_seq', (SELECT MAX(fileid)
    FROM file));
    SELECT SETVAL('filename_filenameid_seq', (SELECT
    MAX(filenameid) FROM filename));
                    
    SELECT SETVAL('fileset_filesetid_seq', (SELECT
    MAX(filesetid) FROM fileset));
                    
    SELECT SETVAL('job_jobid_seq', (SELECT MAX(jobid) FROM job));
    SELECT SETVAL('jobmedia_jobmediaid_seq', (SELECT
    MAX(jobmediaid) FROM jobmedia));
    SELECT SETVAL('media_mediaid_seq', (SELECT MAX(mediaid) FROM media));
    SELECT SETVAL('path_pathid_seq', (SELECT MAX(pathid) FROM path));
                    
    SELECT SETVAL('pool_poolid_seq', (SELECT MAX(poolid) FROM pool));
    

  14. At this point, start up Bacula, verify your volume library and perform a test backup to make sure everything is working properly.

Upgrading PostgreSQL

If you upgrade PostgreSQL, you must reconfigure, rebuild, and re-install Bacula otherwise you are likely to get bizarre failures. If you to modify the bacula.spec file to account for the new PostgreSQL version. You can do so by rebuilding from the source rpm. To do so, you may need install from rpms and you upgrade PostgreSQL, you must also rebuild Bacula.

Credits

Many thanks to Dan Langille for writing the PostgreSQL driver. This will surely become the most popular database that Bacula supports.


Installing and Configuring SQLite

Installing and Configuring SQLite -- Phase I

If you use the ./configure --with-sqlite statement for configuring Bacula, you will need SQLite version 2.8.16 or later installed. Our standard location (for the moment) for SQLite is in the dependency package depkgs/sqlite-2.8.16. Please note that the version will be updated as new versions are available and tested.

You may install and use SQLite version 3.x with Bacula by using: ./configure --with-sqlite3. You should ensure that when the database is created that you have used

PRAGMA synchronous = NORMAL;
otherwiset SQLite version 3.x is 4 to 10 times slower than version 2.8.16.

Installing and Configuring is quite easy.

  1. Download the Bacula dependency packages
  2. Detar it with something like:

    tar xvfz depkgs.tar.gz

    Note, the above command requires GNU tar. If you do not have GNU tar, a command such as:

    zcat depkgs.tar.gz | tar xvf -

    will probably accomplish the same thing.

  3. cd depkgs

  4. make sqlite

At this point, you should return to completing the installation of Bacula.

Please note that the ./configure used to build Bacula will need to include --with-sqlite.


Installing and Configuring SQLite -- Phase II

This phase is done after you have run the ./configure command to configure Bacula.

Bacula will install scripts for manipulating the database (create, delete, make tables etc) into the main installation directory. These files will be of the form *_bacula_* (e.g. create_bacula_database). These files are also available in the <bacula-src>/src/cats directory after running ./configure. If you inspect create_bacula_database, you will see that it calls create_sqlite_database. The *_bacula_* files are provided for convenience. It doesn't matter what database you have chosen; create_bacula_database will always create your database.

At this point, you can create the SQLite database and tables:

  1. cd <install-directory>

    This directory contains the Bacula catalog interface routines.

  2. ./make_sqlite_tables

    This script creates the SQLite database as well as the tables used by Bacula. This script will be automatically setup by the ./configure program to create a database named bacula.db in Bacula's working directory.

Linking Bacula with SQLite

If you have followed the above steps, this will all happen automatically and the SQLite libraries will be linked into Bacula.

Testing SQLite

We have much less "production" experience using SQLite than using MySQL. SQLite has performed flawlessly for us in all our testing. However, several users have reported corrupted databases while using SQLite. For that reason, we do not recommend it for production use.

If Bacula crashes with the following type of error when it is started:

Using default Catalog name=MyCatalog DB=bacula
Could not open database "bacula".
sqlite.c:151 Unable to open Database=/var/lib/bacula/bacula.db.
ERR=malformed database schema - unable to open a temporary database file
for storing temporary tables

this is most likely caused by the fact that some versions of SQLite attempt to create a temporary file in the current directory. If that fails, because Bacula does not have write permission on the current directory, then you may get this errr. The solution is to start Bacula in a current directory where it has write permission.

Re-initializing the Catalog Database

After you have done some initial testing with Bacula, you will probably want to re-initialize the catalog database and throw away all the test Jobs that you ran. To do so, you can do the following:

  cd <install-directory>
  ./drop_sqlite_tables
  ./make_sqlite_tables

Please note that all information in the database will be lost and you will be starting from scratch. If you have written on any Volumes, you must write an end of file mark on the volume so that Bacula can reuse it. Do so with:

   (stop Bacula or unmount the drive)
   mt -f /dev/nst0 rewind
   mt -f /dev/nst0 weof

Where you should replace /dev/nst0 with the appropriate tape drive device name for your machine.


The Bacula internal database is no longer supported, please do not use it.

Internal Bacula Database

Previously it was intended to be used primarily by Bacula developers for testing; although SQLite is also a good choice for this. We do not recommend its use in general.

This database is simplistic in that it consists entirely of Bacula's internal structures appended sequentially to a file. Consequently, it is in most cases inappropriate for sites with many clients or systems with large numbers of files, or long-term production environments.

Below, you will find a table comparing the features available with SQLite and MySQL and with the internal Bacula database. At the current time, you cannot dynamically switch from one to the other, but must rebuild the Bacula source code. If you wish to experiment with both, it is possible to build both versions of Bacula and install them into separate directories.

Feature SQLite or MySQL Bacula
Job Record Yes Yes
Media Record Yes Yes
FileName Record Yes No
File Record Yes No
FileSet Record Yes Yes
Pool Record Yes Yes
Client Record Yes Yes
JobMedia Record Yes Yes
List Job Records Yes Yes
List Media Records Yes Yes
List Pool Records Yes Yes
List JobMedia Records Yes Yes
Delete Pool Record Yes Yes
Delete Media Record Yes Yes
Update Pool Record Yes Yes
Implement Verify Yes No
MD5 Signatures Yes No

In addition, since there is no SQL available, the Console commands: sqlquery, query, retention, and any other command that directly uses SQL are not available with the Internal database.

Kern Sibbald 2008-01-31