|
The sample commands in this document assume you're running a recent version of Debian. If you're running some other linux distribution or unix flavor, you'll need to substitute the appropriate commands.
Most of these commands will need to be completed as root. Text in blue are commands which would be typed. As always, please do a sanity check that the instructions are correct for your setup before blindly following them.
This step is optional, but recommended. BackupPC4AFS performs a "vos backup" of each volume to create a new .backup volume to immediately dump. This ensures that the dump is the freshest possible. If no other process in the cell is creating .backup volumes (such as a "vos backupsys" cron job), then it is also possible to very quickly tell when a volume was last dumped by BackupPC4AFS by simply looking at the Backup time in the output of "vos examine volume".
If you plan to run BackupPC4AFS alongside another vos dump or afs backup based mechanism, you should take steps to ensure that volume operations (vos backup, vos dump, and backup dump) do not attempt to operate on the same .backup volume at the same time to prevent errors due to volume locking.
Debian offers a version of apache with SSL. It is not advised to send usernames and passwords over a non SSL-encrypted http session.
vm1: # apt-get install apache-ssl Reading package lists... Done Building dependency tree... Done The following extra packages will be installed: apache-common apache2-utils ssl-cert Suggested packages: apache apache-perl apache-doc The following NEW packages will be installed: apache-common apache-ssl apache2-utils ssl-cert 0 upgraded, 4 newly installed, 0 to remove and 4 not upgraded. Need to get 1696kB of archives. After unpacking 4764kB of additional disk space will be used. Do you want to continue [Y/n]? y Get:1 http://a-deb-mirror etch/main apache2-utils 2.2.3-4 [341kB] Get:2 http://a-deb-mirror etch/main apache-common 1.3.34-4.1 [848kB] Get:3 http://a-deb-mirror etch/main ssl-cert 1.0.14 [11.1kB] Get:4 http://a-deb-mirror etch/main apache-ssl 1.3.34-4.1 [496kB] Fetched 1696kB in 0s (6228kB/s) Preconfiguring packages ... Selecting previously deselected package apache2-utils. (Reading database ... 88007 files and directories currently installed.) Unpacking apache2-utils (from .../apache2-utils_2.2.3-4_i386.deb) ... Selecting previously deselected package apache-common. Unpacking apache-common (from .../apache-common_1.3.34-4.1_i386.deb) ... Selecting previously deselected package ssl-cert. Unpacking ssl-cert (from .../ssl-cert_1.0.14_all.deb) ... Selecting previously deselected package apache-ssl. Unpacking apache-ssl (from .../apache-ssl_1.3.34-4.1_i386.deb) ... Setting up apache2-utils (2.2.3-4) ... Setting up apache-common (1.3.34-4.1) ... Setting up ssl-cert (1.0.14) ... Setting up apache-ssl (1.3.34-4.1) ... Creating config file /etc/apache-ssl/httpd.conf with new version Creating config file /etc/apache-ssl/srm.conf with new version Creating config file /etc/apache-ssl/access.conf with new version Creating config file /etc/apache-ssl/modules.conf with new version Starting apache-ssl 1.3 web server.... vm1:~#
vm1:~# /etc/init.d/apache-ssl stop Stopping apache-ssl 1.3 web server.... vm1:~#
vm1:~# adduser --home /home/backuppc4afs --shell /bin/bash backuppc4afs
Adding user `backuppc4afs' ...
Adding new group `backuppc4afs' (1000) ...
Adding new user `backuppc4afs' (1000) with group `backuppc4afs' ...
Creating home directory `/home/backuppc4afs' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for backuppc4afs
Enter the new value, or press ENTER for the default
Full Name []: BackupPC4AFS User
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [y/N] y
vm1:~#
Feel free to change this location. It is critical that it either have sufficient space for the data you plan to back up (fulls plus incrementals) or that you mount another disk or partition here which does. Mounting a large RAID array at this location is recommended.
vm1:~# mkdir -p /var/backuppc4afs vm1:~# chown backuppc4afs:backuppc4afs /var/backuppc4afs vm1:~# chmod 700 /var/backuppc4afs vm1:~#
Visit http://www.sourceforge.net/projects/backuppc4afs and click the "Download BackupPC4AFS" link. Save the file to a convenient location such as /home/backuppc4afs.
As of the date this document was written, the newest release is version 3.0.0.2. Please use the newest version of BackupPC4AFS available.
Change to the download directory and unpack the distribution
vm1:/home/backuppc4afs# tar -zxvpf BackupPC4AFS-3.0.0.2.tar.gz BackupPC4AFS-3.0.0/ BackupPC4AFS-3.0.0/LICENSE BackupPC4AFS-3.0.0/cgi-bin/ < ... full fill listing truncated for brevity ... > BackupPC4AFS-3.0.0/images/0011001.gif BackupPC4AFS-3.0.0/images/logo-orig.gif vm1:/home/backuppc4afs#
Perform the installation. The cgi-bin directory shown below is correct for apache on Debian Etch. Please change it to your distribution's value if it's different.
vm1:/home/backuppc4afs# cd BackupPC4AFS-3.0.0
vm1:/home/backuppc4afs/BackupPC4AFS-3.0.0# perl ./configure.pl
Is this a new installation or upgrade for BackupPC? If this is
an upgrade please tell me the full path of the existing BackupPC
configuration file (eg: /etc/BackupPC/config.pl). Otherwise, just
hit return.
--> Full path to existing main config.pl []?
I found the following locations for these programs:
bzip2 => /bin/bzip2
cat => /bin/cat
df => /bin/df
gtar/tar => /bin/tar
gzip => /bin/gzip
hostname => /bin/hostname
nmblookup =>
par2 =>
perl => /usr/bin/perl
ping => /bin/ping
rsync => /usr/bin/rsync
sendmail => /usr/sbin/sendmail
smbclient =>
split => /usr/bin/split
ssh/ssh2 => /usr/bin/ssh
--> Are these paths correct? [y]? y
Please tell me the hostname of the machine that BackupPC will run on.
--> BackupPC will run on host [vm1]? vm1
BackupPC should run as a dedicated user with limited privileges. You
need to create a user. This user will need read/write permission on
the main data directory and read/execute permission on the install
directory (these directories will be setup shortly).
The primary group for this user should also be chosen carefully.
The data directories and files will have group read permission,
so group members can access backup files.
--> BackupPC should run as user [backuppc]? backuppc4afs
Please specify an install directory for BackupPC. This is where the
BackupPC scripts, library and documentation will be installed.
--> Install directory (full path) [/usr/local/BackupPC]? /usr/lib/BackupPC4AFS
Please specify a data directory for BackupPC. This is where all the
PC backup data is stored. This file system needs to be big enough to
accommodate all the PCs you expect to backup (eg: at least several GB
per machine).
--> Data directory (full path) [/data/BackupPC]? /var/BackupPC4AFS
BackupPC can compress pool files, but it needs the Compress::Zlib
package installed (see www.cpan.org). Compression will provide around a
40% reduction in pool size, at the expense of cpu time. You can leave
compression off and run BackupPC without compression, in which case you
should leave the compression level at 0 (which means off). You could
install Compress::Zlib and turn compression on later, but read the
documentation first about how to do this. Or the better choice is
to quit, install Compress::Zlib, and re-run configure.pl.
--> Compression level [0]? 0
BackupPC has a powerful CGI perl interface that runs under Apache.
A single executable needs to be installed in a cgi-bin directory.
This executable needs to run as set-uid backuppc4afs, or
it can be run under mod_perl with Apache running as user backuppc4afs.
Leave this path empty if you don't want to install the CGI interface.
--> CGI bin directory (full path) []? /usr/lib/cgi-bin
BackupPC's CGI script needs to display various GIF images that
should be stored where Apache can serve them. They should be
placed somewher under Apache's DocumentRoot. BackupPC also
needs to know the URL to access these images. Example:
Apache image directory: /usr/local/apache/htdocs/BackupPC
URL for image directory: /BackupPC
The URL for the image directory should start with a slash.
--> Apache image directory (full path) []? /var/www/BackupPC
--> URL for image directory (omit http://host; starts with '/') []? /BackupPC
Ok, we're about to:
- install the binaries, lib and docs in /usr/lib/BackupPC4AFS,
- create the data directory /var/BackupPC4AFS,
- create/update the config.pl file /etc/BackupPC/config.pl,
- optionally install the cgi-bin interface.
--> Do you want to continue? [y]? y
Created /usr/lib/BackupPC4AFS/bin
Created /usr/lib/BackupPC4AFS/doc
Created /usr/lib/BackupPC4AFS/lib/BackupPC/CGI
Created /usr/lib/BackupPC4AFS/lib/BackupPC/Config
Created /usr/lib/BackupPC4AFS/lib/BackupPC/Lang
Created /usr/lib/BackupPC4AFS/lib/BackupPC/Storage
Created /usr/lib/BackupPC4AFS/lib/BackupPC/Xfer
Created /usr/lib/BackupPC4AFS/lib/BackupPC/Zip
Created /var/www/BackupPC
Created /var/BackupPC4AFS
Created /var/BackupPC4AFS/pool
Created /var/BackupPC4AFS/cpool
Created /var/BackupPC4AFS/pc
Created /var/BackupPC4AFS/trash
Created /etc/BackupPC
Created /var/log/BackupPC
Installing binaries in /usr/lib/BackupPC4AFS/bin
Installing library in /usr/lib/BackupPC4AFS/lib
Installing images in /var/www/BackupPC
Making init.d scripts
Installing docs in /usr/lib/BackupPC4AFS/doc
Installing config.pl and hosts in /etc/BackupPC
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.026 ms
--- localhost ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.026/0.026/0.026/0.000 ms
Installing cgi script BackupPC_Admin in /usr/lib/cgi-bin
Ok, it looks like we are finished. There are several more things you
will need to do:
- Browse through the config file, /etc/BackupPC/config.pl,
and make sure all the settings are correct. In particular,
you will need to set $Conf{CgiAdminUsers} so you have
administration privileges in the CGI interface.
- Edit the list of hosts to backup in /etc/BackupPC/hosts.
- Read the documentation in /usr/lib/BackupPC4AFS/doc/BackupPC.html.
Please pay special attention to the security section.
- Verify that the CGI script BackupPC_Admin runs correctly. You might
need to change the permissions or group ownership of BackupPC_Admin.
If this is an upgrade and you are using mod_perl, you will need
to restart Apache. Otherwise it will have stale code.
- BackupPC should be ready to start. Don't forget to run it
as user backuppc4afs! The installation also contains an
init.d/backuppc script that can be copied to /etc/init.d
so that BackupPC can auto-start on boot. This will also enable
administrative users to start the server from the CGI interface.
See init.d/README.
- To use the AFS features, be certain to read the README-AFS file.
Enjoy!
Edit /etc/BackupPC/config.pl and change/correct the location of voswrap.pl near the end of the file.
In the example above, BackupPC4AFS was installed into /usr/lib/BackupPC4AFS. If you changed the location, make sure to use the correct value.:
BEFORE:
# AFS additions
# document these
# Path to the voswrap.pl script on the server
- $Conf{VosPath} = '$Installdir/bin/voswrap.pl';
# Full command to run voswrap.pl on the server for a backup.
$Conf{VosCmd} = '$voswrapPath \\
AFTER
# AFS additions
# document these
# Path to the voswrap.pl script on the server
+ $Conf{VosPath} = '/usr/lib/BackupPC4AFS/bin/voswrap.pl';
# Full command to run voswrap.pl on the server for a backup.
$Conf{VosCmd} = '$voswrapPath \\
Debian's apache doesn't support suid cgi-bins.
vm1:/home/backuppc4afs/BackupPC4AFS-3.0.0# chmod u-s /usr/lib/cgi-bin/BackupPC_Admin vm1:/home/backuppc4afs/BackupPC4AFS-3.0.0# ls -la /usr/lib/cgi-bin/ total 48 drwxr-xr-x 2 root root 4096 2007-05-24 00:01 . drwxr-xr-x 83 root root 40960 2007-05-24 00:01 .. -r-xr-xr-- 1 backuppc4afs backuppc4afs 3997 2007-05-24 00:01 BackupPC_Admin
Make apache run as the backuppc4afs user. Edit /etc/apache-ssl/httpd.conf and change the User and Group.
262,263c262,263 - User www-data - Group www-data --- + User backuppc4afs + Group backuppc4afs
BackupPC4AFS uses apache's REMOTE_USER variable to control access. This can be set using any existing authentication method, but is normally set via an .htaccess file. Enable .htaccess files for the apache cgi-bin directory by adding an AllowOverride to the cgi-bin Directory entry:
BEFORE:
<Directory /usr/lib/cgi-bin/>
- AllowOverride None
Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
AFTER
<Directory /usr/lib/cgi-bin/>
+ AllowOverride All
Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
Review the instructions for setting up a generic or kerberized .htaccess file.
Add one or more users as a BackupPC4AFS admin. This is done by adding them to the $Conf{CgiAdminUsers} variable in /etc/BackupPC/config.pl. Separate multiple users with spaces.
non-kerberized example
$Conf{CgiAdminUsers} = 'backuppc4afs otheruser';
kerberized example
$Conf{CgiAdminUsers} = 'someuser@MY.KRB5REALM.COM someotheruser@MY.KRB5REALM.COM ';
vm1:~# /etc/init.d/apache-ssl start Starting apache-ssl 1.3 web server.... vm1:~#
vm1:~# cp /home/backuppc4afs/BackupPC4AFS-3.0.0/init.d/debian-backuppc /etc/init.d/backuppc
vm1:~# update-rc.d backuppc4afs defaults
Adding system startup for /etc/init.d/backuppc4afs ...
/etc/rc0.d/K20backuppc4afs -> ../init.d/backuppc4afs
/etc/rc1.d/K20backuppc4afs -> ../init.d/backuppc4afs
/etc/rc6.d/K20backuppc4afs -> ../init.d/backuppc4afs
/etc/rc2.d/S20backuppc4afs -> ../init.d/backuppc4afs
/etc/rc3.d/S20backuppc4afs -> ../init.d/backuppc4afs
/etc/rc4.d/S20backuppc4afs -> ../init.d/backuppc4afs
/etc/rc5.d/S20backuppc4afs -> ../init.d/backuppc4afs
vm1:~# /bin/sh /etc/init.d/backuppc4afs start
Starting backuppc: ok.
vm1:~#
Visit https://your.host.name.dom/cgi-bin/BackupPC_Admin and log in using the username and password you previously added to your .htaccess file.
If you do not see the prompt for username and password, double-check your .htaccess setup and file permissions. The user running apache (backuppc4afs in this example) must be able to read the .htaccess file and the .htpasswd file if it exists.
Once logged in, if you don't see all 12 links under "Server", including "Edit Config" and "Edit Hosts", double-check that the username you're using is one of the listed CgiAdminUsers in config.pl and that you restarted backuppc4afs after making any changes to config.pl.
In order to perform vos dump operations without tokens, we need to copy the AFS cell's keyfile to the backuppc4afs server. This keyfile must be protected. Only the backuppc4afs user should be able to read it. This means that the backuppc4afs server should be kept as secure as your AFS fileservers. This install document won't devolve into a security lecture, but suffice it to say you should turn off all unnecessary services and wrap or firewall the remaining services.
vm1:~# mkdir -p /etc/openafs/server vm1:~# chmod 700 /etc/openafs/server vm1:~# cd /etc/openafs vm1:/etc/openafs# scp -R root@fs1:/etc/openafs/server server/ root@fs1's password: KeyFile 100% 16 0.0KB/s 00:00 ThisCell 100% 16 0.0KB/s 00:00 CellServDB 100% 16 0.0KB/s 00:00 UserList 100% 16 0.0KB/s 00:00 vm1:/etc/openafs/server# chown -R backuppc4afs:backuppc4afs . vm1:/etc/openafs/server# chmod 700 KeyFile
The OpenAFS documentation has more information on doing this, but the general idea is to create one or more AFS volumesets (a list of volumes to backed up together at the same time and according to the same dump schedule). To these volumesets you add one or more volentries specifying the volumes to backup. These volume entries are server/partition/volumename triads and each may be be exact or a regular expression. Since they may be regular expressions, any of the fields can also contain wildcards. Remember, all of the volumes in a volumeset will be dumped at the same time.
Just as with the original, old-school AFS backups, a single volume may belong to more than one volumeset--accidentally or by clever design. Backing up the same volume in multiple volumesets will increase the amount of disk space required.
Under normal circumstances, you want to backup the AFS .backup volumes rather than the readwrite volumes. To create a volumeset named "testvolumeset1" which will back up volumes on any servers, any partition, and with volumename matching the regular expression "user.<anything>.backup" issue the following commands. Notice that "." represents any single character, ".*" represents zero or more of any single character. Literal periods must be escaped with a backslash: "\.".
vm1:/etc/openafs# backup -localauth backup> addvolset -name testvolumeset1 backup> addvolentry -name testvolumeset1 -server .* -partition .* -volumes test\..*\.backupAnd to verify the result
backup> listvolset -name testvolumeset1
Volume set testvolumeset1:
Entry 1: server .*, partition .*, volumes: test\..*\.backup
backup> quit
Once configured correctly, the BackupPC4AFS "getvols.pl" script should yield a list of volumes when given a volumeset.
vm1:/etc/openafs# /usr/lib/BackupPC4AFS/bin/getvols.pl testvolumeset1
volumeset testvolumeset1
backup: waiting for job termination
Entry 1: server .*, partition .*, volumes: test\..*\.backup
VOLUME:test.stephen.backup
VOLUME:test.www.backup
vm1:/etc/openafs#
The volumes listed above, test.stephen.backup and test.www.backup, are the volumes which will be dumped when a backup of the "host" afs_testvolumeset1 triggers.
At a minimum, you'll want to configure the following settings. Suggested values are shown, but please read /etc/backuppc/config.pl or the BackupPC configuration documentation for full details. These settings can be changed via the web by a user previously added to CgiAdminUsers by clicking on "Edit Config", or by editing config.pl directly.
Server | Wakeup Schedule | WakeupSchedule: 23 Server | Concurrent Jobs | MaxBackups: 3 (preliminary tests on my server (dual 2.8GHz HT on gigE) reveal diminishing throughput returns at 4 simultaneous vos dumps. YMMV.) Schedule | Full Backups | FullPeriod: 20.67 Schedule | Full Backups | FullKeepCnt: 3 Schedule | Full Backups | FullKeepCntMin: 3 Schedule | Full Backups | FullAgeMax: 90 Schedule | Incremental Backups | IncrPeriod: 0.67 Schedule | Incremental Backups | IncrKeepCnt: 42 Schedule | Incremental Backups | IncrKeepCntMin: 42 Schedule | Incremental Backups | IncrAgeMax: 90 Schedule | Incremental Backups | IncrLevels: 3,2,5,4,7,6,9,8,1,3,2,5,4,7,6,9,8,1,3,2 Xfer | Xfer Settings | XferMethod: vos
Note that IncrLevels does not include the level 0 (full) dump. The levels list presented here is based on the Towers of Hanoi dump sequence. A full discussion of backup rotations is beyond the scope of this installation doc. For background on designing backup schedules, I recommend UNIX(R) System Administration Handbook by Nemeth, Snyder, SeeBass, and Hein. (it's old, but good.)
It is CRITICAL that the XferMethod be set to "vos" either globally as shown above, or on a per-host basis for each afs volumeset if BackupPC4AFS is going to backup non-afs hosts too. Don't forget to Save your changes.
Add each volumeset to BackupPC4AFS as a host. The hostname for an afs volumeset is "afs_volumesetname"
Hosts | Hosts | Add host: afs_testvolumeset1 dhcp: no/unchecked user: backuppc4afs moreUsers: blank
Repeat as necessary for additional volumesets. After changing any settings, you must restart the backuppc process by doing one of the following:
Test the setup by performing a backup. Click "Host Summary", "afs_testvolumeset1" (or whatever you called your host), ", then "Start Full Backup" (twice in a row).