Oh no, Backup - New Information!

NetBackup Intro

Once again the backup / restore method for bama has changed. We are using a different suite of software ("NetBackup") on bama to store backups in a new tape library with eight tape drives shared among several servers. The new system has reduced backup times by another factor of 2, so that most nights bama can be backed up in an hour. Along with this comes a change in how users will recover lost files. The old methods, recover or nwrecover, are no longer available.

We keep each night's backup for two months. For weekly full backups, made on Friday nights (running midnight - Saturday morning), this includes all files as they existed at the time of backup, regardless of whether they are later changed or deleted. Daily incremental backups on all other nights include only files added or changed since the previous backup. As with the old software, a deleted file must have been on the system for at least one day to be included in a backup.

Individual users can run the program to restore files they have changed or deleted. You can restore individual files, subdirectory trees, and search through multiple backup copies of the same file reaching back over several dates. There are two parts to the backup system. The first is a database that lists all backed-up files along with the backup dates. The second is the actual tape library that holds the files. When you ask to restore a file it is first looked up in the database. When the software receives a go-ahead to restore the file, a command is issued to the tape device to mount the tape and the restore continues.

Command-line restore

From the Unix command prompt type bp -r. The program will display a text-based menu screen (1) in your terminal window. At this point you can press h for the online help, but the important parts are described here. You will need to set a few settings first, then search for backed-up files and select some to restore, finally initiating the restore and exiting the bp program. Once begun, the restore will continue in the background (you can view progress through an optional log file).

Press a to change the destination path.
WARNING: When restoring an older version of an existing file, NetBackup will either overwrite the existing file or not restore the file at all, unless you change the destination path. We recommend changing the destination path to put restored files in a different directory from the originals, and making this the first thing you do each time you run bp. You'll be prompted for paths to Restore From: and Restore To:. When restoring files, bp will simply edit the beginning of each filename to replace the Restore From: path (if it appears) with the Restore To: path. The Restore To: path does not have to exist before you initiate the restore.

Examples:
- To redirect files from wherever they might have been within your home directory, into a new subdirectory in your home directory:
```
Enter Restore From: (/user0/username/)
Enter Restore To: (/user0/username/) /user0/username/restored.20070726
        
```
- To redirect files from anywhere they might have been on bama, into a new temporary directory: If you do this, you should create the temporary directory first to ensure that you can write to it.
```
Enter Restore From: (/user0/username/) /
Enter Restore To: (/user0/username/) /work/scratch/username/
        
```
- If you are restoring only a single file, you can use this feature to give the restored file a new name (e.g., make a copy of your email inbox into an IMAP folder):
```
Enter Restore From: (/user0/username/) /user0/username/.newmail
Enter Restore To: (/user0/username/) /user0/username/mail/OLD_INBOX
        
```

Press p to change the search path if necessary. When bp starts, this will be the current directory, so normally you would be searching in your home directory. The top line of the menu screen shows the current Search Path. If you are looking for files elsewhere on bama, cd into the directory before running bp, or set this option. If it asks whether to "Resolve links in New File Path?"; press enter to accept the default.

Press d to change the date range if necessary. The default date range is from the beginning of the most recent full backup to the end of today. Dates should be entered as MM/DD/YY. If you do not include times of day, the default is 00:00:00 on the start date and 23:59:59 on the end date. Note that these dates and times specify which backups to include in the search, not when your files might have changed or what point in time you want to go back to. To see the full contents of a directory that has been on bama for some time, make sure at least one Friday night (full backups run midnight - Saturday morning) is included in the date range.
Examples:
- To search all backups, set the start date to long ago:
```
Current Start Date:    07/21/2007 00:00:05
    New Start Date:    1/1/1
        
```
- To see backups of a directory made last week, up to Wednesday morning:
```
Current Start Date:    07/21/2007 00:00:05
    New Start Date:    7/14/07

  Current End Date:    07/24/2007 23:59:59
      New End Date:    7/18/07 6:00
        
```

Press c to change the directory depth if necessary. By default, the search listing (described below) shows only one level of directory contents at a time, and navigating into and out of subdirectories can be awkward. Enter 0 to show all levels of subdirectory contents, but beware this may slow the search or even crash the program if you search a large enough directory hierarchy.

Press s to search for files among the backups. Depending on the search path and other settings, it may take a while for the server to return a list of files. While in search mode:
- Press s to select or unselect a file for restore. Selected files are marked with an asterisk (*). If you select a directory, all files and directories within it will be restored, but files may be restored from a more recent backup than the directory itself. You may need to select individual files, or change the date range (see above), to restore the correct versions.
- Press x to toggle between brief and verbose listings. The default is brief, which shows only the time of backup and the filename; a verbose listing is more similar to ls -l output. To see all columns in a verbose listing, you may need to use right and left arrow keys, or widen the window before running bp.
- Press i to navigate into a subdirectory, or z to go out to the parent directory. These keys may not do what you expect; when you navigate into a subdirectory, only files that were included in that particular backup of the subdirectory are shown. To see all versions of a file in a subdirectory, you can either change the search path (see above) to that particular directory, or increase the directory depth (see above), before entering search mode.
- Press o or q to return to the menu screen.
Press e to review your selections. In edit mode you can:
- Press s to unselect a file.
- Press a to add a selection by typing it in directly.
- Press c to clear all selections.
- Press o or q to return to the menu screen.

Press i to initiate the restore. This will ask a few questions (pressing enter to use the default is usually appropriate), then start running the restore in the background and return you to the menu screen:

Restoring the following files

/user0/username/.newmail 

Overwrite if file(s) exist? (y/n) (n): 
Use progress log? (y/n) (y):
Enter Progress File Path:  (/user0/username/bplog.rest.001) 

diagnostic output will be logged to /user0/username/bplog.rest.001

Use destination path substitution? (y/n) (y): 
Rename hard links relative to the destination path? (y/n) (y): 
Rename soft links relative to the destination path? (y/n) (y): 

Initiating Restore . . .

The restore was successfully initiated.

Press q to exit the menu screen, then q again to exit the bp program.

A restore may take a matter of minutes or hours to complete, depending on the files selected, the arrangement of backups on tape, other activity of the tape library, and so on. To keep track of the progress of the restore, you can periodically read the log file:

  $ tail bplog.rest.###
  Restore started 07/25/2007 16:11:57

  16:11:58 (9060.xxx) Restore job id 9060 will require 1 image.
  16:11:58 (9060.xxx) Media id E50384 is needed for the restore.

  16:12:33 (9060.001) Restoring from image created Sat Jul 07 01:24:10 2007
  16:12:35 (9060.001) TAR STARTED
  16:12:40 (9060.001) INF - Waiting for mount of media id E50384 on server
                      bama.ua.edu for reading.

To keep the log file open, displaying new output whenever it appears, you can use

  $ tail -f bplog.rest.###

When the restore is completed, the log file should include a status message to that effect. Press ctrl + c to stop watching the log file.

Xwindows restore

If you are running Xwindows software on your PC (or Mac) when you connect to bama, then you can run the Xwindows NetBackup software. Use your usual method to enable the display of Xwindows programs (Mac users may need to change X11 forwarding options). Then type jbpSA to start the Xwindows (Java) version of NetBackup. The program will prompt you to log in. Your bama account name and password are required. This program also has online help available (the sections on restoring "Normal Backups" and finding items to restore are the most useful).

Make sure you are viewing the "Restore Files" tab of the window. Refresh the listing (in the View menu) in order to see files and directories available to restore. You may need to adjust the date range or other options, or use the search tool to locate files. By default only one version of each file is shown: the most recent backup of the file within the date range. To see multiple versions, change this option in the View menu. As with the old software, you "mark" the files or directories you want to restore by clicking in the box next to the name. Selected files or directories are marked with check marks, and partially selected directory structures are marked with slashes. When you are ready to restore, click the "Restore..." button at lower right.

The "Restore Marked Files" dialog appears. This dialog lets you change the destination path for all of the restored files, or set individual destinations for each restored file. We strongly recommend using one of these options; see the WARNING about destination paths above. Other options in the dialog can usually be left at default settings. Click the "Start Restore" button to initiate the restore. It will run in the background, and you can view progress in the "Task Progress" tab of the jbpSA window.

Man pages and online help are available for both of these commands.

Footnote

1 - Lower-level Unix commands are available to do restores entirely from the command line, but are not supported. See man bprestore.

© 2000,2007, The University of Alabama. The information included here is for the University of Alabama central computing facility as it was configured on the document date. It may or may not apply to other Unix systems.

Amended August, 2007 Vol. 4 Index Back to Vol. 4, No. 3		Search:

	E-mail Internet Access Software Assistance Computer Labs Software Licensing Web Publishing Calendaring Computer Security Unix Training

	Oh no, Backup - New Information! NetBackup Intro Once again the backup / restore method for bama has changed. We are using a different suite of software ("NetBackup") on bama to store backups in a new tape library with eight tape drives shared among several servers. The new system has reduced backup times by another factor of 2, so that most nights bama can be backed up in an hour. Along with this comes a change in how users will recover lost files. The old methods, recover or nwrecover, are no longer available. We keep each night's backup for two months. For weekly full backups, made on Friday nights (running midnight - Saturday morning), this includes all files as they existed at the time of backup, regardless of whether they are later changed or deleted. Daily incremental backups on all other nights include only files added or changed since the previous backup. As with the old software, a deleted file must have been on the system for at least one day to be included in a backup. Individual users can run the program to restore files they have changed or deleted. You can restore individual files, subdirectory trees, and search through multiple backup copies of the same file reaching back over several dates. There are two parts to the backup system. The first is a database that lists all backed-up files along with the backup dates. The second is the actual tape library that holds the files. When you ask to restore a file it is first looked up in the database. When the software receives a go-ahead to restore the file, a command is issued to the tape device to mount the tape and the restore continues. Command-line restore From the Unix command prompt type bp -r. The program will display a text-based menu screen (1) in your terminal window. At this point you can press h for the online help, but the important parts are described here. You will need to set a few settings first, then search for backed-up files and select some to restore, finally initiating the restore and exiting the bp program. Once begun, the restore will continue in the background (you can view progress through an optional log file). Press a to change the destination path. WARNING: When restoring an older version of an existing file, NetBackup will either overwrite the existing file or not restore the file at all, unless you change the destination path. We recommend changing the destination path to put restored files in a different directory from the originals, and making this the first thing you do each time you run bp. You'll be prompted for paths to Restore From: and Restore To:. When restoring files, bp will simply edit the beginning of each filename to replace the Restore From: path (if it appears) with the Restore To: path. The Restore To: path does not have to exist before you initiate the restore. Examples: To redirect files from wherever they might have been within your home directory, into a new subdirectory in your home directory: Enter Restore From: (/user0/username/) Enter Restore To: (/user0/username/) /user0/username/restored.20070726 To redirect files from anywhere they might have been on bama, into a new temporary directory: If you do this, you should create the temporary directory first to ensure that you can write to it. Enter Restore From: (/user0/username/) / Enter Restore To: (/user0/username/) /work/scratch/username/ If you are restoring only a single file, you can use this feature to give the restored file a new name (e.g., make a copy of your email inbox into an IMAP folder): Enter Restore From: (/user0/username/) /user0/username/.newmail Enter Restore To: (/user0/username/) /user0/username/mail/OLD_INBOX Press p to change the search path if necessary. When bp starts, this will be the current directory, so normally you would be searching in your home directory. The top line of the menu screen shows the current Search Path. If you are looking for files elsewhere on bama, cd into the directory before running bp, or set this option. If it asks whether to "Resolve links in New File Path?"; press enter to accept the default. Press d to change the date range if necessary. The default date range is from the beginning of the most recent full backup to the end of today. Dates should be entered as MM/DD/YY. If you do not include times of day, the default is 00:00:00 on the start date and 23:59:59 on the end date. Note that these dates and times specify which backups to include in the search, not when your files might have changed or what point in time you want to go back to. To see the full contents of a directory that has been on bama for some time, make sure at least one Friday night (full backups run midnight - Saturday morning) is included in the date range. Examples: To search all backups, set the start date to long ago: Current Start Date: 07/21/2007 00:00:05 New Start Date: 1/1/1 To see backups of a directory made last week, up to Wednesday morning: Current Start Date: 07/21/2007 00:00:05 New Start Date: 7/14/07 Current End Date: 07/24/2007 23:59:59 New End Date: 7/18/07 6:00 Press c to change the directory depth if necessary. By default, the search listing (described below) shows only one level of directory contents at a time, and navigating into and out of subdirectories can be awkward. Enter 0 to show all levels of subdirectory contents, but beware this may slow the search or even crash the program if you search a large enough directory hierarchy. Press s to search for files among the backups. Depending on the search path and other settings, it may take a while for the server to return a list of files. While in search mode: Use arrow keys to move the selection arrow pointer (->) up or down. Press space or b to go forward or backward a page at a time. Press s to select or unselect a file for restore. Selected files are marked with an asterisk (). If you select a directory, all files and directories within it will be restored, but files may be restored from a more recent backup than the directory itself. You may need to select individual files, or change the date range (see above), to restore the correct versions. Press x to toggle between brief and verbose listings. The default is brief, which shows only the time of backup and the filename; a verbose listing is more similar to ls -l output. To see all columns in a verbose listing, you may need to use right and left arrow keys, or widen the window before running bp. Press i to navigate into a subdirectory, or z to go out to the parent directory. These keys may not do what you expect; when you navigate into a subdirectory, only files that were included in that particular backup of the subdirectory are shown. To see all versions of a file in a subdirectory, you can either change the search path (see above) to that particular directory, or increase the directory depth (see above), before entering search mode. Press o or q to return to the menu screen. Press e to review your selections. In edit mode you can: Press s to unselect a file. Press a to add a selection by typing it in directly. Press c to clear all selections. Press o or q to return to the menu screen. Press i to initiate the restore. This will ask a few questions (pressing enter to use the default is usually appropriate), then start running the restore in the background and return you to the menu screen: Restoring the following files /user0/username/.newmail Overwrite if file(s) exist? (y/n) (n): Use progress log? (y/n) (y): Enter Progress File Path: (/user0/username/bplog.rest.001) diagnostic output will be logged to /user0/username/bplog.rest.001 Use destination path substitution? (y/n) (y): Rename hard links relative to the destination path? (y/n) (y): Rename soft links relative to the destination path? (y/n) (y): Initiating Restore . . . The restore was successfully initiated. Press q to exit the menu screen, then q again to exit the bp program. A restore may take a matter of minutes or hours to complete, depending on the files selected, the arrangement of backups on tape, other activity of the tape library, and so on. To keep track of the progress of the restore, you can periodically read the log file: $ tail bplog.rest.### Restore started 07/25/2007 16:11:57 16:11:58 (9060.xxx) Restore job id 9060 will require 1 image. 16:11:58 (9060.xxx) Media id E50384 is needed for the restore. 16:12:33 (9060.001) Restoring from image created Sat Jul 07 01:24:10 2007 16:12:35 (9060.001) TAR STARTED 16:12:40 (9060.001) INF - Waiting for mount of media id E50384 on server bama.ua.edu for reading. To keep the log file open, displaying new output whenever it appears, you can use $ tail -f bplog.rest.### When the restore is completed, the log file should include a status message to that effect. Press ctrl + c to stop watching the log file. Xwindows restore If you are running Xwindows software on your PC (or Mac) when you connect to bama, then you can run the Xwindows NetBackup software. Use your usual method to enable the display of Xwindows programs (Mac users may need to change X11 forwarding options). Then type jbpSA to start the Xwindows (Java) version of NetBackup. The program will prompt you to log in. Your bama account name and password are required. This program also has online help available (the sections on restoring "Normal Backups" and finding items to restore are the most useful). Make sure you are viewing the "Restore Files" tab of the window. Refresh the listing (in the View menu) in order to see files and directories available to restore. You may need to adjust the date range or other options, or use the search tool to locate files. By default only one version of each file is shown: the most recent backup of the file within the date range. To see multiple versions, change this option in the View menu. As with the old software, you "mark" the files or directories you want to restore by clicking in the box next to the name. Selected files or directories are marked with check marks, and partially selected directory structures are marked with slashes. When you are ready to restore, click the "Restore..." button at lower right. The "Restore Marked Files" dialog appears. This dialog lets you change the destination path for all of the restored files, or set individual destinations for each restored file. We strongly recommend using one of these options; see the WARNING* about destination paths above. Other options in the dialog can usually be left at default settings. Click the "Start Restore" button to initiate the restore. It will run in the background, and you can view progress in the "Task Progress" tab of the jbpSA window. Man pages and online help are available for both of these commands. Footnote 1 - Lower-level Unix commands are available to do restores entirely from the command line, but are not supported. See man bprestore. © 2000,2007, The University of Alabama. The information included here is for the University of Alabama central computing facility as it was configured on the document date. It may or may not apply to other Unix systems.
Copyright © 1998-2007 The University of Alabama \| Disclaimer \| Contact: Help.Desk@ua.edu