2008/11/28

Net Backup

Starting and Stopping Netbackup

Stopping Netbackup
􀁺 /usr/openv/netbackup/bin/K77netbackup --> graceful shutdown
􀁺 /usr/openv/netbackup/bin/bpps -a --> check for any remaining processes
􀁺 /usr/openv/netbackup/bin/goodies/bp.kill_all ---> kills all remaining netbackup processes, not necessarily graceful
􀁺 /usr/openv/netbackup/bin/bpps -a --> check for any remaining processes
􀁺 kill -9 for any remaining. NOTE: unkillable processes may require a reboot

Starting Netbackup
􀁺 /usr/openv/netbackup/bin/S77netbackup --> after bp.kill_all, to restart

Common Tasks

Starting the Administration GUI
􀁺 java from the windows client
􀁺 x-windows from the server - /usr/openv/netbackup/bin/xnb &


Checking Backup Status
􀁺 Activity Monitor or
􀁺 /usr/openv/netbackup/bin/admincmd/bpdbjobs -report


Cleaning a tape manually
􀁺 Identify the drive name to be cleaned
􀁻 tpclean -L
􀁺 Manually clean the drive:
􀁻 tpclean -C

Determining what tapes were used for a backup
􀁺 GUI
􀁻 Backup and Restore --> Find the file system --> Preview Media Button
􀁺 CLI
􀁻 Find the correct backup images
􀂄 bpimagelist -U -client -d -e
􀁻 Find the media used for those images
􀂄 bpimagelist -U -client -d -e -media


Listing the files in a backup

Find the tape(s) used (above procedure using bpimagelist)
cd /usr/openv/netbackup/db/jobs/done
Run the following script and redirect it's output to a text file:
for file in `grep MOUNTING *grep awk '{print $1}'sed 's/:MOUNTING//'`
do
echo $file
grep PATH_WRITTEN $fileawk '{print $3}'
echo " "
echo "==========================================End of
Image======================================"
echo " "
done


This process works for NBU V3.4:
􀁺 cd /usr/openv/netbackup/db/images/
􀁺 ls -ltr --> this will identify the directory with the proper date
􀁺 verify directory with "bpdbm -ctime
􀁺 cd
􀁺 ls -ltr --> lists all of the backups for this client on this date
􀁺 cat __.f awk '{print $10}' --> this prints out the files in the backup

For NBU > V3.4

􀁺 bpflist --help --> undocumented netbackup command to list files from a binary .f file


Inventory the Robot
􀁺 Inventory Robot --> /opt/openv/volmgr/bin/vmcheckxxx -rt robot_type -rn robot_number -list
(where robot_type is tld, acs, . . .)
􀁺 Inventory Robot and Update Configuration --> /opt/openv/volmgr/bin/vmupdate -rt robot_type -rn robot_number -list (where robot_type is tld, acs, . . .)

Listing Properties of the Volume Pools
􀁺 vmpool -listall


Scratch Tapes

􀁺 Count scratch tapes: /usr/openv/volmgr/bin/vmquery -pn Scratch grep -c "robot slot"
􀁺 Moving tapes to the scratch pool

􀁻 If Needed - Expire the tape
􀂄 bpexpdate -ev -d 0 -force -host
􀁻 Move the tape
􀂄 vmchange -p 2 -m


Checking Drive Usage
􀁺 /usr/openv/volmgr/bin/vmoprcmd

Taking a drive down or up
􀁺 /usr/openv/volmgr/vmoprcmd -down
􀁺 /usr/openv/volmgr/vmoprcmd -up

Performing a Restore
􀁺 From the GUI
􀁻 user backup & restore --> configuration --> client
􀁻 user backup & restore --> configuration --> client to restore
􀁻 directory to search
􀁻 directory depth
􀁻 date range
􀁻 file --> browse backups for restore

Adding New Tapes to the Library
􀁺 Using the GUI
􀁻 Media Management --> Actions --> New --> Single Volume . . -->
􀁻 Media Type (ie DLT)
􀁻 Robot Type (ie TLD)
􀁻 Media ID (from Inventory)
􀁻 Slot Number (from Inventory)
􀁻 Robot Number (ie 0)
􀁻 Volume Group
􀁻 Volume Pool (ie Scratch)


􀁺 Using the CLI
􀁻 vmadd -m -mt -verbose -rt -b -rn -rc1 -p -mm
􀂄 vmpool -listall --> lists all pools, both name and number
􀁻 For example: vmadd -m 000151 -mt dlt -verbose -rt tld -b 000151 -rn 0 -rc1 8 -p 2 -mm
0


Re-using Tapes from other systems or older Netbackups
􀁺 Expire the media
􀁻 bpexpdate -ev MEDIA_ID -d 0 -force -host HOST
􀁺 Deassign the media
􀁻 vmquery -deassignbyid MEDIA_ID 4 0
􀁺 Move to the scratch pool
􀁻 vmchange -m MEDIA_ID -p POOL#
􀁺 Relabel the media
􀁻 bplabel -ev CIM572 -d dlt -p Scratch


Changing the attributes of media
􀁺 Changing the barcode
􀁻 vmchange -barcode CYM100D -m CYM100
􀁺 Changing the Volume Pool
􀁻 vmchange -m MEDIA_ID -p POOL#


To expire media
􀁺 bpexpdate -ev -d 0 -force -host

To unfreeze media
􀁺 List the frozen media
􀁻 /usr/openv/netbackup/bin/goodies/available_media grep -i FROZEN
􀁺 Unfreeze the media
􀁻 bpmedia -unfreeze -ev -h

To relabel a tape
􀁺 bplabel -ev -d -p
􀁺 bplabel -ev 000687 -d dlt -p TriVrgt_OFFSITE

To remove media from the Netbackup database
􀁺 Verify that there are no images on the tape
􀁻 bpimmedia -mediaid 000687 -L
􀁺 Expire the tape
􀁻 bpexpdate -ev 000687 -d 0 -host scorpius -force
􀁺 Get the status and pool number of the tape
􀁻 vmquery -m 000687
􀁺 Deassign the tape
􀁻 vmquery -deassignbyid
􀁻 vmquery -deassignbyid 000687 4 0x0
􀁺 Delete the tape
􀁻 vmdelete -m 000687


Installing the Netbackup Client
􀁺 /update_clients -ForceInstall -ClientList /tmp/clients.lst
􀁻 requires that TMPDIR and TEMPDIR be set correctly

Excludng files from backup on a client
􀁺 Create /usr/openv/netbackup/exclude_list
􀁺 Put the file specifications of the files/directories to be excluded
􀁻 /mnt/directory/*


Displaying Information about a Tape
􀁺 vmquery -m --> Displays attributes about a particular tape
􀁺 bpmedialist -U -mcontents -ev 000687 --> Displays media contents
􀁺 bpmedialist -U -mlist --> List of all media
􀁺 bpmedialist -U -mlist -ev CYM966 --> Listing of a particular media id
􀁺 bpimmedia -mediaid 000687 -L --> Listing of images on a tape


Robtest Commands
􀁺 Starting robtest
􀁻 robtest
􀁻 1 --> to select TLD 0
􀁺 Getting help
􀁻 ?
􀁺 Looking at contents of the tape drives
􀁻 s d
􀁺 Looking at the contents of the library
􀁻 s s
􀁺 Moving a tape from a drive to a library slot
􀁻 s d --> to identify drive number that has tape (Contains Cartridge = yes,
Barcode=XXXXXX)
􀁻 s s --> to identify an empty slot in the tape library (Netbackup will need to be reinventoried)
􀁻 m d# s# --> from from drive # to slot #
􀁻 s d --> verify the tape drive is empty
􀁻 s s --> verify the library slot has the tape


Configuration Files
/usr/openv/netbackup/bp.conf

􀁺 configuration file, sets backup server and backup clients
􀁺 force statement must be correct
􀁺 client to browse from
􀁺 client to restore to

/usr/openv/volmgr/vmconf


Logfiles


To utilize logfiles, create the corresponding directory in /usr/openv/netbackup/logs


Server Logfile directories:
􀁺 admin - adminstrative commands
􀁺 bpbrm - backup and restore manager
􀁺 bpcd - client daemon
􀁺 bpdbjobs - database manager program process
􀁺 bpdm - disk manager process
􀁺 bpjava-msvc - Java application server authentication service
􀁺 bpjava-usvc - process that services Java requests
􀁺 bprd - request daemon process
􀁺 bpsched - scheduler process that runs on master servers
􀁺 bptm - tape/optical media management process
􀁺 user-ops - required directory for use by Java programs
􀁺 xbpadm - X based administration utility
􀁺 xbpmon - X based job monitor process

Client Logfile directories:
􀁺 bp - client user interface process
􀁺 bparchive - archive program
􀁺 bpbackup - backup program
􀁺 bpbkar - program that generates golden images
􀁺 bpcd - client daemon
􀁺 bpjava-msvc - Java application server authentication service
􀁺 bpjava-usvc - process that services Java requests
􀁺 bplist - program that lists backed up and archived files
􀁺 bpmount - program that determines local mountpoints and wildcard expansion for multiple
streams
􀁺 bphdb - Oracle database backup program start process
􀁺 db_log - database specific extension log
􀁺 tar - tar process log during restores
􀁺 user_ops


Media Manager logging automatically goes to the system log using syslogd logging facility

.Logging will only occur if these directories are created. These directories will generate a lot of data and should be deleted when no longer necessary.

To increase the amount of logging information set VERBOSE=2 in /usr/open/netbackup/bp.conf
(default is VERBOSE=1)


Processes
ltid
acsd
vmd


Useful Commands

bpcllist - list classes
bpclinfo -L --> displays info about a class
vmpool - volume pools
vmpool -listall
vmpool -listscratch
bplabel -ev -d hcart
bpbackup db --> backs up the catalog
bpclclients --> lists the clients for a particular policy (class)


Troubleshooting
bperror -statuscode <-- displays information about the netbackup error. No Backups are running:

􀁺 Check system log file for error messages
􀁺 Stop and restart all the netbackup processes
􀁺 Look for a downed drive
􀁻 /usr/openv/volmgr/bin/vmoprcmd
􀁻 /usr/openv/volmgr/bin/vmoprcmd -up 0 --> this will bring up drive 0 if it's control shows
as down
􀁺 Look for pending requests
􀁻 /usr/openv/volmgr/bin/vmoprcmd or gui --> device management
􀁻 If there is a pending request either re-assign it to a drive, or deny the request


Downed drive does not come back up or does not stay up
􀁺 Check for a hardware problem by looking for messages on the tape library
􀁺 Make sure there is not a tape stuck in the drive
􀁻 Use robtest (described above) to look at the drives
􀂄 If there is a tape stuck in the drive, try to remove it using robtest
􀂄 If robtest fails, then you must manually remove it.


Verify the Client is communicating properly:
􀁺 bpclncmd -ip --> from both client and server
􀁺 bpclntcmd -hn --> from both client and server
􀁺 bpclntcmd -pn --> from client only


Device Actions
Device Management --> info about tape drives
􀁺 dlt
􀁺 hcart (ultrium)


Media Actions


Media id must agree with # of the tape
Create a media id
􀁺 actions -->new-->single volume-->dlt cart (not dlt2)
􀁺 put it into the "netbackup" volume pool

Netbackup Client


To check things out do this:
It could be a couple things. Mostly DNS, bp.conf, or something stupid. On the client run this command
/usr/openv/netbackup/bin/bpclntcmd -pn
/usr/openv/netbackup/bin/bpclntcmd -server "server name"
/usr/openv/netbackup/bin/bpclntcmd ip "ip_address"

One of these usually fails and your able to fix it right off
1074 ./bpclntcmd -hn corpbu1
1075 ./bpclntcmd -ip 10.194.1.129
1076 ping 10.194.1.129
1077 ./bpclntcmd -hn corpldv1
1078 ./bpclntcmd -hn corpbu1.corporate.vox.net
1079 ping corpldv1
1080 ./bpclntcmd -ip 10.194.1.120

Must be able to resolve correctly from the master server and the client or it will not work!!!


Veritas NetBackup Performance Tuning Tips

Is your Veritas (Symantec, now) NetBackup Server agonizingly slow in taking backups? I found this undocumented feature on Veritas’ website, and quadrupled my backup speeds. Without going into too much details, I’ll give a brief on what was happening. From a local hard drive on a Linux box, backup onto a tape library was running at 16 MB/sec.

That’s slower than a snail with arthritis. Turns out, NetBackup uses certain default values for shared memory buffer sizes (details on Veritas’ website). The values happen to be 8 x 32KB of shared mem buffers.

To increase the size of the buffers and the number of buffers, you have to create the following two files:

/usr/openv/netbackup/db/config/SIZE_DATA_BUFFERS
/usr/openv/netbackup/db/config/NUMBER_DATA_BUFFERS


Simply fill in the values that you need for each. A good number for SIZE_DATA_BUFFERS would be 131072 (128KB), or, if your tape drive supports it, 262144. You can also choose to have 16 such buffers by typing in this number in the NUMBER_DATA_BUFFERS file. Note that you may have to increase the shared mem limits on your system before doing this.

This is applicable to a whole lot of previous versions of NetBackup starting from 3.4, and it works with the latest 6.0 version as well.