Dungog-backup

From Dungog Wiki

Contents 1 Multiple Automated Backups 1.1 Overview 1.2 Configure Destinations 1.3 Configure Jobs 1.4 Help 1.4.1 Setup 1.4.2 Drive summary errors 1.4.3 Restore 1.4.4 SSH 1.4.5 CD and DVD writing 1.4.6 Rsync 1.4.7 USB or hard drive

Multiple Automated Backups

yum install dungog-backup

server-manager > dungog.net > Backup

Using either tar or rsync dungog-backup allows you to create multiple automated backup sets
to remote samba/windows or NFS 'shares'
to a 2nd drive or USB drive on the server
to a cd or dvd writer on the server
or to a remote account with ssh enabled

tgz files are limited in size only by the destination space
All of the Destinations you backup to are shown in a table with disk usage

After each backup admin is sent an email with the size of the backup and the remaining free space at the destination
eg. iran2 backup completed, dir 7.0M, free space 43G on //notebook/share
or CD backup failed, tar size 945 MB, cd is 540 MB

Overview

Configure Destinations

Configure Jobs

Help

Setup

• Safeguards

To ensure the destination is reachable and we do not fill up the servers /mnt directory we look for a plain text file called marker.txt in the root of the destination.

To prevent server resources being overused only one rsync and one tgz backup is permitted to run at once, this can be overridden with these db commands for tgz

db dungog setprop backup overlap allow

for rsync

db dungog setprop rsync overlap allow

Emails to admin warn of these and other errors, and advise of success, please check for them.

• Methods

tgz, allows you to save files of any size the operating system allow, however the integrity of the data after the file reaches 2 gb is unclear.

Rsync, copies all your files to a directory called $description it uses links and preserves permissions on ext3, on windows shares it is a simple copy.

RsyncNP, removes parent directories,does not insert the description in path, doesn't save copies, more below

rsyncDVD, copies files to /tmp/rsyncDVD, tests directory size is under the CD/DVD size, then burns the directory tree, more below

Rsync to ssh isn't implemented, use dungog-rsync.

• Copies

tgz, makes a copy of the backup file and appends a -1 for the first backup
Rsync, creates a copy in a new directory $description-1

• Backup file name

Use the file name smebackup.tgz or the entry you gave for the description.
Rsync creates a dir called $description

• How to partially backup

The config files should be regularly backed up

A useful method is do a SME default backup, then exclude your large ibays and possibly some directories of users so you can fit the backup on your destination, The excluded ibays and directories can be backed up separately, or they may not need backing up, eg an mp3 collection you have archived elsewhere.

If you are only doing a data backup you do not need the sme config files You can select exclude everything then add backup entries in the other directory section.

Default directories and files

the dir= and files= show what is being backed up

#default
{
  dirs =
    home/e-smith
    etc/e-smith/templates-custom
    etc/e-smith/templates-user-custom
    etc/ssh
    root
  files =
    etc/sudoers
    etc/passwd
    etc/shadow
    etc/group
    etc/gshadow
    etc/samba/secrets.tdb
    etc/samba/smbpasswd
}
#exclude config files
{
  files = none
  dirs  = home/e-smith/files/ibays
          home/e-smith/files/users
}
#exclude config, users
{
  files =  none
  dirs  =  home/e-smith/files/ibays
}
#exclude config, ibays
{
  files = none
  dirs  = home/e-smith/files/users
}
#exclude config, ibays, everything
{
  files = none
  dirs  = none
}

• Include/Exclude other directories

By entering the full path you can exclude or include other directories from the backup set, excluded directories are shown red.

If a user keeps bulk data or temporary files under a specific directory, this can be excluded while his valuable work is saved.

An included directory may be /usr/local/whizbang/config or /opt
eg. files not normally backed up.

Drive summary errors

When calculating the drive summary you will recieve offline as a standard reply when there has been a connection error. To track down the cause view the logs

tail -20 /var/log/messages

Processing event: calc-df

8890: Connection to ritmo failed

SMB connection failed

/bin/df: /mnt/dungogbackup/kiwi/backup: Permission denied

Accepted publickey for stephen from 192.168.35.15 port 32952 ssh2

ssh: connect to host a20.2321.net port 22: No route to host

mount.smbfs: entering daemon mode for service \kiwi\backup, pid=4673

these log entries show

1. smb connection failing because PC was turned off
2. user access denied, permissions
3. valid ssh connection
4. invalid ssh, pc was off
5. valid smb connection to kiwi

Restore

Restoring .tgz files

• restore all files

files are restored relative to current path use / for system restore or /anywhere/else otherwise

with ssh, from / dir of server to be restored ssh user@remote-server.net cat home/sample.tgz | tar -zxvf -

with share or nfs, mount share, from / dir of server to be restored tar -zxvf /mnt/share/sample.tgz

• copy file to server first from a windows share

from / dir of server to be restored

/usr/bin/smbclient //server/share password -U username [of windows share]

smb: \> get smebackup.tgz [or subdir\smebackup.tgz ]
smb: \> quit

or in one command

( echo get smebackup.tgz ) | ( /usr/bin/smbclient //server/share password -U username )

Note. Before and after the restore you will need to run some SME commands

signal-event pre-restore
-> restore files
signal-event post-upgrade
signal-event reboot

• selective restore

check existence and path

tar -ztf sample.tgz |grep netlogon
 home/e-smith/files/samba/netlogon/
 home/e-smith/files/samba/netlogon/netlogon.bat

extract a file tar -zxvf sample.tgz home/e-smith/files/samba/netlogon/netlogon.bat

• Restoring files from rsync

mount share and cp
verify permissions are correct before replacing system files
they won't be if you rsynced to a windows share

SSH

• Setting up ssh keys

To connect with ssh to the user on a remote server you need to place the public key of root for this server into ~/.ssh/authorized_keys of the remote user.

cd /root/.ssh
ssh-keygen -t rsa [accept file name id_rsa][don't enter a passphrase]
ssh user@remote-server.net mkdir .ssh
scp id_rsa.pub user@remote-server.net:.ssh/this-server.pub
ssh user@remote-server.net
cd .ssh
cat this-server.pub >> authorized_keys
[or scp id_rsa.pub user@remote-server.net:.ssh/authorized_keys] for first entry

now you should be able to "ssh user\@remote-server.net" without a passord prompt

• Checking disk space on remote server

it is assumed the partition the remote user is on is either /dev/sda3 or /dev/hda3 if it isn't you need to set a hidden setting

a. find the destination identifier used for dungog-backup, mouse over the modify link in the "Backup Drive Summary" table, the value is similar to drive123456

b. find the partition used on the remote server

ssh user@remote-server.net df -h

and take a guess or ask the server administrator if unsure

c. add the hidden setting

db dungog setprop drive123456 fstab /dev/hdb4

d. changing port to connect to on remote server if you may have changed the ssh port of the remote server. set the port for those servers with the following commands

run this once

db dungog set remoteport setting

then for each server

db dungog setprop remoteport mail.server2.com 2211

verify

db dungog show remoteport

remoteport=setting
   mail.server2.com=2211
   www.server3.com=2212

CD and DVD writing

• Setting up

nothing needs to be done on sme7 if you have problems, insert a CD/DVD with valid data and try

mount /dev/cdrom

it should return the warning below, but it will mount to /media/cdrecorder/

mount: block device /dev/hdb is write-protected, mounting read-only

• Burn methods

Data is written to /tmp before being burnt to cd or dvd
before burning it checks that your disk size is large enough
You also need the spare disk space

method 1. - tgz
creates a single .tgz file /tmp/cdbackup.tgz
this will preserve permissions

method 2. - rsyncDVD
Creates a directory /tmp/rsyncDVD then burns to CD or DVD

This allows you to restore individual files more easily and avoids any problem with file sizes larger than 2gb. However file permissions and ownership are lost.

On dvd's files or paths with two periods aren't recognised. eg. mail or files such as \opt\.program.data

On a dvd drive, rsync to a CD disk fails, we think a warning error is stopping the closing off of the cd ? [cd's on a CD Drive works]

Rsync

• rsync Options

you can set extra options with a db value, eg.

db dungog setprop bus92725 rsync --bwlimit=25

the bus????? value can be read by waving your mouse over the modify link in the backup panel check with

db dungog show bus92725

• rsyncNP

This option was added for a client with special needs

1. email notification reports size of backup as the size of all files in the share
2. marker.txt is not tested
3. excludes require an edit before they will be ignored

eg

pico -w /etc/dungog/excludelist2-bus92725

where 92725 is found waving the mouse of the modify link in the panel

to exclude these three dirs change from

/home/e-smith/files/ibays/Primary
/home/e-smith/files/ibays/backup
/home/e-smith/files/ibays/dungog

to

/ibays/Primary
/ibays/backup
/ibays/dungog

then don't re-save this back up set again or the above is recreated.

USB or hard drive

• Hard Drive

See this howto for adding a 2nd drive to SME, Thanks Michael http://www.contribs.org/contribs/mblotwijk/HowToGuides/AddExtraHardDisk.htm

If you only have one drive it would probably be better to use a 2nd drive in a RAID, http://wiki.contribs.org/Raid

• USB Drive

Setup your USB Drive with these instructions, http://wiki.contribs.org/USBDisks

Check your USB cable and port are high speed, ie test by copying data, say 1 GB in 2 minutes

Currently the backup routine expects the Drive to be mounted, the following will be added after testing

signal-event local