Thursday, December 26, 2013

Creating backups of your home folder

I don't think I have to tell you that creating backups is necessary. You never know how and when disaster will strike. To prevent loss of data I have been using the following strategy for a while:
  • create a backup on a weekly basis to 2 disks on-site
  • switch one of the on-site disks with an off-site disk (stored in a safe in a bank)
Having an off-site disk is important. It helps you recovering from a real disaster like fire or theft. You could rent some online cloud storage to facilitate your off-site backups, but I think my solution (given the amount of data) is cheaper and faster. To create my weekly backup, I use the following script:
#!/bin/sh
# Author: Brice Burgess - bhb@iceburg.net
# multi_backup.sh -- backup to a local drive using rsync.
#         Uses hard-link rotation to keep multiple backups.

# Directories to backup. Seperate with a space. Exclude trailing slash!
SOURCES="/home/kenneth"

# Directory to backup to. This is where your backup(s) will be stored. No spaces in names!
# :: NOTICE :: -> Make sure this directory is empty or contains ONLY backups created by
#	                        this script and NOTHING else. Exclude trailing slash!
#TARGET="/media/backup/Folders"
TARGET="/run/media/kenneth/backup/Folders"

# Set the number of backups to keep (greater than 1). Ensure you have adaquate space.
ROTATIONS=3

# Your EXCLUDE_FILE tells rsync what NOT to backup. Leave it unchanged if you want
# to backup all files in your SOURCES. If performing a FULL SYSTEM BACKUP, ie.
# Your SOURCES is set to "/", you will need to make use of EXCLUDE_FILE.
# The file should contain directories and filenames, one per line.
# A good example would be:
# /proc
# /tmp
# *.SOME_KIND_OF_FILE
EXCLUDE_FILE="/path/to/your/exclude_file.txt"

# Comment out the following line to disable verbose output
VERBOSE="-v"

#######################################
########DO_NOT_EDIT_BELOW_THIS_POINT#########
#######################################

# Set name (date) of backup.
BACKUP_DATE="`date +%F_%H-%M`"

if [ ! -x $TARGET ]; then
  echo "Backup target does not exist or you don't have permission!"
  echo "Exiting..."
  exit 2
fi

if [ ! $ROTATIONS -gt 1 ]; then
  echo "You must set ROTATIONS to a number greater than 1!"
  echo "Exiting..."
  exit 2
fi

#### BEGIN ROTATION SECTION ####

BACKUP_NUMBER=1
# incrementor used to determine current number of backups

# list all backups in reverse (newest first) order, set name of oldest backup to $backup
# if the retention number has been reached.
for backup in `ls -dXr $TARGET/*/`; do
  if [ $BACKUP_NUMBER -eq 1 ]; then
    NEWEST_BACKUP="$backup"
  fi

  if [ $BACKUP_NUMBER -eq $ROTATIONS ]; then
    OLDEST_BACKUP="$backup"
    break
  fi

  BACKUP_NUMBER=`/usr/bin/expr $BACKUP_NUMBER + 1`
done

# Check if $OLDEST_BACKUP has been found. If so, rotate. If not, create new directory for this backup.
if [ $OLDEST_BACKUP ]; then
  # Set oldest backup to current one
  mv $OLDEST_BACKUP $TARGET/$BACKUP_DATE
else
  mkdir $TARGET/$BACKUP_DATE
fi

# Update current backup using hard links from the most recent backup
if [ $NEWEST_BACKUP ]; then
  cp -al $NEWEST_BACKUP. $TARGET/$BACKUP_DATE
fi
#### END ROTATION SECTION ####


# Check to see if rotation section created backup destination directory
if [ ! -d $TARGET/$BACKUP_DATE ]; then
  echo "Backup destination not available. Make sure you have write permission in TARGET!"
  echo "Exiting..."
  exit 2
fi

echo "Verifying Sources..."
for source in $SOURCES; do
  echo "Checking $source..."
  if [ ! -x $source ]; then
    echo "Error with $source!"
    echo "Directory either does not exist, or you do not have proper permissions."
    exit 2
  fi
done

if [ -f $EXCLUDE_FILE ]; then
  EXCLUDE="--exclude-from=$EXCLUDE_FILE"
fi

echo "Sources verified. Running rsync..."
for source in $SOURCES; do

  # Create directories in $TARGET to mimick source directory hiearchy
  if [ ! -d $TARGET/$BACKUP_DATE/$source ]; then
    mkdir -p $TARGET/$BACKUP_DATE/$source
  fi

  rsync $VERBOSE --exclude=$TARGET/ $EXCLUDE -a --delete $source/ $TARGET/$BACKUP_DATE/$source/

done

exit 0
I did not create this script, so I don't take any credits for this. I just wanted to share this, since it has been serving my needs for a couple of years now and having a decent script is crucial in your backup strategy. If creating a backup requires you to type in a lot of difficult to remember commands, chances are you'll give up after a few weeks. So please, copy-paste the above script in a file called backup.sh, make it executable (chmod +x ./backup.sh), put it in your home folder (so the backup script is also being backed up :) ) and execute:
cd
./backup.sh
on a weekly basis. That's it.

openSUSE 13.1

2 years ago, I finally decided to ditch Ubuntu (kubuntu actually) and give openSUSE a try. Since that day, it has been running fine and stable without any problems. 12.1 was getting old, however, and I thought the other weekend was the perfect timing to re-install my main computer with the latest-and-greatest openSUSE 13.1.

Like with 12.1, installation went like a breeze. As with 12.1, I downloaded the live DVD and installed from there. It was only after the first reboot, network was not available (it was available in the live DVD) and I had to enable networking manually in YaST, not sure why that happened. YaST, by the way, has been rewritten entirely in Ruby to allow better development for new modules. The previous version of YaST was written in YCP which was a very inflexible language from what I read. Anyway, as a normal user, you probably won't see any differences.

Since I have an NVIDIA card, I downloaded and installed the latest drivers (331.20 at the time of writing) manually. I guess you could use a more automated way to do this, but since I have been compiling NVIDIA drivers for a while now (even in Ubuntu), I don't mind. This is where the problems started. Although this driver seems to work fine, it somehow makes running virtual machines in VMware's player impossible. I have found a thread in openSUSE's forum that confirms this problem and provides a solution. You need to rollback the driver to an older version (in this case 325.15) to allow your virtual machines to run. This problem is also described here. I guess this will be solved in a more recent version of the driver.
The older version of the nvidia driver (325.15) has a tearing problem, however, which means horizontal lines can appear while playing videos cutting the images in pieces. This is not a dramatic problem, but is annoying enough to start google'ing for a solution. In my case, the following command solves the problem:

nvidia-settings --assign CurrentMetaMode="DFP-0: 1920x1080 { ForceCompositionPipeline = On }"
I'm not sure how to make the above command "permanent" yet, but for now, it does the trick and I don't reboot very often.

I am using an ldap for centralized user management. YaST has a module (called "ldap client") which makes it easy to use this ldap for authentication and authorization in openSUSE. In 12.1 this worked without any problems, in 13.1 it was a little more difficult to get it going. In 13.1, the authentication and authorization is backed by sssd which requires the ldap to support the more secure ldaps:// protocol. Since my ldap setup at home is very old and doesn't need to be very secure (after all, it's the "home" network), it didn't support the ldaps:// protocol yet. After some tinkering my ldap supported LDAPS and was ready for sssd :)

The same server is also running samba in a "domain mode" setup. This server serves as a domain controller for my virtual Windows 7 and my old HP computer, also running Windows 7 now. It also serves as a domain controller for samba running on my openSUSE box. So installing samba was next. This is where I encountered the first real openSUSE bug. Since openSUSE is using systemd (instead of the older System V) and a tmpfs to store volatile data (such as /var/run) it relies on "some" kind of configuration to get these volatile folders populated. Unfortunately /var/run/samba is never created which prevents smbd and nmbd from starting. I don't use samba very often, so I only noticed samba was not started after discovering dolphin was reacting very slow. Dolphin took 30 seconds or so to start and navigating through folders also took several seconds. Starting dolphin from the console, revealed there was a problem communicating with the samba service hence a fix was needed. A solution was also found on the openSUSE forum.
systemd allows a Linux system to start very quickly, since it allows to start most of the services in parallel, instead of sequential. My Linux knowledge dates back from the time everything was still using System V. Everything was simple back then, systemd is more complicated and requires some studying :)

As already mentioned in a previous post, I am a long-time f-spot user. Since my collection of photo's has reached 10.000+, f-spot started to crash every now and then. Since it is pretty much dead (activity on the mailing list is almost "zero") and alternatives are available, I decided to migrate my collection to digikam. I can't compare the set of features provided by f-spot with the features available in digikam, since the only reason I use this software is to "tag" my photo's. digikam also has a flickr plugin, which makes it easy to upload photo's to my flickr account. Neat. I haven't seen digikam crashing yet, so I guess it can handle my collection better than f-spot did.
I'm not even sure you can run f-spot on openSUSE 13.1 because last time I tried on a virtual openSUSE running a 13.1 RC, it didn't work. Anyway, f-spot is another gtk-ish application down the drain :)

I also read half a dozen things to do after installing openSUSE 13.1 to solve the usual codec and GTK related issues. Also, apply what is said in one of the comments:

For LibreOffice it is actually "gtk3-theme-oxygen" not "gtk3-theme-engine" and you need to install "libreoffice-gnome" to make it use Gtk.
otherwise Libre Office will just remain plain ugly. The only application for which I could not resolve its "uglyness" is Apache Directory Studio. This is based on Eclipse 3.x which crashes upon startup. You can find more on this problem here. You have to add this property to your environment
GTK2_RC_FILES=/usr/share/themes/Raleigh/gtk-2.0/gtkrc
to make it work. Looks very oldschool now, but at least it doesn't crash anymore and I don't need Apache Directory Studio very often anyway.

So after some issues, I have a shiny new installation of openSUSE running. I've encountered some problems while setting up everything, but I guess this is more related to the specific setup I have (ldap for centralized user management, samba, ...) than it is related to general issues with Linux or openSUSE.