Thursday, December 26, 2013

Creating backups of your home folder

I don't think I have to tell you that creating backups is necessary. You never know how and when disaster will strike. To prevent loss of data I have been using the following strategy for a while:
  • create a backup on a weekly basis to 2 disks on-site
  • switch one of the on-site disks with an off-site disk (stored in a safe in a bank)
Having an off-site disk is important. It helps you recovering from a real disaster like fire or theft. You could rent some online cloud storage to facilitate your off-site backups, but I think my solution (given the amount of data) is cheaper and faster. To create my weekly backup, I use the following script:
# Author: Brice Burgess -
# -- backup to a local drive using rsync.
#         Uses hard-link rotation to keep multiple backups.

# Directories to backup. Seperate with a space. Exclude trailing slash!

# Directory to backup to. This is where your backup(s) will be stored. No spaces in names!
# :: NOTICE :: -> Make sure this directory is empty or contains ONLY backups created by
#	                        this script and NOTHING else. Exclude trailing slash!

# Set the number of backups to keep (greater than 1). Ensure you have adaquate space.

# Your EXCLUDE_FILE tells rsync what NOT to backup. Leave it unchanged if you want
# to backup all files in your SOURCES. If performing a FULL SYSTEM BACKUP, ie.
# Your SOURCES is set to "/", you will need to make use of EXCLUDE_FILE.
# The file should contain directories and filenames, one per line.
# A good example would be:
# /proc
# /tmp

# Comment out the following line to disable verbose output


# Set name (date) of backup.
BACKUP_DATE="`date +%F_%H-%M`"

if [ ! -x $TARGET ]; then
  echo "Backup target does not exist or you don't have permission!"
  echo "Exiting..."
  exit 2

if [ ! $ROTATIONS -gt 1 ]; then
  echo "You must set ROTATIONS to a number greater than 1!"
  echo "Exiting..."
  exit 2


# incrementor used to determine current number of backups

# list all backups in reverse (newest first) order, set name of oldest backup to $backup
# if the retention number has been reached.
for backup in `ls -dXr $TARGET/*/`; do
  if [ $BACKUP_NUMBER -eq 1 ]; then

  if [ $BACKUP_NUMBER -eq $ROTATIONS ]; then

  BACKUP_NUMBER=`/usr/bin/expr $BACKUP_NUMBER + 1`

# Check if $OLDEST_BACKUP has been found. If so, rotate. If not, create new directory for this backup.
if [ $OLDEST_BACKUP ]; then
  # Set oldest backup to current one

# Update current backup using hard links from the most recent backup
if [ $NEWEST_BACKUP ]; then

# Check to see if rotation section created backup destination directory
if [ ! -d $TARGET/$BACKUP_DATE ]; then
  echo "Backup destination not available. Make sure you have write permission in TARGET!"
  echo "Exiting..."
  exit 2

echo "Verifying Sources..."
for source in $SOURCES; do
  echo "Checking $source..."
  if [ ! -x $source ]; then
    echo "Error with $source!"
    echo "Directory either does not exist, or you do not have proper permissions."
    exit 2

if [ -f $EXCLUDE_FILE ]; then

echo "Sources verified. Running rsync..."
for source in $SOURCES; do

  # Create directories in $TARGET to mimick source directory hiearchy
  if [ ! -d $TARGET/$BACKUP_DATE/$source ]; then
    mkdir -p $TARGET/$BACKUP_DATE/$source

  rsync $VERBOSE --exclude=$TARGET/ $EXCLUDE -a --delete $source/ $TARGET/$BACKUP_DATE/$source/


exit 0
I did not create this script, so I don't take any credits for this. I just wanted to share this, since it has been serving my needs for a couple of years now and having a decent script is crucial in your backup strategy. If creating a backup requires you to type in a lot of difficult to remember commands, chances are you'll give up after a few weeks. So please, copy-paste the above script in a file called, make it executable (chmod +x ./, put it in your home folder (so the backup script is also being backed up :) ) and execute:
on a weekly basis. That's it.

openSUSE 13.1

2 years ago, I finally decided to ditch Ubuntu (kubuntu actually) and give openSUSE a try. Since that day, it has been running fine and stable without any problems. 12.1 was getting old, however, and I thought the other weekend was the perfect timing to re-install my main computer with the latest-and-greatest openSUSE 13.1.

Like with 12.1, installation went like a breeze. As with 12.1, I downloaded the live DVD and installed from there. It was only after the first reboot, network was not available (it was available in the live DVD) and I had to enable networking manually in YaST, not sure why that happened. YaST, by the way, has been rewritten entirely in Ruby to allow better development for new modules. The previous version of YaST was written in YCP which was a very inflexible language from what I read. Anyway, as a normal user, you probably won't see any differences.

Since I have an NVIDIA card, I downloaded and installed the latest drivers (331.20 at the time of writing) manually. I guess you could use a more automated way to do this, but since I have been compiling NVIDIA drivers for a while now (even in Ubuntu), I don't mind. This is where the problems started. Although this driver seems to work fine, it somehow makes running virtual machines in VMware's player impossible. I have found a thread in openSUSE's forum that confirms this problem and provides a solution. You need to rollback the driver to an older version (in this case 325.15) to allow your virtual machines to run. This problem is also described here. I guess this will be solved in a more recent version of the driver.
The older version of the nvidia driver (325.15) has a tearing problem, however, which means horizontal lines can appear while playing videos cutting the images in pieces. This is not a dramatic problem, but is annoying enough to start google'ing for a solution. In my case, the following command solves the problem:

nvidia-settings --assign CurrentMetaMode="DFP-0: 1920x1080 { ForceCompositionPipeline = On }"
I'm not sure how to make the above command "permanent" yet, but for now, it does the trick and I don't reboot very often.

I am using an ldap for centralized user management. YaST has a module (called "ldap client") which makes it easy to use this ldap for authentication and authorization in openSUSE. In 12.1 this worked without any problems, in 13.1 it was a little more difficult to get it going. In 13.1, the authentication and authorization is backed by sssd which requires the ldap to support the more secure ldaps:// protocol. Since my ldap setup at home is very old and doesn't need to be very secure (after all, it's the "home" network), it didn't support the ldaps:// protocol yet. After some tinkering my ldap supported LDAPS and was ready for sssd :)

The same server is also running samba in a "domain mode" setup. This server serves as a domain controller for my virtual Windows 7 and my old HP computer, also running Windows 7 now. It also serves as a domain controller for samba running on my openSUSE box. So installing samba was next. This is where I encountered the first real openSUSE bug. Since openSUSE is using systemd (instead of the older System V) and a tmpfs to store volatile data (such as /var/run) it relies on "some" kind of configuration to get these volatile folders populated. Unfortunately /var/run/samba is never created which prevents smbd and nmbd from starting. I don't use samba very often, so I only noticed samba was not started after discovering dolphin was reacting very slow. Dolphin took 30 seconds or so to start and navigating through folders also took several seconds. Starting dolphin from the console, revealed there was a problem communicating with the samba service hence a fix was needed. A solution was also found on the openSUSE forum.
systemd allows a Linux system to start very quickly, since it allows to start most of the services in parallel, instead of sequential. My Linux knowledge dates back from the time everything was still using System V. Everything was simple back then, systemd is more complicated and requires some studying :)

As already mentioned in a previous post, I am a long-time f-spot user. Since my collection of photo's has reached 10.000+, f-spot started to crash every now and then. Since it is pretty much dead (activity on the mailing list is almost "zero") and alternatives are available, I decided to migrate my collection to digikam. I can't compare the set of features provided by f-spot with the features available in digikam, since the only reason I use this software is to "tag" my photo's. digikam also has a flickr plugin, which makes it easy to upload photo's to my flickr account. Neat. I haven't seen digikam crashing yet, so I guess it can handle my collection better than f-spot did.
I'm not even sure you can run f-spot on openSUSE 13.1 because last time I tried on a virtual openSUSE running a 13.1 RC, it didn't work. Anyway, f-spot is another gtk-ish application down the drain :)

I also read half a dozen things to do after installing openSUSE 13.1 to solve the usual codec and GTK related issues. Also, apply what is said in one of the comments:

For LibreOffice it is actually "gtk3-theme-oxygen" not "gtk3-theme-engine" and you need to install "libreoffice-gnome" to make it use Gtk.
otherwise Libre Office will just remain plain ugly. The only application for which I could not resolve its "uglyness" is Apache Directory Studio. This is based on Eclipse 3.x which crashes upon startup. You can find more on this problem here. You have to add this property to your environment
to make it work. Looks very oldschool now, but at least it doesn't crash anymore and I don't need Apache Directory Studio very often anyway.

So after some issues, I have a shiny new installation of openSUSE running. I've encountered some problems while setting up everything, but I guess this is more related to the specific setup I have (ldap for centralized user management, samba, ...) than it is related to general issues with Linux or openSUSE.

Wednesday, May 30, 2012

f-spot dead?

I've been using f-spot for quite some time now to manage my photo collection. When looking at their homepage, I noticed the last version was released somewhere in 2010 (?). In software terms, this makes f-spot ancient technology :) so I started to wonder.

Browsing through their mailinglist, I came across this thread. In short, f-spot isn't dead yet, but it is not actively maintained anymore. This means that stability issues and bugs probably won't get addressed any time soon. Fortunately, someone wrote a script to migrate the f-spot database to digikam. Maybe I should give this a try ... I'll keep you posted.

Friday, April 27, 2012

Using the Qube as a NAS device 2

In an earlier post I forgot to mention that installing NetBSD using the Restore CD option will wipe out your whole drive. So, before I started, I disconnected the hard drive and installed everything on the CF card. A basic installation of NetBSD does not require a lot of space. The 1 GB CF card was big enough for a basic install, but, to be able to use pkgsrc, I installed a 4 GB card.

After I verified the installation was working properly, I decided to connect the hard drive again. This is where the problems started :(

VIA Technologies VT83C572 USB Controller (USB serial bus, revision 0x02) at 
pci0 dev 9 function 2 not configured
tlp1 at pci0 dev 12 function 0: DECchip 21143 Ethernet, pass 4.1
tlp1: interrupting at level 2
tlp1: Ethernet address 00:10:e0:00:3c:5d
lxtphy1 at tlp1 phy 1: LXT970 10/100 media interface, rev. 3
lxtphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
wd0 at atabus0 drive 0: 
wd0: 977 MB, 1986 cyl, 16 head, 63 sec, 512 bytes/sect x 2001888 sectors
wd1 at atabus0 drive 1: 
wd1: 298 GB, 620181 cyl, 16 head, 63 sec, 512 bytes/sect x 625142448 sectors 
Kernelized RAIDframe activated
viaide0:0:1: lost interrupt
       type: ata tc_bcount: 512 tc_skip: 0
viaide0:0:1: bus-master DMA error: missing interrupt, status=0x61
viaide0:0:1: device timeout, c_bcount=512, c_skip0
wd1d: device timeout reading fsbn 0 (wd1 bn 0; cn 0 tn 0 sn 0), retrying
viaide0 channel 0: reset failed for drive 0 drive 1
viaide0:0:1: lost interrupt
       type: ata tc_bcount: 512 tc_skip: 0
viaide0:0:1: bus-master DMA error: missing interrupt, status=0x61
viaide0:0:1: device timeout, c_bcount=512, c_skip0
wd1d: device timeout reading fsbn 0 (wd1 bn 0; cn 0 tn 0 sn 0), retrying
... (several minutes, same message)
viaide0:0:1: lost interrupt
       type: ata tc_bcount: 512 tc_skip: 0
viaide0:0:1: bus-master DMA error: missing interrupt, status=0x61
viaide0:0:1: device timeout, c_bcount=512, c_skip0
wd1d: device timeout reading fsbn 1 (wd1 bn 1; cn 0 tn 0 sn 1), retrying
viaide0:0:1: lost interrupt
       type: ata tc_bcount: 512 tc_skip: 0
viaide0:0:1: bus-master DMA error: missing interrupt, status=0x61
viaide0:0:1: device timeout, c_bcount=512, c_skip0
wd1d: device timeout reading fsbn 1 (wd1 bn 1; cn 0 tn 0 sn 1), retrying
wd1: soft error (corrected)
wd1: no disk label
boot device: wd0
Hmmm, strange. This was definitely not a hardware problem, since the disk is fairly new and was working without any issues in Debian Lenny and Squeeze.

I suspected some kind of DMA problem, so I asked the friendly (yes, they are very friendly, not the RTFM kind) people for help. They suggested to recompile the kernel, and make it more suitable for running on my Qube. The following procedure describes how to do this on a recent Linux box (tried on openSUSE 12.1):

Go somewhere on the Linux machine, and do this:

mkdir src tools obj
cvs -d checkout -PA  -rnetbsd-5 src
This will get you the latest 5.x source tree.
Go to src/sys/arch/cobalt/conf/
Copy GENERIC to some other name, edit it and save it.
Go back to src
Now, create tools ./ -m cobalt tools
Now, build the kernel ./ -m cobalt -T ./tooldir.Linux-3.1.9-1.4-desktop-x86_64 -O ../obj kernel=MyKernel
After a while it will end up in the obj/sys/.../netbsd
Put the kernel (netbsd) in / on the Qube, and rename the old kernel netbsd.old You can choose which kernel is booted at the Qube's boot prompt, by simply typing its name (if you're connected to the serial console).

Disabling DMA in the kernel config (GENERIC) was fairly easy. Locate the following lines:

# IDE drives
# Flags are used only with controllers that support DMA operations
# and mode settings (e.g. some pciide controllers)
# The lowest order four bits (rightmost digit) of the flags define the PIO
# mode to use, the next set of four bits the DMA mode and the third set the
# UltraDMA mode. For each set of four bits, the 3 lower bits define the mode
# to use, and the last bit must be 1 for this setting to be used.
# For DMA and UDMA, 0xf (1111) means 'disable'.
# 0x0fac means 'use PIO mode 4, DMA mode 2, disable UltraDMA'.
# (0xc=1100, 0xa=1010, 0xf=1111)
# 0x0000 means "use whatever the drive claims to support".
wd*             at atabus? drive ? flags 0x0000
Now, do what the documentation says and put 0xf to disable DMA:
wd*             at atabus? drive ? flags 0x0ff0
After compiling the kernel using the above procedure, the Qube boots just fine. What's more important; I can now actually use the machine as a NAS device. I already produced several backups using mondo on the Qube, without any problems.

Wednesday, April 25, 2012

Using the Qube as a NAS device 1

Some time ago I described my adventures in installing Debian (Lenny at that time) on an old Qube 2. It was cool to install it, but I didn't find a proper use for it. So it has been gathering dust for a while, until I decided to use it as a backup server for my desktop and wyse. So, this weekend I decided to fire up my favorite mondo script and create a backup for the wyse (before dist-upgrading it to squeeze).

Unfortunately this didn't work very well. The moment the network card in the Qube gets some load it starts to hang. After some time in the backup process, I couldn't ping the Qube any more. The serial console didn't respond either, so I think it died. Putting a newer network card in the box's free PCI slot did not solve my issues either. It seems that Debian (or maybe even Linux) is just not stable enough to cope with a lot of network traffic / load. So in short, Debian failed miserably turning the Qube in a decent (albeit slow) NAS.

After googling for a while, I found another valuable alternative to run on the Qube. Enter NetBSD.
Now, I've been using Linux for quite some time now and I know how to move around on the command line, but - I thought - NetBSD is a whole different league. Fortunately, NetBSD provides a very simple procedure to install the Qube using a so called Restore CD. The idea is simple:

  • find an old i386 machine that can boot from a CD-ROM;
  • connect the Qube with a cross-cable to the i386 machine;
  • boot the i386 machine with the Restore CD;
  • netboot the Qube
You will see that the Qube will netboot from the i386, which will allow you to start the install procedure. Just follow the instructions step by step and in the end, you will have a working Qube 2 running NetBSD.

Monday, November 21, 2011

openSuSE 12.1

I've been using Ubuntu for quite some time now. Ever since the issues I've encountered while upgrading from Hoary, to Breezy, to Dapper, I decided to stick with LTS (or Long Term Support) releases. I don't care about using bleeding edge technology, I just want a PC that works.
Being a first hour kubuntu with KDE4 user, I was a little bit displeased with the latest LTS release, so the past years I've been looking at different other distributions to see which one supports KDE best. Distributions with rolling releases earned extra points :)
In 1998 SuSE 6.0 was released and I remember buying a big box containing 6 CDs in a local bookstore. I liked SuSE a lot. It was easy to install and maintain. The only downside was the RPM dependency hell you can end up with once you tried to install packages that were not available on the CDs. YAST already existed back then and was already doing a great job configuring the system (ncurses based, of course). Since openSuSE is very KDE oriented, I decided to give openSuSE 12.1 a try.
SuSE still uses YAST for most of its system configuration and YAST is still doing a great job. Integrating your system with an LDAP for fetching your users is a simple point and click operation (something that does not exist in kubuntu). As with kubuntu, using the proprietary nvidia drivers is a manual operation. So far, the only annoying thing I found in SuSE is its multimedia support. I had to include third party repositories to be able to install VLC and mplayer (or maybe I am doing it wrong). Burning plain audio CDs from K3B from mp3 is also not working yet, but I'll figure this out some day.
This doesn't mean I ditched Ubuntu, well kubuntu actually, completely. I just wanted to try something different, which is what Linux is all about :)

Tuesday, September 20, 2011

Fosdem 2012 ...

... the dates are set. February 4 & 5, 2012 ... yeah!

Monday, September 19, 2011

NK Concept Car, all pieces arrived ... finally

Today the final package containing 14 missing red lift arms arrived. I had to place 5 orders in 5 different stores, for a total of EUR 380 to get all the necessary parts for this car. Most parts (and the largest order) where found in Juergen's store. I am still missing the wheel covers. These have become very rare and very expensive. Since I think these are ugly, I might not order these anyway. Pictures coming soon!