Friday, December 31, 2010

Restoring files manually from a mondo backup

As already mentioned in earlier posts, I uses mondo to create system backups of all my systems. By system backups I mean; create a backup of the live system, without any personal data. In short, this is a backup of everything except home folders. The home folders are backed up using rsync and external disks, but this is another story.
Restoring from a mondo backup is easy; simply burn the images, put it in a cd drive and nuke the system (yes, this is a restore option in mondo :) ).
If you just want to restore some files from the backup, without overwriting the whole system, there's an easy way to do this. Suppose you created an ISO a while ago and these are stored on some disk. First, you will need to mount the ISO as a loopback device like so:

mount -o loop MyIso.iso /mnt

The ISO is now mounted in /mnt.
Now locate the file you want to restore (I was going to restore smb.conf):

$ grep smb.conf /mnt/archives/filelist.*
/mnt/archives/filelist.11:/var/lib/ucf/cache/:etc:samba:smb.conf
/mnt/archives/filelist.36:/usr/share/man/man5/smb.conf.5.gz
/mnt/archives/filelist.43:/usr/share/doc/samba-doc/htmldocs/manpages/smb.conf.5.html
/mnt/archives/filelist.44:/usr/share/doc/samba-doc/examples/dce-dfs/smb.conf
/mnt/archives/filelist.44:/usr/share/doc/samba-doc/examples/smb.conf.default.gz
/mnt/archives/filelist.44:/usr/share/doc/samba-doc/examples/tridge/smb.conf
/mnt/archives/filelist.44:/usr/share/doc/samba-doc/examples/tridge/smb.conf.fjall
/mnt/archives/filelist.44:/usr/share/doc/samba-doc/examples/tridge/smb.conf.lapland
/mnt/archives/filelist.44:/usr/share/doc/samba-doc/examples/tridge/smb.conf.vittjokk
/mnt/archives/filelist.44:/usr/share/doc/samba-doc/examples/tridge/smb.conf.WinNT
/mnt/archives/filelist.46:/usr/share/doc/smbldap-tools/examples/smb.conf
/mnt/archives/filelist.48:/usr/share/samba/smb.conf
/mnt/archives/filelist.48:/usr/share/samba/smb.conf.dapper
/mnt/archives/filelist.48:/usr/share/samba/smb.conf.etch
/mnt/archives/filelist.48:/usr/share/samba/smb.conf.gutsy
/mnt/archives/filelist.5:/etc/samba/smb.conf
/mnt/archives/filelist.5:/etc/samba/smb.conf.ucf-dist
$

The file I needed was somewhere in an archive identified by number 5.
Now create some folder in a temporary location:

$ mkdir /tmp/restore
$ cd /tmp/restore

Now restore the contents of the archive you want (you might need to download and install afio):

afio -ZP bzip2 -i /mnt/archives/5.afio.bz2

The whole archive is now restored underneath /tmp/restore. So my smb.conf is now located underneath /tmp/restore/etc/samba/smb.conf. Pretty cool eh.

Anyway, I don't deserve any credits for this post, I found a lot of help here.

EDIT: Apparently there is an easier, more graphical way to restore stuff from a mondo archive.

Thursday, December 30, 2010

Failed to issue the StartTLS instruction: Protocol error

I encountered the error mentioned in the title of this post after upgrading my samba install on Debian Lenny using Enterprise Samba binaries. The latest version they distribute at the time of writing is 3.4.9. My samba install talks to an ldap backend and the above error was shown upon starting the new version. Seems they added (or changed the default) option for the ldap protocol in smb.conf. Adding:

ldap ssl = off
makes the error go away.
Cool.

Tuesday, December 28, 2010

Building queries ... the easy way

On a past project, users needed to be able to create a custom query and execute it. To do this, the user was able to select a field, an operator and select or fill in a value. If, for example, we were searching for people living in Belgium, the user would select person.address.country for the field, like for the operator and fill in "BE" for the value. I'm sure you all know how the resulting SQL would look like.

There are different ways to create a query builder. If you're using Hibernate, the above example could easily be translated to HQL using the criteria API. In some cases, however, the resulting SQL is not exactly what you wanted (e.g. not performing well) or maybe Hibernate is just not capable generating a correct SQL. Unfortunately, at that time, we discovered we were suffering both aforementioned problems, so the only solution was to create our own query builders using string concatenation. Yes, this can get ugly really fast, when not being careful, but I managed to create something that works and looks nice ... more or less.

On a more recent project, users were also capable of creating custom queries. Instead of re-using parts of the query builders written in the past, I decided to ask Google for help. Surely there had to be someone or some project out there that came across the same issues I was having with Hibernates criteria API. During my search, I came across querydsl which seemed to do what I wanted to accomplish.

querydsl comes in different "flavours". You can use it to query your database using HQL or SQL. You can even use it to query objects in a simple collection (similar to lambdaj I guess). To do this, querydsl uses so called "Q" entities. So, if I'm querying a person and its addresses, I would have a QPerson and a QAddress. These entities can be generated from your JPA annotated domain model, or can be reverse engineered from your database. Of course, you can generate these "Q" entities by hand, since it's a very clean and easy to understand API. Once you have your "Q" entities in place, you can start querying the database in a very fluent API, like this: query.from(person).join(address).on(person.id.eq(address.person)).where(address.country.like("BE")). This API is also type-safe (HQL or criteria aren't), so you won't be querying a person's name using it's birth date :)

The only remaining difficulty was to translate the criteria sent by the user interface to something understood by querydsl. To do this, I created something that "observes" the creation of the "Q" entities. When a QPerson has a field "firstName", the observer knows this. The observer also knows there is a relation between the QPerson and the QAddress entities expressed by the relation person.address. In the end, the observer holds a complete mapping between the fields available in my "Q" entities and their actual names (e.g.: StringPath("FIRST_NAME") maps to person.firstName). Once you understand the insides of querydsl, this isn't very hard to create. Finally, you just need to merge this mapping with the criteria sent by the user interface and your query is ready to be executed against the database.

I wish I could publish the code written on this project, but the nature of the project doesn't allow me to do so. In this post, I just wanted to tell you something about querydsl and how easy it is to plug this into an existing project. It's also very extendible and has a very active community and core developer. Should you have any questions on this, please leave a comment.

Saturday, December 11, 2010

Updating the location for your photo's in f-spot

I've been using f-spot for quite some time now to manage and categorise all my pictures. In older Ubuntu versions (8.04 and earlier), the default location for f-spot to store its photo's was: /home/username/Photos. For some reason, Ubuntu 10.04 (probably earlier versions as well) changed this folder to: /home/username/Pictures/Photos. I didn't notice this until today, while I was importing photo's and checking the contents of /home/username/Photos only to see nothing was copied to this location. That's when I discovered they changed the default location :)
Since f-spot is backed by an sqlite database, this wasn't very hard to solve:
  • create a backup of the sqlite database photos.db (should be underneath /home/username/.config/f-spot)
  • now update the location with sqlite3:

    kenneth@pavane:/data/home/kenneth/.config/f-spot$ sqlite3 photos.db
    SQLite version 3.6.22
    Enter ".help" for instructions
    Enter SQL statements terminated with a ";"
    sqlite> update photos set base_uri=replace(base_uri, 'file:///home/kenneth/Pictures', 'file:///home/kenneth');
    sqlite> update photo_versions set base_uri=replace(base_uri, 'file:///home/kenneth/Pictures', 'file:///home/kenneth');
    .exit
  • now move your pictures to their correct location

Phew, solved.