Recently I fooled around with GlusterFS, a distributed file system especially intended for use as backend storage. The first impression is quite good – as long as you use it for the right task.

Background

GlusterFS is a distributed file system which is supposed to scale to large storage sizes. Besides file distribution it also offers “RAID” like features: if you have two GlusterFS servers you can either stripe the data on both of them, or mirror them. Or, if you got more servers, you can even create more complex setups with a mixture of striping and mirroring. The client protocol is very similar to NFS, and thus clients can GlusterFS servers via GlusterFS-Fuse or directly via NFS.

The advantage of a distributed file system is obvious: if one server goes down, the clients fail over to another server, and you still have the data. Also, you can add more servers in case you need more storage. A distinct advantage of GlusterFS compared to for example AFS or Ceph is that you do not need to worry about setting up special servers dedicated to processes like addressing the data. Each GlusterFS server has the same rights and tasks as the next (in simple setups, highly complex setups may vary.

Competition

GlusterFS faces competition first of all in its own field: distributed file systems as mentioned above (Ceph, AFS, etc.). But for simpler setups you can also create similar results with a shared block level device like DRBD and and NFS or even cluster file systems like GFS and OCFS. And in the end GlusterFS steps up to fight against high cost storage systems (SAN, …).

The Setup

For my tests of GlusterFS I used a couple of CentOS 6 VMs in KVM on my main machine. I followed the well written GlusterFS-CentOS-Howto by falco. However, one thing which is probably not outlined enough in the howto: GlusterFS really depends on host name resolution. And really depending means that an entry in /etc/hosts on the GlusterFS machines is not enough. You should have a working DNS!

That said, the easiest way to accomplish this with a running KVM test setup is to just add the necessary entries to /etc/hosts on the host side (!). These information are forwarded to the VM guests and provide a reliable name resolution.

Tests and Results

I run a couple of tests: shutting down a server, shutting down the interfaces, shutting down the GlusterFS services, etc. The data on the machines were a couple of various text files, nothing fancy.

And all in all, I must say I was impressed with the features and capabilities: the setup was reliable, files where automatically distributed, servers started offline where resynced automatically, and so on. The collision detection is pretty neat, it works on file level: even if there occurs a split brain, i/o is only blocked for files which are actually affected. This is – of course and by design – a huge advantage over shared block devices which block the access to the entire …read more
Source: FULL ARTICLE at Planet KDE