NFS versus POHMELFS

Just food for thought, and something to keep our eyes on.

Source: http://www.linux-mag.com/cache/7399/1.html

POHMELFS is a new file system that is designed to take a step beyond classic NFS focusing on improved performance. It’s name even contains the term “Parallel” that indicates that the clients interact with multiple servers. In particular, it has the ability to balance reading from multiple servers and also do simultaneous writes to different remote servers. It is designed in the classic server-client model. As with NFS, POHMELFS exports a directory from each server so it relies on an underlying file system to read and write data to the physical devices themselves. In addition, it is designed as an object based file system (more on that below).

The POHMELFS website has a very comprehensive list of features. For the sake of completeness these features are summarized here:

    * One of the most important attributes is the ability to write to multiple servers and balance reading between multiple servers.

    * POHMELFS has a local coherent cache for data and metadata (this basically adds some of the features of FS-Cache and CacheFS to the network file system.)

    * It includes locking (a necessary feature for a shared file system). It was originally designed for byte-range locking but according to the website, all Linux file systems lock the whole inode. So the developers decided to lock the whole object during writing. But POHMELFS has the ability to allow different clients to simultaneously write into the same page via different offsets with the result that the file will be coherent on all clients and all servers (not a small feat).

    * All events are completely asynchronous with the only exceptions being hard links and symlinks. These events include the creation of the objects as well as data reading and writing.

    * POHMELFS is designed to have a flexible object architecture that is optimized for network processing. Network processing is a potential weak point for distributed file systems since file systems can be “chatty” and create many small messages that are not always optimal for networks. The design of the object architecture allows for very long paths to the objects and the ability remove arbitrary size directories with a single network command.

    * The server portion is multi-threaded and scalable and, perhaps more importantly, is in user space. There is only a driver for POHMELFS in the kernel. The client and the server are all in user-space and interact with the driver. Assuming that the driver does not change very much, then as POHMELFS evolves only the user-space tools evolve. Consequently, new evolutions don’t require new kernels. This also means that development can progress at a very fast rate.

    * POHMELFS utilizes a transaction model for all its operations. Each transaction is an object which may embed multiple commands that are to be completed atomically. This design also means that it will resend transactions to different servers if there is a timeout or an error on the initially contacted server. This design maintains high data integrity and does not desynchronize the file system state in the event of a server failure or a network failure. An end result of this design is that if a server goes down the clients can switch to a different one automatically.

    * It has the ability for the clients to dynamically add or remove servers from a working set.

    * POHMELFS is also designed for strong authentication with the possibility of data encryption in the network channel.

    * It has extended attribute support

    * It can do read-only mounts and also has the ability to limit maximum size of the exported directory.

But POHMELFS isn’t done. While the current version is usable and ready for testing it is still in the “staging” area of the kernel (i.e. not ready for prime time). In addition the developers are porting it to use a distributed hash table called Elliptics Network. There are lots of features in Elliptics network that work well for a distributed file system as previously discussed.

While POHMELFS might not be ready to replace NFS, it does have some benefits that can prove very useful in some situations. However, be warned that POHMELFS is very much a work in progress. It is in the “staging” area of kernel drivers and the developer has announced that POHMELFS will be ported to use a distributed has table.