Object on GPFS

Hello to all. As you might have seen from Jez’s recent post to the mailing list, IBM are hoping to get more involved with the user group in the coming year and part of that will be blogs from developers. This is the first such blog and I’ll be looking at the recently updated red paper for deploying an object store on top of an existing GPFS cluster.

Firstly though a quick introduction. I am a developer in the IBM Manchester Lab and my current role is as an architect for RAS (Reliability, Availability and Servicability), with specific focus on the integration of protocols with GPFS that you should be seeing later this year. In Manchester we have a team of ~10 developers all working towards these future GPFS releases, including James Wormwell, an author on the updated red paper and team lead for the development of the install tool that was made available with the update. Hopefully you should get a chance to meet some of us over the year at the main user group meeting or the various “coffee shop” meetups that we’re hoping to have.

On to the main topic of this blog post. As I’ve mentioned, an update to the red paper for deploying an object store (specifically Openstack Swift) on GPFS was released at the end of last year. The red paper is available here and walks through everything you will need, from introducing the concept and use cases of an object store, through preparing and deploying on a GPFS cluster, to back up and restore to protect against data loss and corruption.

Also included in the red paper are the benefits of deploying an object store on GPFS vs other solutions. From my perspective the main advantage is that we can make use of GPFS’s data protection capabilities and remove the need for Swift’s three-way replication. Aside from the obvious saving in disk space, this means that all object nodes in the cluster have access to all of the objects stored on the system, resulting in no need to rebalance when adding or removing nodes and no network traffic between nodes to retrieve objects. There are a number other benefits to the Swift on GPFS solution but I’ll keep the marketing spiel to a minimum and if you’re interested you can read more in the red paper.

The last thing I want to mention, and the major new addition with the update, is the installer. In the first version of the red paper there were a number of scripts that were provided to help with the installation. These were useful and helped speed up the process significantly if you knew how to use them but they weren’t always the easiest to use, particularly if something went wrong. The installer helps to solve this problem by being a single tool that you can use for all steps of the installation. It is a tool written in python that makes use of chef to install and configure the object store on each of your nodes.

The installer covering all aspects of the process makes avoiding problems during installation much easier, as it means it can perform plenty of checks to ensure what is being done is sensible. Additionally the installer is also capable of managing your object install, allowing you to easily add and remove nodes. All of this should make Elastic Storage Object an easy to use tool for anyone who might be interested in adding an object store to their environment in the future.

Hopefully my first foray into the blogosphere has been interesting and maybe even useful. If you have any comments, questions or suggestions for topics for future blog posts please get in touch via the mailing list.