Shared Parallel Filesystems in Heterogeneous Linux Multi-Cluster Environments (2005)

Project Collaborators

Project Overview

In the spring of 2005, we examined parallel filesystems for shared deployment across multiple Linux clusters running different hardware architectures and operating systems. Specifically, we deployed PVFS2, GPFS, Lustre, and TerraFS in our test environment containing Intel Xeon, Intel x86-64, and IBM PPC970 systems. As part of our investigation, we commented on the feature sets of each filesystem, described our implementation and configuration experiences, and presented initial performance benchmark results. Our analysis showed that all of the parallel filesystems outperform a legacy NFS system but with different levels of complexity. Each of the filesystems demonstrated the best performance under certain conditions.

In the future, we plan to continue this investigation using additional interconnects, such as Infiniband, and perform additional tests examining parallel multi-writer file access performance.

Related Papers and Files

  1. A. Boggs, J. Cope, S. McCreary, M. Oberg, H.M. Tufo, T. Voran, and M. Woitaszek, "Improving Cluster Management with Scalable Filesystems", proceedings of the 7th LCI International Conference on Linux Clusters: The HPC Revolution, Norman, Oklahoma, April 2006.
  2. J. Cope, M. Oberg, H.M. Tufo, and M. Woitaszek, "Shared Parallel File Systems in Heterogeneous Linux Multi-Cluster Environments", proceedings of the 6th LCI International Conference on Linux Clusters: The HPC Revolution.