Difference: StoragePerformanceTest (3 vs. 4)

Revision 42011/02/08 - Main.HaifengPi

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Scalability Test on BeStMan? and BeStMan2?

Line: 6 to 6
  The work of 2010 was documented in Measurement of BeStMan Scalability
Changed:
<
<
In the last 3 months (from Nov. 2010 to Jan. 2011), the testing work continued for the new release of BeStMan? . The scalability tool, glideTester, was used primarily, which was very efficient and easy to operate. The lcg clients were running at UCSD. We achieved similar performance in the newly deployed instance of BeStMan? at FNAL as that of 2010. Follow shows the preliminary results for the latest BeStMan? release (2.0.5):
>
>
In the last 3 months (from Nov. 2010 to Jan. 2011), the testing work continued for the new release of BeStMan? . The scalability tool, glideTester, was used primarily, which was very efficient and easy to operate. The lcg clients were running at UCSD. We achieved similar performance in the newly deployed instance of BeStMan? at FNAL as that of 2010. The file system is the local drive at the BeStMan? server. Follow shows the preliminary results done in Jan. 2011 for the latest BeStMan? release (2.0.5):
 
Number of lcg clients Server Processing Rate (Hz)
100 78.3
Line: 16 to 16
 
900 73.2
1100 70.1
Changed:
<
<
With a single BeStMan? server mounted to a standard HDFS system, we expect the server processing rate for a small meta data operation (e.g. listing all the items in the directories with ~10s of files or subdirectories) is ~50 Hz. The large requests that involves thousands of files or subdirectories, the processing time for finishing the request and number of requests that server can finish simultaneously will be significantly longer and smaller respectively.
>
>
With a single BeStMan? server mounted to a standard HDFS system, we measured the server processing rate for a small meta data operation (e.g. listing all the items in the directories with ~10s of files or subdirectories) is ~40-50 Hz. The large requests that involves thousands of files or subdirectories, the processing time for finishing the request and number of requests that server can finish simultaneously will be significantly longer and smaller respectively.
  From the scalability point of view, the BeStMan? is less likely a bottleneck of the storage system of a typical Tier-2 center at LHC. More work will be on optimizing the configuration and better understanding the dependency between server processing rate and concurrency of clients.

Scalability of HDFS and System I/O architecture

 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback