OSG Scalability, Reliability and Usability

The Open Science Grid is relying on several network facing services to deliver resources to its users. In order to avoid unexpected structural service problems, OSG has established a working group dedicated to testing the scalability and reliability of the services it uses. This team tests both existing and proposed software packages up to (and possibly slightly beyond) scales expected to be reached in the foreseeable future, evaluating and documenting both the response times and the failure modes.

This page provides an aggregation point of the above mentioned activity.

Activities list

Major activities:

• OSG CE And Submitter Testing And Monitoring
• GRAM5 tests
• CREAM tests
• GridFTP tests
• Hadoop FS tests
• dCache tests
• xrootd tests without SRM
• IO Profiling
• Storage Performance Test
• Test of Network Capacity Using PhEDEx Data Transfers
• Condor Scalability tests
• Condor TCP vs UDP comparison
• Static Web download test

Minor activities:

• ARC tests
• Condor Fuzzy Testing

Publications

Conference publications:

• Scalability of network facing services used in the Open Science Grid, CHEP2010 [ Preprint ]
• An update on the scalability limits of the Condor batch system, CHEP2010 [ Preprint ]
• Using Condor glideins for distributed testing of network-facing services, IWHGA2010 [ Paper | Preprint ]

Technical reports:

• Loadtest_condor – A workload generating framework for testing scalability and reliability of the Condor system [ OSG Doc 1013 ]
• Procs_monitor - A process-level resource monitor [ OSG Doc 1012 ]
• Evaluation of new Compute Element software for the Open Science Grid: GRAM5 and CREAM [ OSG Doc 1006 ]

Internal documents

• Requirements of a test bed

-- IgorSfiligoi - 2011/02/01