TWiki> UCSDTier2 Web>BDIItests (revision 3)EditAttach

This page documents bdii loadtests done by Sanjay Padhi for OSG.

Description of tools used

We use the python interface to root in order to plot results in realtime while the test is running, and then store the results histograms in rootfile. The logic of the test is as follows:

  • Run N python threads
  • Each thread queries the bdii with:
    • os.popen("ldapsearch -xLLL -p2170 -h -b o=grid","r")
  • Each thread collects the return, and records the time it took to complete the query.
  • The average return time for the N threads is logged in hprof histogram, after all N threads have returned.
  • Run the next N threads. The N threads are launched at most once per second. However, given that the return time of the ldapsearch is several seconds, the launch of the N threads is really much less often than once per second.
  • Continue doing this for a fixed amount of time dt


Sunday September 14th

Ran a few different short tests, then one longer test of a few hours. For the longer run we picked: N = 15 and dt = 12000 seconds = 200 minutes = 3h 20min, and 18000 seconds = 300min = 5h respectively.

We then ran this test simultaneously from CERN (12000 seconds) and UCSD (18000 seconds). The CERN test ended at 2:35 Monday September 15th CERN time, while the UCSD one ended at 19:39 pacific on the 14th, i.e. 2h and 4min later.

  • Response time for the bdii querries from CERN:

  • Response time for the bdii queries from UCSD:

  • bdii host system monitoring: network traffic:

  • bdii host system monitoring: netstat:

  • bdii host system monitoring: processes:

  • bdii host system monitoring: loadavg:

Understanding the client profile better (Monday September 15th)

To understand the client profile better, we did a series of tests where we varied N first on just one machine, and then having the same N but running the test program 4 times in parallel on 4 different (but identical hardware) hosts.

We find the the time per query depends significantly on the number of parallel python threads, but not significantly on whether we run one or 4 simultaneously.

N time for 4 in parallel time for one by itself
1 1.7-2.2s 2.2
10 3.2-3.5s 3.2
20 4.8-5.0s 5.0

-- FkW - 10 Sep 2008

Topic attachments
I Attachment Action Size Date Who Comment
gifgif bdii-from-cern.gif manage 12.5 K 2008/09/15 - 10:53 FkW Response time for the bdii querries from CERN
gifgif bdii-from-ucsd.gif manage 29.4 K 2008/09/15 - 11:00 FkW bdii response fro UCSD
pngpng riley-if_eth0-day.png manage 26.4 K 2008/09/15 - 10:54 FkW bdii host system monitoring: network traffic
pngpng riley-load-day.png manage 29.0 K 2008/09/15 - 10:55 FkW bdii host system monitoring: loadavg
pngpng riley-netstat-day.png manage 26.8 K 2008/09/15 - 10:54 FkW bdii host system monitoring: netstat
pngpng riley-processes-day.png manage 22.6 K 2008/09/15 - 10:55 FkW bdii host system monitoring: processes
Edit | Attach | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r3 - 2008/09/15 - 13:19:04 - FkW
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback