Difference: OsgBdiiTesting (1 vs. 6)

Revision 62008/12/22 - Main.FkW

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

OSG Bdii Testing

Line: 14 to 14
  The script bdii_par.py is configured to send 15 queries at a time to the server is-dev.grid.iu.edu:2170. The script then waits for the queries to return, counting successful and failed queries for a designated total amount of time.
Changed:
<
<
In order to test the success and failure rates, we vary the number of script instances running simultaneously. For all of the tests, we distribute the instances over five machines at the UCSD T2. The number of instances run are 5, 10, 15, 20, 25, and 30. For each of these points, the test was run for 12 hours, with the 5, 20, and 30 points also being tested for shorter time periods (2 or 3 hours).
>
>
In order to test the success and failure rates, we vary the number of script instances running simultaneously. For all of the tests, we distribute the instances over five 8 core machines at the UCSD T2. The number of instances run are 5, 10, 15, 20, 25, and 30. There are thus at most 6 instances per 8 core machine running. For each of these points, the test was run for 12 hours, with the 5, 20, and 30 points also being tested for shorter time periods (2 or 3 hours).
 

Results

Line: 23 to 23
 Our main findings are:
  • The number of fails (and the rate of fails) increase as the number of script instances increase.
  • There are no fails below a certain number of instances: 10.
Changed:
<
<
  • The total rate of queries never exceeds 20 Hz.
>
>
  • There are no fails below a certain rate of querries: 10Hz.
  • The total rate of queries never exceeds 20 Hz. As we run more simultaneous querries,the querries take longer, and a larger fraction of them fails.
 

Detailed Results

Line: 47 to 48
 

Failure rate versus Number of processes

Changed:
<
<
What is plotted here is the failure rate as a function of the total number of script instances running concurrently. This data is in the table above in the first and last columns.
>
>
What is plotted here is the failure rate as a function of the total number of script instances running concurrently. This data is in the table above in the first and last columns. Recall that each of these processes is an instance of our script. Each script runs 15 querries in parallel.
 
Changed:
<
<
bdii_fails_v_proc.jpg
<-- width="696" height="468" -->
>
>
bdii_fails_v_proc.jpg
<-- width="696" height="468" -->
 

Script Instances Histogram

What is plotted below is a histogram of the individual script instances. The top histogram is the individual script success rate, that is, the number of queries passed for that script divided by the total time that that script ran for. The bottom histogram is the individual script failure rate: number of failed queries for each script divided by that script's total run time. In both histograms, the colors are the total number of script instances running for that instance, as indicated in the legend. The different plots are overlayed, not stacked.

Changed:
<
<
bdii_query_data.jpg
>
>
bdii_query_data.jpg
  -- WarrenAndrews - 2008/12/18

Revision 52008/12/20 - Main.WarrenAndrews

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

OSG Bdii Testing

Line: 49 to 49
  What is plotted here is the failure rate as a function of the total number of script instances running concurrently. This data is in the table above in the first and last columns.
Changed:
<
<
bdii_fails_v_proc.jpg
<-- width="696" height="468" -->
>
>
bdii_fails_v_proc.jpg
<-- width="696" height="468" -->
 

Script Instances Histogram

What is plotted below is a histogram of the individual script instances. The top histogram is the individual script success rate, that is, the number of queries passed for that script divided by the total time that that script ran for. The bottom histogram is the individual script failure rate: number of failed queries for each script divided by that script's total run time. In both histograms, the colors are the total number of script instances running for that instance, as indicated in the legend. The different plots are overlayed, not stacked.

Changed:
<
<
bdii_query_data.jpg
>
>
bdii_query_data.jpg
  -- WarrenAndrews - 2008/12/18
Changed:
<
<
META FILEATTACHMENT attachment="bdii_query_data.jpg" attr="" comment="" date="1229649105" name="bdii_query_data.jpg" path="bdii_query_data.jpg" size="32948" stream="bdii_query_data.jpg" tmpFilename="/usr/tmp/CGItemp8340" user="WarrenAndrews" version="2"
META FILEATTACHMENT attachment="bdii_fails_v_proc.jpg" attr="" comment="" date="1229723160" name="bdii_fails_v_proc.jpg" path="bdii_fails_v_proc.jpg" size="16110" stream="bdii_fails_v_proc.jpg" tmpFilename="/usr/tmp/CGItemp7802" user="WarrenAndrews" version="1"
>
>
META FILEATTACHMENT attachment="bdii_query_data.jpg" attr="" comment="" date="1229745592" name="bdii_query_data.jpg" path="bdii_query_data2.jpg" size="23010" stream="bdii_query_data2.jpg" tmpFilename="/usr/tmp/CGItemp11791" user="WarrenAndrews" version="3"
META FILEATTACHMENT attachment="bdii_fails_v_proc.jpg" attr="" comment="" date="1229743018" name="bdii_fails_v_proc.jpg" path="bdii_fails_v_proc.jpg" size="15912" stream="bdii_fails_v_proc.jpg" tmpFilename="/usr/tmp/CGItemp11858" user="WarrenAndrews" version="2"
META FILEATTACHMENT attachment="bdii_fails_v_proc2.jpg" attr="" comment="" date="1229743268" name="bdii_fails_v_proc2.jpg" path="bdii_fails_v_proc2.jpg" size="14838" stream="bdii_fails_v_proc2.jpg" tmpFilename="/usr/tmp/CGItemp7861" user="WarrenAndrews" version="1"
META FILEATTACHMENT attachment="bdii_query_data2.jpg" attr="" comment="" date="1229745671" name="bdii_query_data2.jpg" path="bdii_query_data2.jpg" size="23010" stream="bdii_query_data2.jpg" tmpFilename="/usr/tmp/CGItemp7828" user="WarrenAndrews" version="1"

Revision 42008/12/19 - Main.WarrenAndrews

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

OSG Bdii Testing

Line: 21 to 21
 

Summary of Results

Our main findings are:

Changed:
<
<
  • The number of fails (and the rate of fails) increase as the number of processes increase.
  • There are no fails below a certain number of processes: 10.
>
>
  • The number of fails (and the rate of fails) increase as the number of script instances increase.
  • There are no fails below a certain number of instances: 10.
 
  • The total rate of queries never exceeds 20 Hz.

Detailed Results

Run Summary Table

Changed:
<
<
The following table summarizes the results of the different runs. Table entries are the total of all the processes in a given run, unless the column says average, in which case it is the average of the different processes.
>
>
The following table summarizes the results of the different runs. Table entries are the total of all the processes (script instances) in a given run, unless the column says average, in which case it is the average of the different processes.
  What is meant by 'Input time' is the parameter in the script which tells the script how long to run. 'Average run time' is the average of the actual time each script runs for, as output by the script. 'Average pass (fail) rate' is 'Total queries passed (failed)' divided by 'Average run time.'
Line: 45 to 45
 
30 2 7337 109202 14.9 388 0.052
30 12 43464 659036 15.16 2419 0.055
Added:
>
>

Failure rate versus Number of processes

What is plotted here is the failure rate as a function of the total number of script instances running concurrently. This data is in the table above in the first and last columns.

bdii_fails_v_proc.jpg

<-- width="696" height="468" -->
 

Script Instances Histogram

What is plotted below is a histogram of the individual script instances. The top histogram is the individual script success rate, that is, the number of queries passed for that script divided by the total time that that script ran for. The bottom histogram is the individual script failure rate: number of failed queries for each script divided by that script's total run time. In both histograms, the colors are the total number of script instances running for that instance, as indicated in the legend. The different plots are overlayed, not stacked.

Line: 54 to 60
 -- WarrenAndrews - 2008/12/18

META FILEATTACHMENT attachment="bdii_query_data.jpg" attr="" comment="" date="1229649105" name="bdii_query_data.jpg" path="bdii_query_data.jpg" size="32948" stream="bdii_query_data.jpg" tmpFilename="/usr/tmp/CGItemp8340" user="WarrenAndrews" version="2"
Added:
>
>
META FILEATTACHMENT attachment="bdii_fails_v_proc.jpg" attr="" comment="" date="1229723160" name="bdii_fails_v_proc.jpg" path="bdii_fails_v_proc.jpg" size="16110" stream="bdii_fails_v_proc.jpg" tmpFilename="/usr/tmp/CGItemp7802" user="WarrenAndrews" version="1"

Revision 32008/12/19 - Main.WarrenAndrews

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

OSG Bdii Testing

Line: 33 to 33
  What is meant by 'Input time' is the parameter in the script which tells the script how long to run. 'Average run time' is the average of the actual time each script runs for, as output by the script. 'Average pass (fail) rate' is 'Total queries passed (failed)' divided by 'Average run time.'
Changed:
<
<
Num processes Input time (h) Average run time (s) Total queries passed Average pass rate Total queries failed Average fail rate
>
>
Num processes Input time (h) Average run time (s) Total queries passed Average pass rate (Hz) Total queries failed Average fail rate (Hz)
 
5 2 7204 70095 9.73 0 0
5 12 43202 425055 9.84 0 0
10 12 43218 635995 14.7 5 0.00012
Line: 49 to 49
  What is plotted below is a histogram of the individual script instances. The top histogram is the individual script success rate, that is, the number of queries passed for that script divided by the total time that that script ran for. The bottom histogram is the individual script failure rate: number of failed queries for each script divided by that script's total run time. In both histograms, the colors are the total number of script instances running for that instance, as indicated in the legend. The different plots are overlayed, not stacked.
Changed:
<
<
bdii_query_data.jpg
>
>
bdii_query_data.jpg
  -- WarrenAndrews - 2008/12/18

Revision 22008/12/19 - Main.WarrenAndrews

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

OSG Bdii Testing

Line: 21 to 21
 

Summary of Results

Our main findings are:

Changed:
<
<
  • The number of fails (and the rate of fails) increase as the number of processes increases
>
>
  • The number of fails (and the rate of fails) increase as the number of processes increase.
 
  • There are no fails below a certain number of processes: 10.
  • The total rate of queries never exceeds 20 Hz.

Detailed Results

Changed:
<
<
[TABLE]
>
>

Run Summary Table

The following table summarizes the results of the different runs. Table entries are the total of all the processes in a given run, unless the column says average, in which case it is the average of the different processes.

What is meant by 'Input time' is the parameter in the script which tells the script how long to run. 'Average run time' is the average of the actual time each script runs for, as output by the script. 'Average pass (fail) rate' is 'Total queries passed (failed)' divided by 'Average run time.'

Num processes Input time (h) Average run time (s) Total queries passed Average pass rate Total queries failed Average fail rate
5 2 7204 70095 9.73 0 0
5 12 43202 425055 9.84 0 0
10 12 43218 635995 14.7 5 0.00012
15 3 10872 160183 14.7 47 0.0043
15 12 43221 820458 19.0 102 0.0023
20 3 10847 161719 14.90 176 0.016
20 12 43312 834140 19.3 775 0.017
25 12 43380 666060 15.4 1665 0.038
30 2 7337 109202 14.9 388 0.052
30 12 43464 659036 15.16 2419 0.055

Script Instances Histogram

  What is plotted below is a histogram of the individual script instances. The top histogram is the individual script success rate, that is, the number of queries passed for that script divided by the total time that that script ran for. The bottom histogram is the individual script failure rate: number of failed queries for each script divided by that script's total run time. In both histograms, the colors are the total number of script instances running for that instance, as indicated in the legend. The different plots are overlayed, not stacked.

Revision 12008/12/19 - Main.WarrenAndrews

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"

OSG Bdii Testing

Introduction

A series of tests are done in order to better understand how OSG's central US server will respond to a high volume of bdii queries. The tests are done using the script bdii_par.py (from bdiisrc.tar.gz). One of the main goals is to meausre the failure rate of bdii queries as a function of the query rate. Also measured is the rate of both successful and failed queries as a function of the number of script instances running.

The tar can be found at: /pnfs/t2.ucsd.edu/data4/cms/phedex/store/user/spadhi/osg/bdii/bdiisrc.tar.gz

Implementation

The script bdii_par.py is configured to send 15 queries at a time to the server is-dev.grid.iu.edu:2170. The script then waits for the queries to return, counting successful and failed queries for a designated total amount of time.

In order to test the success and failure rates, we vary the number of script instances running simultaneously. For all of the tests, we distribute the instances over five machines at the UCSD T2. The number of instances run are 5, 10, 15, 20, 25, and 30. For each of these points, the test was run for 12 hours, with the 5, 20, and 30 points also being tested for shorter time periods (2 or 3 hours).

Results

Summary of Results

Our main findings are:

  • The number of fails (and the rate of fails) increase as the number of processes increases
  • There are no fails below a certain number of processes: 10.
  • The total rate of queries never exceeds 20 Hz.

Detailed Results

[TABLE]

What is plotted below is a histogram of the individual script instances. The top histogram is the individual script success rate, that is, the number of queries passed for that script divided by the total time that that script ran for. The bottom histogram is the individual script failure rate: number of failed queries for each script divided by that script's total run time. In both histograms, the colors are the total number of script instances running for that instance, as indicated in the legend. The different plots are overlayed, not stacked.

bdii_query_data.jpg

-- WarrenAndrews - 2008/12/18

META FILEATTACHMENT attachment="bdii_query_data.jpg" attr="" comment="" date="1229649105" name="bdii_query_data.jpg" path="bdii_query_data.jpg" size="32948" stream="bdii_query_data.jpg" tmpFilename="/usr/tmp/CGItemp8340" user="WarrenAndrews" version="2"
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback