Difference: GridFTPLVSTests2016 (6 vs. 7)

Revision 72016/12/14 - Main.CliftonPotter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

GridFTP? -LVS Tests

Line: 7 to 7
  To test the throughput and client load limits of the LVS and the GridFTP? servers
Changed:
<
<
  • A variable (range: 100-8k) number of 30-40 minute jobs were submitted via condor_submit and put into the condor_q(ueue)
  • Each job proceeds by selecting a file from a list at random, gfal-copying that file (writing it to /dev/null) and then sleeping for a variable amount of time (10 sec) before repeating the process and continuing for 30-40 minutes.
>
>
  • A variable (range: 100-8k) number of 30-45 minute jobs were submitted via condor_submit and put into the condor_q(ueue)
  • Each job proceeds by selecting a file from a list at random, gfal-copying that file (writing it to /dev/null) and then sleeping for a variable amount of time (2-10 sec) before repeating the process and continuing for 30-45 minutes.
 
  • The throughput (Gbps) of each individual gftp-X.t2.ucsd.edu server as well as the number of active jobs was recorded at 30 second time intervals through each client's execution
Changed:
<
<
  • Objective is to test 6 GridFTP? Servers on the LVS setup and to see if the system performs as expected
>
>
  • Objective is to test 6 GridFTP? Servers on the LVS setup and to see if the system perform as expected
 
  • !!!Number of clients active when maximum throughput occurs
  • !!!Total Throughput/Client (Gbps/Client)
Deleted:
<
<
<--td {border: 1px solid #ccc;}br {mso-data-placement:same-cell;}-->
<--td {border: 1px solid #ccc;}br {mso-data-placement:same-cell;}-->
<--td {border: 1px solid #ccc;}br {mso-data-placement:same-cell;}-->
 

Example Code

gfcTest.sh

Line: 45 to 43
 

Data

Changed:
<
<

<--td {border: 1px solid #ccc;}br {mso-data-placement:same-cell;}-->
# Submitted JobsTest Duration (min)# Active Jobs @ Max ThroughputStacked Max Throughput (Gbps)Total Throughput/Client (Gbps)
100301005.5320.06
10003096614.260.02
15004026917.20.07
200040158614.360.01
22504048114.20.03
25004079526.430.04
40003051620.030.04
800030219518.390.01
40004089613.650.02
80004039719.830.05

>
>

10 second sleep time

Initially the jobs submitted slept for 10 seconds in between each execution of gfal-copy. The data was inconsistent and no clear correlations were found. The average total bandwidth at max throughput was 17.5 Gbps with an average of 740 clients active each having an individual bandwidth of ~0.032 Gbps.

2 second sleep time

The jobs submitted which slept for 2 seconds in between each instance of gfal-copy were much more consistent. The average total bandwidth at max throughput was ~17.5 Gbps. The number of clients active at max throughput greatly stabilized and remained consistent in the range of 900-1200 jobs active with an average of 1011 active clients at max throughput each having an individual bandwidth of ~0.017 Gbps.

 
META FILEATTACHMENT attachment="TotalMaxThroughputPerGFTPNumClientsActive.pdf" attr="" comment="" date="1481263128" name="TotalMaxThroughputPerGFTPNumClientsActive.pdf" path="TotalMaxThroughputPerGFTPNumClientsActive.pdf" size="41204" stream="TotalMaxThroughputPerGFTPNumClientsActive.pdf" tmpFilename="/tmp/mR8DYzNs4m" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="MaxThroughputPerClient.pdf" attr="" comment="" date="1481263141" name="MaxThroughputPerClient.pdf" path="MaxThroughputPerClient.pdf" size="42545" stream="MaxThroughputPerClient.pdf" tmpFilename="/tmp/boZ795y4Ud" user="CliftonPotter" version="1"
Added:
>
>
META FILEATTACHMENT attachment="ClientsActiveOutOfQueued_10secSleep.pdf" attr="" comment="" date="1481745666" name="ClientsActiveOutOfQueued_10secSleep.pdf" path="ClientsActiveOutOfQueued_10secSleep.pdf" size="42026" stream="ClientsActiveOutOfQueued_10secSleep.pdf" tmpFilename="/tmp/34Z3kTBMQu" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="MaxThruPerClientVsActiveClients_2secSleep.pdf" attr="" comment="" date="1481745578" name="MaxThruPerClientVsActiveClients_2secSleep.pdf" path="MaxThruPerClientVsActiveClients_2secSleep.pdf" size="43140" stream="MaxThruPerClientVsActiveClients_2secSleep.pdf" tmpFilename="/tmp/u8mcVgIsSB" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="MaxThruPerClientVsActiveClients_10secSleep.pdf" attr="" comment="" date="1481745644" name="MaxThruPerClientVsActiveClients_10secSleep.pdf" path="MaxThruPerClientVsActiveClients_10secSleep.pdf" size="43518" stream="MaxThruPerClientVsActiveClients_10secSleep.pdf" tmpFilename="/tmp/M4qJYDnxMm" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="MaxThruVsNumActiveClients_2secSleep.pdf" attr="" comment="" date="1481745631" name="MaxThruVsNumActiveClients_2secSleep.pdf" path="MaxThruVsNumActiveClients_2secSleep.pdf" size="41773" stream="MaxThruVsNumActiveClients_2secSleep.pdf" tmpFilename="/tmp/dM1fkt93v6" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="MaxThruVsNumActiveClients_10secSleep.pdf" attr="" comment="" date="1481745651" name="MaxThruVsNumActiveClients_10secSleep.pdf" path="MaxThruVsNumActiveClients_10secSleep.pdf" size="42175" stream="MaxThruVsNumActiveClients_10secSleep.pdf" tmpFilename="/tmp/7u4KM2ORbg" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="ClientsActiveOutOfQueued_2secSleep.jpg" attr="" comment="" date="1481745993" name="ClientsActiveOutOfQueued_2secSleep.jpg" path="ClientsActiveOutOfQueued_2secSleep.jpg" size="121654" stream="ClientsActiveOutOfQueued_2secSleep.jpg" tmpFilename="/tmp/K6OajyqE34" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="ClientsActiveOutOfQueued_10secSleep.jpg" attr="" comment="" date="1481746004" name="ClientsActiveOutOfQueued_10secSleep.jpg" path="ClientsActiveOutOfQueued_10secSleep.jpg" size="120718" stream="ClientsActiveOutOfQueued_10secSleep.jpg" tmpFilename="/tmp/LCaAWxEQ2a" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="MaxThruPerClientVsActiveClients_2secSleep.jpg" attr="" comment="" date="1481746010" name="MaxThruPerClientVsActiveClients_2secSleep.jpg" path="MaxThruPerClientVsActiveClients_2secSleep.jpg" size="102087" stream="MaxThruPerClientVsActiveClients_2secSleep.jpg" tmpFilename="/tmp/GtBUNgo7Sr" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="MaxThruPerClientVsActiveClients_10secSleep.jpg" attr="" comment="" date="1481746018" name="MaxThruPerClientVsActiveClients_10secSleep.jpg" path="MaxThruPerClientVsActiveClients_10secSleep.jpg" size="105115" stream="MaxThruPerClientVsActiveClients_10secSleep.jpg" tmpFilename="/tmp/cGYDxi3Awq" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="MaxThruVsNumActiveClients_2secSleep.jpg" attr="" comment="" date="1481746025" name="MaxThruVsNumActiveClients_2secSleep.jpg" path="MaxThruVsNumActiveClients_2secSleep.jpg" size="94745" stream="MaxThruVsNumActiveClients_2secSleep.jpg" tmpFilename="/tmp/JN8FptUY0N" user="CliftonPotter" version="1"
META FILEATTACHMENT attachment="MaxThruVsNumActiveClients_10secSleep.jpg" attr="" comment="" date="1481746031" name="MaxThruVsNumActiveClients_10secSleep.jpg" path="MaxThruVsNumActiveClients_10secSleep.jpg" size="98326" stream="MaxThruVsNumActiveClients_10secSleep.jpg" tmpFilename="/tmp/XTnWaLtA3R" user="CliftonPotter" version="1"
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback