Difference: WorkForAidan (2 vs. 3)

Revision 32016/01/23 - Main.FkW

Line: 1 to 1

Big Picture

Understand the performance characteristics of jobs running on the global grid infrastructure by correlating information from HTCondor with information from XRootd.


Metric info to describe the dataset

Split the metrics up into just the sample that you can correlate to XRootd, as well as the total. For some of these this split makes no sense. It'll be obvious which ones those are.

Simple numbers

  • start and end date
  • total number of jobs in HTCondor
  • total number of DAGs
  • total number of users
  • total number of sites
  • total amount of walltime consumed
  • total amount of CPU time consumed
  • total amount of data accessed
    • total amount of data as well as fraction of that data that was read
  • total number of datasets
  • total number of releases
  • total number of jobs that accessed
    • AOD or AODSIM
    • MiniAOD?
    • RECO
    • other
  • same as previous but
    • walltime per data type
    • cputime per data type
    • number of users that read the data type

Averages and their variances

For each of the following calculate the average, the stddev, and the median.

  • number of jobs per DAG
  • number of jobs per user
  • number jobs per site
  • number of users per site
  • amount of data read per job
  • fraction of data read per job
  • walltime per job
  • cputime per job
  • cpu efficiency per job
  • the previous five split up by data type
  • walltime per user
  • cputime per user
  • cpu efficiency per user


For all of the previous metrics also each a histogram of the distribution, rather than just average, stddev, and median.


Initial Questions to answer

  • Is the distribution of CPU/walltime different for jobs that read remotely via XRootd then for jobs that read local at the site that they consume storage at?
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback