Difference: CrabTesting (1 vs. 6)

Revision 62009/03/25 - Main.WarrenAndrews

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Crab Grid Test

Line: 23 to 23
 
    • Change the data to run on (optional?): line 91:
fileNames = cms.untracked.vstring('/store/relval/CMSSW_2_1_9/RelValZEE/GEN-SIM-DIGI-RAW-HLTDEBUG-RECO/STARTUP_V7_v2/0000/04419036-F385-DD11-B3A7-001617C3B6E8.root')
Added:
>
>
 
    • Change output (broken--we don't get any output of first test)
    • Change path to only run a single module instead of all defined: line 157 (second to last):
process.p = cms.Path(process.generalmakers)
Line: 46 to 47
 
      • Make sure 'datasetpath' is correct (line below '[CMSSW]')
      • Insert (edit) the following line immediately below the 'datasetpath' line to tell Crab which CMSSW config file to use:
pset=test_new.py
Added:
>
>
 
      • Make sure there is no typo in the following line (at end of CMSSW section):
events_per_job = 50
Added:
>
>
 
      • Optionally, change total events, or events per job if desired
      • If there is a problem with site selection (or if you want to be safe and be sure you run at UCSD), insert the following at the end of this file:
[EDG]
Line: 62 to 67
 
      • To create and submit the job in one command:
crab -create -submit -cfg test_crab.cfg
Added:
>
>
 
      • Optionally, you can create the job and submit separately (where xxx is last output from the first line below):
crab -create -cfg test_crab.cfg
crab -submit -continue xxx
Line: 69 to 76
 
  • Check status of job (xxx is optional, and defined line above):
crab -status -continue xxx
Changed:
<
<
  • Get output of job: is given by the status command above, this should work for completed jobs
crab -getoutput
>
>
  • Get output of job: is dir created by crab, optional: is given by the status command above, this should work for completed jobs
crab -getoutput -c
 

Links

Revision 52008/12/04 - Main.WarrenAndrews

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Crab Grid Test

Line: 57 to 57
 source /code/osgcode/ucsdt2/gLite/etc/profile.d/grid_env.sh

  • Submit crab job
Added:
>
>
    • In order for this to work, you will need a proxy, or a grid cert which crab will use to get you a proxy
    • 2 options:
      • To create and submit the job in one command:
 crab -create -submit -cfg test_crab.cfg
Changed:
<
<
  • Check status of job
>
>
      • Optionally, you can create the job and submit separately (where xxx is last output from the first line below):
crab -create -cfg test_crab.cfg
crab -submit -continue xxx
 
Changed:
<
<
(wandrews note: see ~/Notes)
>
>
  • Check status of job (xxx is optional, and defined line above):
crab -status -continue xxx
 
Changed:
<
<
links:
>
>
  • Get output of job: is given by the status command above, this should work for completed jobs
crab -getoutput

Links

  https://cmsweb.cern.ch/dbs_discovery/

Revision 42008/12/03 - Main.WarrenAndrews

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Crab Grid Test

Changed:
<
<

goal:

  • investigate performance of glidein submission versus direct (condor) submission
>
>

Goal

Investigate performance of glidein submission versus direct (condor) submission
 
Changed:
<
<

implementation:

>
>

Implementation

 

Run ntuplemaker on dataset

We chose the ntuplemaker because it is a useful thing to run, and nontrivial
  • Check out CMSSW (we use 2_1_9 for this exercise, this will create a directory in the pwd where it is called)
Line: 14 to 14
 
  • Check out CMS2
export CVSROOT=:pserver:anonymous:98passwd@cmscvs.cern.ch:/cvs_server/repositories/CMSSW/
Changed:
<
<
cd CMSSW_2_X_X/src
>
>
cd CMSSW_2_1_9/src
 cvs co -r V01-00-04 -d CMS2/NtupleMaker UserCode? /JRibnik/CMS2/NtupleMaker scramv1 b -j 8
Line: 26 to 26
 
    • Change output (broken--we don't get any output of first test)
    • Change path to only run a single module instead of all defined: line 157 (second to last):
process.p = cms.Path(process.generalmakers)
Changed:
<
<
    • Run CMSSW: (note, the first line is common to alias to 'ev')
>
>
  • Run CMSSW: (note: the first line is common to alias to 'ev')
 
eval `scramv1 runtime -sh`
cmsRun test_cfg.py
Changed:
<
<

Install Crab

wget https://cmsweb.cern.ch/crabconf/files/CRAB_2_4_2.tgz --no-check-certificate tar -zxvf CRAB_2_4_2.tgz cd CRAB_2_4_2 ./configure
>
>

Install Crab and Submit Job

  • Install Crab
wget https://cmsweb.cern.ch/crabconf/files/CRAB_2_4_2.tgz --no-check-certificate
tar -zxvf CRAB_2_4_2.tgz
cd CRAB_2_4_2
./configure
 
  • Prepare crab job
    • Go on DBS (link below) and find the dataset you put in your CMSSW config file. Under that dataset, you should see a link 'crabCfg'. This is a sample crab config file for this dataset.
    • Put this file in 'CMS2/NtupleMaker/test/test_crab.cfg'
    • Edit this file:
Added:
>
>
      • Change scheduler to 'condor_g' instead of 'glite' (line 4 or so)
      • Make sure 'datasetpath' is correct (line below '[CMSSW]')
      • Insert (edit) the following line immediately below the 'datasetpath' line to tell Crab which CMSSW config file to use:
pset=test_new.py
      • Make sure there is no typo in the following line (at end of CMSSW section):
events_per_job = 50
      • Optionally, change total events, or events per job if desired
      • If there is a problem with site selection (or if you want to be safe and be sure you run at UCSD), insert the following at the end of this file:
[EDG]
SE_white_list = srm-3.t2.ucsd.edu
  • Source:
source ~/CRAB_2_4_2/crab.sh
source /code/osgcode/ucsdt2/gLite/etc/profile.d/grid_env.sh
 
Added:
>
>
  • Submit crab job
crab -create -submit -cfg test_crab.cfg
 
Changed:
<
<
source ~/CRAB_2_4_2/crab.sh

    • submit crab job
crab -create -submit
>
>
  • Check status of job
  (wandrews note: see ~/Notes)

Revision 32008/12/03 - Main.WarrenAndrews

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Added:
>
>

Crab Grid Test

 
Changed:
<
<
goal:
>
>

goal:

 
  • investigate performance of glidein submission versus direct (condor) submission
Changed:
<
<
implementation:
  • Run ntuplemaker on sample dataset
>
>

implementation:

Run ntuplemaker on dataset

We chose the ntuplemaker because it is a useful thing to run, and nontrivial
  • Check out CMSSW (we use 2_1_9 for this exercise, this will create a directory in the pwd where it is called)
scramv1 project CMSSW CMSSW_2_1_9

  • Check out CMS2
export CVSROOT=:pserver:anonymous:98passwd@cmscvs.cern.ch:/cvs_server/repositories/CMSSW/
cd CMSSW_2_X_X/src
cvs co -r V01-00-04 -d CMS2/NtupleMaker UserCode/JRibnik/CMS2/NtupleMaker
scramv1 b -j 8

  • Edit the CMSSW config file
    • There should be a file 'test_cfg.py' in 'CMSSW_2_1_9/src/CMS2/NtupleMaker/test'
    • Change the data to run on (optional?): line 91:
fileNames = cms.untracked.vstring('/store/relval/CMSSW_2_1_9/RelValZEE/GEN-SIM-DIGI-RAW-HLTDEBUG-RECO/STARTUP_V7_v2/0000/04419036-F385-DD11-B3A7-001617C3B6E8.root')

    • Change output (broken--we don't get any output of first test)
    • Change path to only run a single module instead of all defined: line 157 (second to last):
process.p = cms.Path(process.generalmakers)
    • Run CMSSW: (note, the first line is common to alias to 'ev')
eval `scramv1 runtime -sh`
cmsRun test_cfg.py

Install Crab

wget https://cmsweb.cern.ch/crabconf/files/CRAB_2_4_2.tgz --no-check-certificate tar -zxvf CRAB_2_4_2.tgz cd CRAB_2_4_2 ./configure

  • Prepare crab job
    • Go on DBS (link below) and find the dataset you put in your CMSSW config file. Under that dataset, you should see a link 'crabCfg'. This is a sample crab config file for this dataset.
    • Put this file in 'CMS2/NtupleMaker/test/test_crab.cfg'
    • Edit this file:

source ~/CRAB_2_4_2/crab.sh

 
  • submit crab job
crab -create -submit
Line: 15 to 51
  links:
Added:
>
>
https://cmsweb.cern.ch/dbs_discovery/
 http://hepuser.ucsd.edu/twiki2/bin/view/UCSDTier2/PhysicsAndMCproduction
Added:
>
>
 https://cmsweb.cern.ch/crabconf/files/crab-v2.4.2.html

Revision 22008/12/02 - Main.WarrenAndrews

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Deleted:
<
<
 

goal:

Line: 9 to 8
 implementation:
  • Run ntuplemaker on sample dataset
Added:
>
>
  • submit crab job
crab -create -submit

(wandrews note: see ~/Notes)

links:

http://hepuser.ucsd.edu/twiki2/bin/view/UCSDTier2/PhysicsAndMCproduction https://cmsweb.cern.ch/crabconf/files/crab-v2.4.2.html

 

-- WarrenAndrews - 2008/12/01 \ No newline at end of file

Revision 12008/12/01 - Main.WarrenAndrews

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"

goal: *investigate performance of glidein submission versus direct (condor) submission

implementation:

  • Run ntuplemaker on sample dataset

-- WarrenAndrews - 2008/12/01

 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback