Copied in /etc/yum.repos.d the following repos:
http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/dag.repo
http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/glite-CREAM.repo
http://repository.egi.eu/sw/production/cas/1/current/repo-files/egi-trustanchors.repo

Of course also the OS repo files are needed as well


yum clean all
yum update
yum install ca-policy-egi-core
yum install java-1.6.0-openjdk
yum install xml-commons-apis
yum install glite-CREAM


Installed/updated these RPMs (this won't be needed anymore when glite-CONDOR_utils is released):
http://eticssoft.web.cern.ch/eticssoft/repository/org.glite/glite-info-dynamic-scheduler-condor/1.0.0/noarch/glite-info-dynamic-scheduler-condor-1.0.0-1.noarch.rpm
http://eticssoft.web.cern.ch/eticssoft/repository/org.glite/glite-yaim-condor-utils/5.1.0/noarch/glite-yaim-condor-utils-5.1.0-1.noarch.rpm
http://eticssoft.web.cern.ch/eticssoft/repository/org.glite/org.glite.apel.condor/2.0.6/slc4_ia32_gcc346/glite-apel-condor-2.0.6-2.noarch.rpm

PS: The official instructions are located at http://igrelease.forge.cnaf.infn.it/doku.php?id=doc:guides:devel:install-cream32 but do not include how to install the Condor part.

OSG use of CREAM

OSG is planning in officially suporting CREAM in the near term.

The first step in the process is represented by this planning document.

cvs -d :pserver:anonymous@cdcvs.fnal.gov:/cvs/cd_read_only co -r snapshot_100518_v2plus_Igor_CREAM glideinWMS 

(attached is the siteinfo used on osg-gw-3)

Run yaim:

/opt/glite/yaim/bin/yaim -c -s /root/SiteInfo/site-info.def -n creamCE -n CONDOR_utils



Because in the WN the proper environment is not already defined, it was needed
to customize the CREAM JobWrapper? .
This was done following the instructions reported at:

http://grid.pd.infn.it/cream/field.php?n=Main.HowToCustomizeTheCREAMJobWrapper

("Instructions for CREAM CE >= 1.6 (glite-ce-cream >= 1.12)" section)

adding the following 2 lines:

export OSG_GRID=/code/osgcode/wn-client
. $OSG_GRID/setup.sh

after:

for((idx=0; idx<${#__environment[*]}; idx++)); do
eval export ${__environment[$idx]}
done


Because the UCSD Condor installation requires a special arguments setting
in the Condor submit file, /opt/glite/bin/condor_submit.sh was mofified changing:

arguments = $arguments

into:

arguments = -wrapper_iwd $_CONDOR_SCRATCH_DIR $arguments

The v7.5.X development series has added additional improvements, so anyone looking for maximum performance should use that.

Service startup

• Supported platform: R3.1 production is SL4 only, R3.2 supports SL5, too.

For R3.2 os SL5, see:

http://igrelease.forge.cnaf.infn.it/doku.php?id=doc:guides:devel:install-cream32

Test run

A test on glidein-c against osg-gw-3 (on 2010/02/19):

• Ran 10k 30 minute jobs against CREAM on osg-gw-3, using Condor-G v7.5.0 on glidein-c
• CREAM finished in about 7 hours; for comparison, a similar GT2 run took about 9 hours.
• Condor-G numbers are reality close to the CE ones under CREAM
  • Under CREAM Condor-G was almost always reporting the same numbers as the CE
  • For comparison, under GT2 Condor-G thought jobs were running almost 5 hours after they finished on the CE.
• The test run ran only ~8k jobs; ~2k jobs got held
  • ~1.4k failed while staging in the input sandbox
  • ~600 failed while staging out the output sandbox
  • For comparison, under GT2 I observed no held jobs
• CREAM jobs took much longer than 30min to complete
  • Under CREAM, over half took more than 90mins, with a non negligible fraction taking over 3 minutes
    This may be related to the heavy load on the Condor-G node, due to I/O handled by the gridFTP server
  • For comparison, under GT2 all jobs finished withing 31 minutes
• For detailed results, see cream_10k.ods.
  
  

During the test, the load on the client became very high:
top - 21:08:42 up 310 days, 9:30, 1 user, load average: 41.64, 40.04, 38.89
Likely due to all the gridFTP sessios that were calling back.

- Beacuse we want to reuse an existing installation of Condor, we had to
a) delete /etc/profile.d/condor.sh
b) modify /etc/profile.d/grid-env.sh
gridpath_append "PATH" "/opt/condor-7.2.4/sbin"
gridenv_set "CONDOR_SUBMIT" "/usr/bin/condor_submit"
gridenv_set "CONDOR_STATUS" "/usr/bin/condor_status"
gridenv_set "CONDOR_RM" "/usr/bin/condor_rm"
gridenv_set "CONDOR_Q" "/usr/bin/condor_q"
gridenv_set "CONDOR_CONFIG_LOCAL" "/condor/release//etc/osg-gw-3.local"
gridenv_set "CONDOR_CONFIG" "/condor/release/etc/condor_config"
gridenv_set "CONDORG_INSTALL_PATH" "/condor/release"

- Because the UCSD Condor installation requires a special arguments setting
in the Condor submit file, /opt/glite/bin/condor_submit.sh was mofified changing:

arguments = $arguments

into:

arguments = -wrapper_iwd $_CONDOR_SCRATCH_DIR $arguments

- service tomcat5 stop
- /opt/glite/etc/init.d/glite-ce-blahparser stop
- service tomcat5 start

Client configuration

Condor 7.5.0 was used to run against CREAM.

The condor submit file had the following lines in it:

Universe = grid
grid_resource = cream https://osg-gw-3.t2.ucsd.edu:8443/ce-cream/services/CREAM2 condor osg-gw-3

Installation on osg-gw-3

Massimo Sgaravatto helpped with the installation, resulting in the following instructions:

Installation

- Copied in /etc/yum.repos.d the following repos:

http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/dag.repo
http://grid-deployment.web.cern.ch/grid-deployment/glite/repos/3.2/lcg-CA.repo
http://grid-deployment.web.cern.ch/grid-deployment/glite/repos_beta/3.2/beta-glite-CREAM.repo

Of course also the OS repo files are needed as well

- yum clean all
- yum update
- yum install java-1.6.0-openjdk tomcat5
- yum install lcg-CA
- yum install xml-commons-apis
- yum install glite-CREAM
- yum install perl-XML-Simple


- Installed/updated these RPMs:

http://eticssoft.web.cern.ch/eticssoft/repository/org.glite/glite-info-dynamic-scheduler-condor/1.0.0/noarch/glite-info-dynamic-scheduler-condor-1.0.0-1.noarch.rpm
http://eticssoft.web.cern.ch/eticssoft/repository/org.glite/org.glite.yaim.condor-utils/4.1.1/noarch/glite-yaim-condor-utils-4.1.1-1.noarch.rpm
http://eticssoft.web.cern.ch/eticssoft/repository/org.glite/org.glite.apel.condor/2.0.6/slc4_ia32_gcc346/glite-apel-condor-2.0.6-2.noarch.rpm
http://eticssoft.web.cern.ch/eticssoft/repository/org.glite/org.glite.ce.blahp/1.14.0/sl5_x86_64_gcc412/glite-ce-blahp-1.14.0-5.sl5.x86_64.rpm

Configuration and customizations:

- Customized the conf files (/root/SiteInfo/site-info.def and
/root/SiteInfo/services/glite-creamce)
siteinfo.def file (/opt/glite/yaim/examples/siteinfo/site-info.def)


- Run yaim:

/opt/glite/yaim/bin/yaim -c -s /root/SiteInfo/site-info.def -n creamCE -n CONDOR_utils


- Some changes in /opt/glite/etc/blah.config were manually needed:

Modified:

condor_binpath=/usr/bin
bnotifier_path=/opt/glite/bin/BNotifier
purge_interval=5000000
export CONDOR_CONFIG=/condor/release/etc/condor_config


Added:

blah_disable_wn_proxy_renewal=yes
bnotifier_debug_logfile=/var/tmp/bnotifier.log
bnotifier_debug_level=2
blah_children_restart_interval=0

/opt/glite/etc/blah.config.ucsd is a refernce file with these changes


Because in the WN the proper environment is not already defined, it was needed
to customize the CREAM JobWrapper? .
This was done following the instructions reported at:

http://grid.pd.infn.it/cream/field.php?n=Main.HowToCustomizeTheCREAMJobWrapper

adding the following 2 lines:

export OSG_GRID=/code/osgcode/wn-client
. $OSG_GRID/setup.sh


after:

for((idx=0; idx<${#__environment[*]}; idx++)); do
eval export ${__environment[$idx]}
done


- Because the UCSD Condor installation requires a special arguments setting
in the Condor submit file, /opt/glite/bin/condor_submit.sh was mofified changing:

arguments = $arguments

into:

arguments = -wrapper_iwd $_CONDOR_SCRATCH_DIR $arguments


- service tomcat5 stop
- /opt/glite/etc/init.d/glite-ce-blahparser stop
- service tomcat5 start

CREAM Tests

Introduction

OSG is evaluating various Grid Compute Elements (aka Gatekeepers) to determine which ones it should support. Feature list, ease of use, performance and reliability are all important aspects of the evaluation.

CREAM is a Grid Compute Element developed in EUROPE by gLite:

• The project home page is located at http://grid.pd.infn.it/cream/
• The project code distribution is located at http://glite.web.cern.ch/glite/packages/R3.1/deployment/glite-CREAM/glite-CREAM.asp
• Supported platform: production is SL4 only, SL5 being certified

Client tests

OSG relies hevily on Condor-G for Grid submissions, so it was used for the client testing.

Condor-G only added functional support for CREAM in v7.3.2. Unfortunately that release has VOMS-related bugs that make it unusable in practice, so the upcoming 7.4.0 will be the first one to be usable.

Our group has however collaborated with the Condor team for a long time to get to a release of Condor-Gwith CREAM support. Details can be found on the CREAM Support for CMS page.

Server tests

CREAM can only be installed from RPMs on a SL4 node.
Thawan Kooburat from Madison has done it before in the US and is willing to help.

Waiting for Terrence to get the needed node.

-- IgorSfiligoi - 2009/11/03