TWiki> Sandbox Web>Sandbox Web? >JohnWeigandSandbox>WeigandGlideinInstallationQuestions (2011/01/04, JohnWeigand)EditAttach

glideinWMS Installation

Overview

The intent of this section is to provide an overview of some of the prerequisites and "things you need to think about" before starting a glideinWMS installation. It covers the following areas.

Order of Installation

During the installation process, several of the glideinWMS services need to contact the other services in order to complete their configuration correctly. Due to this dependency, there is a specific order in which the services must be installed:

  1. WMS Collector
  2. Factory
  3. User Pool Collector
  4. Submit
  5. VO Frontend

Required Software

Before you begin make sure the following software is either installed or available for installation.

Sotware Version WMS collector Factory User collector Submit VO Frontend Comments
glideinWMS v2.4 y y y y y  
Condor v7.2.+ y   y y y SL3 x86 binaries should work everywhere.
For x86_64 OSG, you need to install 32 bit compatibility libraries. It is HIGHLY recommended that the SL3 x86 binaries be used.
OSG client current version y   y y y Needed to generate grid/voms proxies.
Python v2.3.4+ y y y y y  
HTTP server latest version   y     y Apache, TUX or server of choice.
RRDTool V1.1.18+   y     y http://dag.wieers.com/rpm/packages/rrdtool/rrdtool-1.2.18-1.el4.rf.i386.rpm
http://dag.wieers.com/rpm/packages/rrdtool/perl-rrdtool-1.2.18-1.el4.rf.i386.rpm
http://dag.wieers.com/rpm/packages/rrdtool/python-rrdtool-1.2.18-1.el4.rf.i386.rpm
M2Crypto v0.17+   y     y This can be installed using yum, if available. Or via a tarball from http://chandlerproject.org/Projects/MeTooCrypto#Downloads
javascript RRD v0.5.0   y     y http://sourceforge.net/projects/javascriptrrd/files/javascriptrrd/
There are some distributions of this with the flot, in which case you will not have to install flot separately.
Flot v0.6   y     y http://code.google.com/p/flot/downloads/list

Security and authentication

Since Condor is the underlying system for communication between glideinWMS services, the following sections of the Condor manual may prove useful to review when making decisions on the establishment of user accounts and the use of certificates versus proxies in your glideinWMS configuration:

The above section on security is important but rather lengthy. The key parts of that document germaine to this section of glideinWMS are:

  • 3.6.3.1 GSI Authentication
  • 3.6.11 User Accounts in Condor
  • 3.6.12 Privilege Separation
The sections that follow will show the recommended settings and user accounts.

User Accounts

The table below shows the UNIX user accounts that will be required for each service. The column '/sbin/nologin' indicates those accounts that should not allow a user to login as. With the exception of the individual user accounts on the Submit node, all of these accounts must be established prior to installation.

Service User account /sbin/nologin Comments
WMS collector condor y This is a condor installation only and should be installed as root user.
Factory gfactory n This is the account the factory processes will run as. The factory should be installed as this user.
e.g., vo_cms, vo_dzero y A user account is required for each VOFrontend that the Factory is servicing.
User collector condor n This is a condor installation only and should be installed as root user.
Submit condor n This is a condor installation only and should be installed as root user.
individual accounts y These will be your end user accounts used for submitting jobs to the grid via glideinWMS.
VO Frontend e.g., vo_cms, vo_dzero y This is the account the VOFrontend processes will run as. The VOfrontend should be installed as this user.

Certificate/Proxies

The table below identifies the certificates/proxies needed for each service.

Service Certificate Proxy Comments
WMS collector host/service   Used by condor to identify itself.
Factory host/service   Although normally co-located with the WMS collector, the factory owns the Condor schedds.
User collector host/service   Used by condor to identify itself.
Submit host/service   The submit node owns the Condor schedds.
  user proxies All users submitting jobs on the submit node naturally require a proxy for authorization on the CE clusters the pilots will be running on.
VO Frontend   service/user This is the only glideinWMS service that has to use a proxy to identify itself to the other services. In many instances the WMS Collector/Factory will NOT be resident on the same site as the other services and OSG security policy prohibits sending the identity of a certificate off-site.
  pilot proxies These are the proxies that will be used by the glidein pilot jobs.

For those services requiring proxies, you will likely need to install the OSG client software if the proxy generation is to be performed on that platform. As an alternative, depending on how you set up access to the various service nodes, you can install the OSG client on the submit node only and distribute the proxy via, for example, an 'scp' to the other nodes.

Condor Authentication

In order for the various Condor daemons to communicate with one another in a secure manner, the identification of trusted users using their GSI identity (DN of the issuer/subject of a certificate or proxy is required in 2 Condor files. The installer will populate these files, and the necessary attributes where applicable, based on your use of a proxy or certificate.

condor_mapfile

Service WMS collector Factory User collector Submit VO Frontend Comments
WMS Collector cert proxy     Each frontend user
(not the pilots) 		 
User Collector     cert cert Each pilot proxy  
Submit     cert cert    
VO Frontend            

The format of the condor_mapfile will look like this for a WMS collector:

  1. GSI "^\/DC\=org\/DC\=doegrids\/OU\=Services\/CN\=cms\-xen21\.fnal\.gov$" condor
  2. GSI "^\/DC\=org\/DC\=doegrids\/OU\=Services\/CN\=cms\-xen22\.fnal\.gov$" vo_dzero
  3. GSI "^\/DC\=org\/DC\=doegrids\/OU\=Services\/CN\=glidein\/cms\-xen22\.fnal\.gov$" vo_cms
  4. GSI (.*) anonymous
  5. FS (.*) \1
  • 1st line: WMS collector proxy
  • 2nd line: VO Frontend CMS VO user
  • 3rd line: VO Frontend dzero VO user
  • The DN (3rd token) must be represented in a regex format, hence the backslashes.

condor_config.local

Condor file WMS collector User collector Submit VO Frontend Comments
CONDOR_LOCATION/certs/condor_mapfile WMS collector (cert)
all VO Frontends (proxy) 		User collector(cert)
Submit (cert)
Factory pilot (proxy) 		User collector(cert)
Submit (cert) 		n/a
no condor daemons
just clients used 		 
CONDOR_LOCATION/condor_local/condor_config.local (GSI_DAEMON_NAME attribute) WMS collector (cert) User collector(cert)
Submit (cert)
Factory pilot (proxy) 		User collector(cert)
Submit (cert)
Factory pilot (proxy) 		n/a
no condor daemons
just clients used 		 

Condor installs for all services

A installation of Condor will be required for each of the services. The table below identifies the information you will need to know for each of those services.

Data needed WMS collector User collector Submit VOfrontend Comments
Condor user     condor VO User (eg, cms, cdf, etc)  
Condor tarball location Full path zipped tarball Full path zipped tarball Full path zipped tarball Full path zipped tarball Validates condor version
Condor installation location /home/glidein/glidecondor /home/glidein/glidecondor /usr/local/glidein/condor
(why not glidecondor for consistency?) 		VO_USER_HOME/glidecondor Creates directory
Installs condor
Condor admin email ADMIN_EMAIL ADMIN_EMAIL ADMIN_EMAIL VO_USER_EMAIL  
Condor config split (y/n)
(condor_config.local/condor_config) 		y y y y  
GSI (proxy / cert) cert cert cert n/a
(no condor daemons - just clients used) 		 
Certificate/key location hostcert.pem
hostkey.pem 		hostcert.pem
hostkey.pem 		hostcert.pem
hostkey.pem 		n/a
(no condor daemons - just clients used) 		 
Use Quill (y/n) ???   ??? n/a NO CLUE
Name for this service WMScollector UserCollector n/a n/a  
Number of schedds 9 n/a 9 n/a Creates secondary schedds e.g.
Creating /home/glidein/glidecondor/condor_local/schedd_glideins1
Number of slave collectors n/a 5 n/a n/a  
User collector node n/a n/a User collector node n/a  
GCB servers     none n/a  
           

Factory Service

Factory Configuration

Data needed Answer Comments
Factory user    
Pilot proxy USER_PROXY Used in user collector
factory config and log file locations /home/glidein/glideinsubmit  
Web data location /var/www/html/glidefactoy Creates subdirectories:
./monitor
./stage
Web URL http://cms-xen21.fnal.gov/glidefactory/  
Factory name factory_xen21  
Factory instance name v1_6  
Condor location /home/glidein/glidecondor (same as WMS collector condor)  
GCB nodes   CLUELESS
grid-mapfile entries    
GSI_DAEMON_NAME entries    
Use gLexec y/n  
Expose Grid env to user jobs y/n  
Create glidein or just configs glidein / configs  
Selectively use all or some of schedds created by WMS collector.    
     

Factory Filters

These should be a separate script that allows you to add/change/delete filters as needed independent of the installation.

Data needed Answer Comments
Use RESS y/n  
Ress location osg-ress-4.fnal.gov  
Ress constraint StringlistMember("VO:cms",!GlueCEAccessControlBaseRule)  
python filter (int(GlueCEPolicyMaxCPUTime)<(25*60))  
     
Use BDII y/n  
BDII location    
     

VO Frontend Service

VO Frontend Configuration

Data needed Answer Comments
Config/log file location VO_FRONTEND_USER_HOME User is defined is part of condor install.
Do they have to be the same?
Pilot proxy location USER_PROXY_LOCATION Used in user collector
Name eg. cms_vofrontend Could include node name
WMS collector node WMS_COLLECTOR_NODE Does the WMS collector need to be running?
User collector node USER_COLLECTOR_NODE User collector needs to be running to select schedds to monitor.
User collector schedds to monitor select schedds to monitor  
     

VO Frontend Filters

These should be a separate script that allows you to add/change/delete filters as needed independent of the installation.

Data needed Answer Comments
Expression to match jobs It is an arbitrary python boolean expression using the dictionaries
glidein and job
A simple example expression would be:
glidein["attrs"]["GLIDEIN_Site"] in job["DESIRED_Sites"].split(",")
If you want to match all (OK for simple setups),
just specify 1 (the default)
Match string: [1] 1 		 
Attributes for desired sites You need to specify a list of pairs
Each pair has the name and type of the attributes.
The possible typoes are s,i,r,b (for string, int, real, bool)
The example above would had:
[("DESIRED_Sites","s")]
I have computed my best estimate for your match string,
please verify and correct if needed.
Job attributes: [[]] 		 
Jobs to monitor (JobUniverse? ==5)&&(GLIDEIN_Is_Monitor ! TRUE)&&(JOB_Is_Monitor ! TRUE)&&(D=! UNDEFINED)&&(s!=UNDEFINED)  
     

Starting/Stopping Services

Service User Starting Stopping
WMS collector glidein $HOME/glidecondor/start_condor.sh killall condor_master
Factory glidein export X509_USER_PROXY=/home/glidein/grid-security/x509_pilot_proxy
$HOME/glideinsubmit/glidein_v1_6/factory_startup start 		$HOME/glideinsubmit/glidein_v1_6/factory_startup stop
VO Frontend vo_user export X509_USER_PROXY=/home/cms/grid-security/x509_cms_pilot_proxy
$HOME/cms_frontend/frontend_startup start 		$HOME/cms_frontend/frontend_startup stop
User collector glidein $HOME/glidecondor/start_condor.sh killall condor_master
User submit node root /etc/init.d/condor start /etc/init.d/condor stop

ini file

A description of the sections and attributes of the glideinWMS installer ini file is in the Glidein Ini File twiki

-- JohnWeigand - 19 Aug 2009

Edit | Attach | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r5 - 2011/01/04 - 19:57:41 - JohnWeigand
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback