Difference: GlideinWMSInstallationNotesFromChrisBoynton (1 vs. 11)

Revision 112009/11/10 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"

Installation Walkthrough and Notes on the GlideinWMS System

Line: 25 to 25
 

STEP 2: Install VDT, GlideinWMS Collector and Factory (on server #1)

Part 1: Check and Install Prerequisite Software

Changed:
<
<
First, we need to verify all the required software is loaded on our system.
>
>
First, we need to verify all the required software is loaded on our system. I performed all of these checks as the root user.

su - root

 
  • A reasonably recent Linux OS (RH/SL4 nad RH/SL5 tested at press time).
cat /etc/redhat-release
Scientific Linux SL release 5.3 (Boron)
Line: 40 to 44
 
Changed:
<
<
    • Installed from the GlideinWMS installer by selecting "Components" and then "Web server".
>
>

yum install httpd
apachectl start

 
Changed:
<
<
    • This is NOT (?) required if you install: pool Collector AND User Schedd
    • Installed from the GlideinWMS installer by selecting "Components" and then "Base Condor installation".
>
>
    • I ended up using version 7.3.1 because 7.3.2 seems to have some bugs.
    • I just grabbed the uncompressed tarball and stuck it in the /root directory.
 
  • The RRDTool package (v1.2.18 or later)
    • The GlideinWMS installer tells us what to install by selecting "Components" and then "rrdtool graphics package".
    • Since no installation takes place, you have to go find the files on the web. The website that lists the installation RPMS points to broken links. I found the RPMs for my 64-bit installation here:
Line: 55 to 62
 
Added:
>
>
  • You also need the (undocumented) FLOT libraries because the installers will ask for it.
wget http://flot.googlecode.com/files/flot-0.6.tar.gz
 

Part 2: Install VDT Grid Software

First, we need to download the glideinWMS software. We're going to use a specific snapshot.

Line: 72 to 82
 Choose [8] for Components and then [a] for OSG VDT Client.
Changed:
<
<
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/opt/vdt]
Directory '/opt/vdt' does not exist, should I create it?: (y/n)
What pacman version should I use?: [3.28]
What VDT cache should I use?: [http://vdt.cs.wisc.edu/vdt_200_cache]
Which platform do you want to use (leave empty for autodetect):
Do you agree to the licenses? [y/n] y
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Do you want OSG_VDT_BASE defined globally?: (y/n) y
>
>
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/opt/vdt]
Directory '/opt/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [3.28]
What VDT cache should I use?: [http://vdt.cs.wisc.edu/vdt_200_cache]
Which platform do you want to use (leave empty for autodetect):
Do you agree to the licenses? [y/n] y
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Do you want OSG_VDT_BASE defined globally?: (y/n) y
 

Part 3: Install GlideinWMS Collector

Line: 95 to 105
 cp -fr glideinWMS/ /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/glideinWMS
Changed:
<
<
Copy our personal certificate to the gfactory's home directory. (MORE ABOUT HOW TO GET THIS LATER)
>
>
We need to generate a certificate key file from the certificate. Do this to create the key file.

. /opt/vdt/setup.sh
openssl pkcs12 -in cert.p12 -out cert.key

Now remove the public keys from the private key file (just delete them from the file):

vi cert.key

Copy our personal certificate to the gfactory's home directory.

 

cp cert.p12 /home/gfactory
cp cert.key /home/gfactory
chown -R gfactory:gfactory /home/gfactory/cert.*

Line: 182 to 204
 

mkdir /var/www/html/glidefactory
chown gfactory:gfactory /var/www/html/glidefactory

Changed:
<
<

If you haven't already started Apache HTTPD as root, it would be a good time to do that. You can just use "yum install httpd" as root if you don't already have it installed.

apachectl start

Let's login as gfactory user and unpack flot and javascriptrrd:

>
>
Let's login as gfactory user and unpack flot and javascriptrrd:
  su - gfactory
unzip javascriptrrd-0.4.2.zip
gunzip -c flot-0.6.tar.gz | tar xvf -
Line: 235 to 248
 Let's also restart the Factory (BUG in start script requires us to be in the correct directory):
Changed:
<
<
cd /home/gfactory/glideinsubmit/glidein_v1_0

./factory_startup start

>
>
cd /home/gfactory/glideinsubmit/glidein_v1_0
./factory_startup start
 

If the factory fails to start, consult the factory's log directory:

Line: 251 to 262
 

Part 1: Check and Install Prerequisite Software

Simply follow the same instructions above in Step 2, Part 1.

Deleted:
<
<

Part 2: Install VDT Grid Software

 
Changed:
<
<
Simply follow the same instructions above in Step 2, Part 2.

Part 3: Install GlideinWMS Frontend

The next system needed is the GlideinWMS frontend system. We will install this on my system named SL2. This system needs most of the pre-requisites of SL1.

>
>
This system needs most of the pre-requisites of the previous server.
 
  • Any Condor-supported OS
  • The OSG client software
  • The Condor distribution
Line: 266 to 273
 
  • The javascriptRRD package (0.4 or later)
  • The perl-Time-HiRes rpm
  • The glideinWMS software
Added:
>
>

Part 2: Install VDT Grid Software

 
Changed:
<
<
Let's assume we've already installed these components. If not, follow the steps in part 1.

We need a user for condor to run under.

su - root
groupadd -g 5003 condor
useradd -c "Condor Daemon" -g 5003 -m -s /bin/bash -u 5003 condor

BEGIN WARNING -- MISSING CONDOR INSTALL (DO THIS PROXY STUFF ON OTHER SERVER AND COPY IT OVER)

>
>
Simply follow the same instructions above in Step 2, Part 2.

Part 3: Install GlideinWMS Frontend

 
Changed:
<
<
We also need our proxy certificate to identify ourselves (this could be the same as the one generated on SL1):
>
>
The next system needed is the GlideinWMS frontend system.
 
Changed:
<
<
su - frontend
mkdir .globus
scp -c blowfish -P 4221 gfactory@sl1:/home/gfactory/cert.p12 /home/frontend/
. /home/frontend/vdt/setup.sh
openssl pkcs12 -in cert.p12 -out cert.key

Now remove the public keys from the private key file:

>
>
So, of course, we need a copy of glideinWMS from CVS:
 
Changed:
<
<
vi cert.key

Now the permissions need to be adjusted on these files in order to generate the proxy:

chmod 644 /home/frontend/cert.p12
chmod 400 /home/frontend/cert.key

>
>
su - root
cvs -d :pserver:anonymous@cdcvs.fnal.gov:/cvs/cd_read_only co -r snapshot_091029 glideinWMS
 
Changed:
<
<
Run this command to generate the proxy for 500 hours:
>
>
We need a user for condor to run under.
 
Changed:
<
<
/home/frontend/vdt/glite/bin/voms-proxy-init -cert /home/frontend/cert.p12 -key /home/frontend/cert.key -out /home/frontend/.globus/x509_service_proxy -valid 500:0.0
>
>
su - root
groupadd -g 5003 condor
useradd -c "Condor Daemon" -g 5003 -m -s /bin/bash -u 5003 condor
 
Changed:
<
<
Now, let's make the cert and key owned by root to prevent any bad stuff from happening:
>
>
Since we already generated a proxy, we can copy it from Server #1.
 
Changed:
<
<
su - root
chown root:root /home/frontend/cert.p12
chown root:root /home/frontend/cert.key
>
>
scp -c blowfish -P 22 root@devg-4:/home/gfactory/.globus/x509_service_proxy /root/x509_service_proxy
 
Changed:
<
<
Proxy was created and installed successfully!

END WARNING

Now we need to move the proxy certificate into a global location:

>
>
Let's copy the proxy certificate into a global location:
  mkdir /etc/grid-security/
cp x509_service_proxy /etc/grid-security/
Changed:
<
<
Let's install the "condor collector" and "condor schedd" AT THE SAME TIME (by selecting 4,5) as root.
>
>
As root, let's install the "condor collector" (aka "pool collector") and "condor schedd" (aka "user schedd") AT THE SAME TIME (by selecting both options simeltaneously as 4,5):
 
Changed:
<
<
su - root
/root/glideinWMS/install/glideinWMS_install
Please select: 4,5
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/opt/vdt]
Directory '/opt/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
What VDT cache should I use?: [http://vdt.cs.wisc.edu/vdt_1101_cache]
Which platform do you want to use (leave empty for autodetect):
Do you agree to the licenses? [y/n] y
Would you like to setup daily rotation of VDT log files? y
Do you want to update the CA certification revocation lists (CRLs) automatically? [y/n] y
Where would you like to install CA files? r
Do you want to automatically update your CA Certificates? [y/n] y
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Which user should Condor run under?: [condor]
Where do you have the Condor tarball? /root/condor-7.3.2-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/opt/glidecondor]
Directory '/opt/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.ext
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
Will you be using a proxy or a cert? (proxy/cert) proxy
Where is your proxy located?: /etc/grid-security/x509_service_proxy
DN:
How many slave collectors do you want?: [5]
What name would you like to use for this pool?: [My pool] devg5-pool
Do you want to use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
GCB node:
How many secondary schedds do you want?: [9]
>
>
su - root
glideinWMS/install/glideinWMS_install
Please select: 4,5
Which user should Condor run under?: [condor]
Where do you have the Condor tarball? condor-7.3.1-linux-x86-rhel5-dynamic.tar.gz
Where do you want to install it?: [/opt/glidecondor]
Directory '/opt/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.myext
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
Do you want to get it from VDT?: (y/n) y
Do you have already a VDT installation?: (y/n) y
Where is the VDT installed?: [/opt/vdt]
Will you be using a proxy or a cert? (proxy/cert) proxy
Where is your proxy located?: /etc/grid-security/x509_service_proxy
DN:
How many slave collectors do you want?: [5]
What name would you like to use for this pool?: [My pool] pool-devg5
What port should the collector be running?: [9618]
Do you want to use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
GCB node:
How many secondary schedds do you want?: [9]
 

Let's restart condor:

Line: 337 to 322
 su - root
groupadd -g 5002 frontend
useradd -c "Frontend Daemon" -g 5002 -m -s /bin/bash -u 5002 frontend
Changed:
<
<
Make sure to get a copy of condor into the frontend's home directory:
>
>
Let's copy the proxy (because we need a second copy) into the frontend user's proxy directory:
 
Changed:
<
<
cp condor-7.3.2-linux-x86_64-rhel5-dynamic.tar.gz /home/frontend/
chown frontend:frontend /home/frontend/condor-7.3.2-linux-x86_64-rhel5-dynamic.tar.gz
>
>
mkdir /home/frontend/.globus
cp x509_service_proxy /home/frontend/.globus/x509_service_proxy
chown -R frontend:frontend /home/frontend/.globus
 
Changed:
<
<
Copy the glideinWMS software into the frontend user's directory from CVS:
>
>
Make sure to get a copy of condor into the frontend's home directory:
 
Changed:
<
<
su - frontend
cvs -d :pserver:anonymous@cdcvs.fnal.gov:/cvs/cd_read_only co -r v2_1 glideinWMS
>
>
cp condor-7.3.1-linux-x86-rhel5-dynamic.tar.gz /home/frontend/
chown frontend:frontend /home/frontend/condor-7.3.1-linux-x86-rhel5-dynamic.tar.gz
 
Changed:
<
<
Now we need the HTTP server installation:
>
>
Copy the glideinWMS software into the frontend user's directory from CVS:
 
Changed:
<
<
su - root
yum install httpd
>
>
cp -fr glideinWMS /home/frontend/
chown -R frontend:frontend /home/frontend/glideinWMS
 
Changed:
<
<
HTTPD is installed
>
>
Copy over the JavaScriptRRD? package into the frontend's home directory:

cp javascriptrrd-0.4.2.zip /home/frontend/
chown frontend:frontend /home/frontend/javascriptrrd-0.4.2.zip

 
Changed:
<
<
Copy over the JavaScriptRRD? package into the frontend's home directory:
>
>
Finally, copy over our flot download:
 
Changed:
<
<
cp javascriptrrd-0.4.2.zip /home/frontend/
chown frontend:frontend /home/frontend/javascriptrrd-0.4.2.zip
su - frontend
unzip javascriptrrd-0.4.2.zip
>
>
cp flot-0.6.tar.gz /home/frontend/flot-0.6.tar.gz
chown frontend:frontend /home/frontend/flot-0.6.tar.gz
 
Changed:
<
<
JavaScriptRRD is installed

Let's also install Flot:

>
>
Let's also create a directory for the web files:
 
Changed:
<
<
wget http://flot.googlecode.com/files/flot-0.5.tar.gz
gunzip -c flot-0.5.tar.gz | tar xvf -
>
>
mkdir /var/www/html/vofrontend
chown frontend:frontend /var/www/html/vofrontend
 
Changed:
<
<
Flot was installed successfully!

Let's create a directory for the host and config log files:

>
>
Let's switch over to the frontend user and decompress some of our files:
 
Added:
>
>
su - frontend
unzip javascriptrrd-0.4.2.zip
gunzip -c flot-0.6.tar.gz | tar xvf - Let's create a directory for the host and config log files:
 mkdir /home/frontend/frontstage
Changed:
<
<
Let's also create a directory for the web files:
>
>
Now comes time to install the frontend. Let's begin:
 
Changed:
<
<
su - root
mkdir /var/www/html/vofrontend
chown frontend:frontend /var/www/html/vofrontend
>
>
/home/frontend/glideinWMS/install/glideinWMS_install
Please select: 7
Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/frontend/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/frontend/flot
Where is your proxy located?: /home/frontend/.globus/x509_service_proxy
What is the mapped name?: me@myemail.myext
Where will you host your config and log files?: [/home/frontend/frontstage]
Where will the web data be hosted?: [/var/www/html/vofrontend]
What Web URL will you use?: [http://devg-5.t2.ucsd.edu/vofrontend/]
Give a name to this VO Frontend?: [myVO-devg-5] vo-devg5
Give a name to this VO Frontend instance?: [v1_0]
What node is the WMS collector running?: devg-4.t2.ucsd.edu
What is the classad identity of the glidein factory?: [gfactory@devg-4.t2.ucsd.edu]
Collector name(s): [devg-5.t2.ucsd.edu:9618]
Do you want to monitor all of them?: (y/n) y
What kind of jobs do you want to monitor?: [(JobUniverse? ==5)&&(GLIDEIN_Is_Monitor ! TRUE)&&(JOB_Is_Monitor ! TRUE)]
Give a name to the main group: [main]
Match string: [True]
Job attributes: []
Factory attributes: []
Do you want to use is to submit glideins: (y/n) [y] y
An empty entry means you are done. ... proxy fname:
DN (leave empty when finished):
Do you want to use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
Do you want to expose the Grid env. to the user jobs?: (y/n) y
Do you want to create the VO Frontend instance (as opposed to just the config file)?: (y/n) [n] n
 
Changed:
<
<
Finished with directory setup!

Now comes time to install the frontend. Let's begin:

>
>
TODO:
 
Changed:
<
<
su - frontend
/home/frontend/glideinWMS/install/glideinWMS_install
Please select: 7
Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/frontend/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/frontend/flot
Where is your proxy located?: /etc/grid-security/x509_service_proxy
Where will you host your config and log files?: [/home/frontend/frontstage]
Where will the web data be hosted?: [/var/www/html/vofrontend]
What Web URL will you use?: [http://devg-5.t2.ucsd.edu/vofrontend/]
Give a name to this VO Frontend?: [myVO-devg-5] devg5-vo
Give a name to this VO Frontend instance?: [v1_0]
What node is the WMS collector running?: devg-4.t2.ucsd.edu
Collector name(s): [devg-5.t2.ucsd.edu]
Do you want to monitor all of them?: (y/n) y
What kind of jobs do you want to monitor?: [(JobUniverse? ==5)&&(GLIDEIN_Is_Monitor ! TRUE)&&(JOB_Is_Monitor ! TRUE)]
Give a name to the main group: [main]
Match string: [True]
Job attributes: []
Factory attributes: []
Do you want to use is to submit glideins: (y/n) [y] y
An empty entry means you are done. ... proxy fname:
DN (leave empty when finished):
Do you want to use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
Do you want to expose the Grid env. to the user jobs?: (y/n) y
Do you want to create the VO Frontend instance (as opposed to just the config file)?: (y/n) [n] n
>
>
Using DN '/DC=org/DC=doegrids/OU=People/CN=Benjamin C. Boynton 991832'
Make sure this DN is in the WMS collector grid-mapfile
  Ok, now add these lines to .bashrc:
Changed:
<
<
vi ~/.bashrc
export X509_USER_PROXY=/etc/grid-security/x509_service_proxy
>
>
vi ~/.bashrc
. /opt/vdt/setup.sh;
export X509_USER_PROXY=/etc/grid-security/x509_service_proxy;
 

Let's create the VO Frontend instance:

Line: 408 to 390
 Let's also restart the VO Frontend:
Changed:
<
<
/home/frontend/frontstage/frontend_devg5-vo-v1_0/frontend_startup stop
/home/frontend/frontstage/frontend_devg5-vo-v1_0/frontend_startup start
>
>
cd /home/frontend/frontstage/frontend_vo-devg5-v1_0/
./frontend_startup start
 

If the frontend fails to start, consult the frontend's log directory:

Changed:
<
<
ls /home/frontend/frontstage/frontend_frontend-sl2-v1_0/log
>
>
ls /home/frontend/frontstage/frontend_vo-devg5-v1_0/log
 

Revision 102009/11/09 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"

Installation Walkthrough and Notes on the GlideinWMS System

Changed:
<
<
DISCLAIMER: Any information you read on this page is a lie. Actually, it isn't but you should treat it as such. These are my personal notes so YMMV! I do not want to be held responsible for your destruction of the world. Having said all of that, enjoy! :)

STEP 1: Obtain, install, and update a Linux distribution

>
>
DISCLAIMER: These are my personal notes, so don't trust them. It's always best to rely on your own notes. You are welcome to use my notes to make your own notes. Also, don't hold me accountable if something goes wrong or you have any problems whatsoever. You've been warned and now are continuing to read at your own risk. Now, here's the good stuff!

Installation Outline

This is an extremely complex installation process for the first timer. Here is a brief outline of all of the steps.

  • STEP 1: Obtain, install, and update a Linux distribution (on ALL servers)
  • STEP 2: Install VDT, GlideinWMS Collector and Factory (on server #1)
    • Part 1: Check and Install Prerequisite Software
    • Part 2: Install VDT Grid Software
    • Part 3: Install GlideinWMS Collector
    • Part 4: Install GlideinWMS Factory
  • STEP 3: Install VDT, GlideinWMS Frontend (on server #2)
    • Part 1: Check and Install Prerequisite Software
    • Part 2: Install VDT Grid Software
    • Part 3: Install GlideinWMS Frontend

STEP 1: Obtain, install, and update a Linux distribution (on ALL servers)

  I used Scientific Linux 5.3 from: http://www.scientificlinux.org/
Changed:
<
<
You can also use Redhat Enterprise 5.3 or CentOS 5.3. My installation was on two VMWare Workstation 6.5.3 virtual machines named SL#1 and SL#2. I selected that they would have 512 MiB? of RAM and up to 16 GiB? hard disks. To use GlideinWMS, there is no requirement to use a 64-bit system. You can use a 32-bit and all of the software will be compatible. The reverse is not necessarily true, but I have not confirmed this yet. It's also a good idea to update the system with the latest recommended patches and security updates before continuing any futher. I also reconfigured /etc/ssh/sshd_config to use a different port and adjusted the VM to acquire an IP address from my router in order to make my virtual machines accessible remotely through SSH. I had to change /etc/hosts to list the names of my systems as FQDNs, aliases, and their local IP addresses. Also, I had to edit /etc/resolv.conf to setup DNS after switching to a static IP in the GUI since it wasn't getting it from DHCP anymore.

STEP 2: Install GlideinWMS

>
>
You can also use Redhat Enterprise 5.3 or CentOS 5.3. My installation was on two VMWare Workstation 6.5.3 virtual machines named SL1 and SL2. I selected that they would have 512 MiB? of RAM and up to 16 GiB? hard disks. To use GlideinWMS, there is no requirement to use a 64-bit system. You can use a 32-bit and all of the software will be compatible. The reverse is not necessarily true, but I have not confirmed this yet. It's also a good idea to update the system with the latest recommended patches and security updates before continuing any futher. I also reconfigured /etc/ssh/sshd_config to use a different port and adjusted the VM to acquire an IP address from my router in order to make my virtual machines accessible remotely through SSH. I had to change /etc/hosts to list the names of my systems as FQDNs, aliases, and their local IP addresses. Also, I had to edit /etc/resolv.conf to setup DNS after switching to a static IP in the GUI since it wasn't getting it from DHCP anymore.

STEP 2: Install VDT, GlideinWMS Collector and Factory (on server #1)

 

Part 1: Check and Install Prerequisite Software

First, we need to verify all the required software is loaded on our system.

Line: 41 to 55
 
Changed:
<
<

Part 2: Install GlideinWMS Collector (my SL1)

>
>

Part 2: Install VDT Grid Software

 
Changed:
<
<
Ok. Let's proceed with installing the collector on SL1. First we need a new user to own the installation. Let's call them gfactory.
>
>
First, we need to download the glideinWMS software. We're going to use a specific snapshot.
 
Changed:
<
<
su - root
groupadd -g 5001 gfactory
useradd -c "GFactory Daemon" -g 5001 -m -s /bin/bash -u 5001 gfactory
>
>
su - root
cvs -d :pserver:anonymous@cdcvs.fnal.gov:/cvs/cd_read_only co -r snapshot_091029 glideinWMS
 
Changed:
<
<
Let's download and copy condor to the gfactory's home directory:
>
>
Now, let's install VDT as root.
 
Changed:
<
<
mv condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz /home/gfactory/
cd /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
>
>
glideinWMS/install/glideinWMS_install
 
Changed:
<
<
We want to begin the installation with our new user, MAKE SURE TO DO THIS AS THE gfactory USER. The installation adds a cron job for gfactory, but the linux OS may not support cron jobs from the root user. Here is how to start the installation:
>
>
Choose [8] for Components and then [a] for OSG VDT Client.
 
Changed:
<
<
mv glideinWMS/ /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/glideinWMS
su - gfactory
cd glideinWMS/install
./glideinWMS_install
>
>
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/opt/vdt]
Directory '/opt/vdt' does not exist, should I create it?: (y/n)
What pacman version should I use?: [3.28]
What VDT cache should I use?: [http://vdt.cs.wisc.edu/vdt_200_cache]
Which platform do you want to use (leave empty for autodetect):
Do you agree to the licenses? [y/n] y
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Do you want OSG_VDT_BASE defined globally?: (y/n) y
 
Changed:
<
<
Choose the option "glideinWMS Collector" and then let's answer the questions for the installer interactively. Ignore warnings/errors about CA certificates not being installed. The VDT installer doesn't do it, but the glideWMS installer does. Here are the installer answers:
>
>

Part 3: Install GlideinWMS Collector

Let's proceed with installing the collector on SL1. First we need a new user to own the installation. Let's call them gfactory.

 
Changed:
<
<
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/home/gfactory/vdt] /home/gfactory/vdt
Directory '/home/gfactory/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
>
>
su - root
groupadd -g 5001 gfactory
useradd -c "GFactory Daemon" -g 5001 -m -s /bin/bash -u 5001 gfactory
 
Changed:
<
<
This next question is a trick question, don't type anything here, just press enter or you'll get an error.
>
>
Let's download (http://www.cs.wisc.edu/condor/) and copy condor to the gfactory's home directory:
 
Changed:
<
<
Which platform do you want to use (leave empty for autodetect):
WARNING: It appears that SELinux is enabled on this computer. ... Please press enter to continue the installation, or control-c to cancel.
Do you agree to the licenses? [y/n] y
Where would you like to install CA files? l
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Where do you have the Condor tarball? /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/home/gfactory/glidecondor]
Directory '/home/gfactory/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.com
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
>
>
cp condor-7.3.1-linux-x86-rhel5-dynamic.tar.gz /home/gfactory/
cd /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/condor-7.3.1-linux-x86-rhel5-dynamic.tar.gz
 
Changed:
<
<
In order for the Factory to submit to the grid, we need a proxy on all of the servers. However, we needed VDT to be installed before this is possible. Well, at least the minimal VDT is now installed, so we can use VOMS (Virtual Organization Membership Service) to create the proxy. Let's pause our installation in this window and open up another terminal window. In this new terminal, we will generate the proxy:
>
>
Let's copy the glideinWMS we already downloaded as root to the gfactory user's home directory.

cp -fr glideinWMS/ /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/glideinWMS

Copy our personal certificate to the gfactory's home directory. (MORE ABOUT HOW TO GET THIS LATER)

cp cert.p12 /home/gfactory
cp cert.key /home/gfactory
chown -R gfactory:gfactory /home/gfactory/cert.*

 
Changed:
<
<
Using root, I put my certificate file cert.p12 in the gfactory directory, then I switched back to gfactory.
>
>
We should update our gfactory user's environment to point at the VDT we just installed. We put it in .bashrc so that the environment is affected in both interactive and non-interactive bash shells.
 
Changed:
<
<
su - root
cd <the directory where the certificate is stored>
cp cert.p12 /home/gfactory
exit
su - gfactory
>
>
vi /home/gfactory/.bashrc
 
Changed:
<
<
Now we need to source the VDT setup script in order to work with the VDT software.
>
>
Let's add the following at the bottom of the file under "# User specific aliases and functions":
 
Changed:
<
<
. /home/gfactory/vdt/setup.sh
>
>
. /opt/vdt/setup.sh;
 
Changed:
<
<
In order to initialize the voms proxy, I need the private key out of my cert. I was able to get my private key using this command and I saved it to a file called cert.key:
>
>
In order for each of the servers to communicate on the grid, we need a valid proxy on all of the servers. The proxy allows the servers to act on behalf of you. However, we needed VDT Grid Software to be installed before this is possible. Since at least the minimal VDT is now installed, we can use VOMS (Virtual Organization Membership Service) to create the proxy.

In order to create a proxy, you first need your personal certificate. I received my certificate by my web browser after I contacted my sponsor who is already authenticated for instructions and applied on the website which distributes them. If you're not trying to get authenticated on someone else's grid, you can probably just make your own. More details on how to do that later...

We will now generate the proxy using my personal certificate:

Using root, I put my certificate file cert.p12 in the gfactory directory, then I switched into the gfactory user's account.

 
Added:
>
>
cp cert.p12 /home/gfactory
exit
su - gfactory In order to initialize the voms proxy, I need the private key out of my cert. I was able to get my private key using this command and I saved it to a file called cert.key:
 openssl pkcs12 -in cert.p12 -info
Line: 99 to 134
 chmod 644 /home/gfactory/cert.p12
chmod 400 /home/gfactory/cert.key
Changed:
<
<
Run this command to generate the proxy for 500 hours:
>
>
Let's make a directory to hold the proxy:
 
Changed:
<
<
/home/gfactory/vdt/glite/bin/voms-proxy-init -cert /home/gfactory/cert.p12 -key /home/gfactory/cert.key -out /home/gfactory/.globus/x509_service_proxy -valid 500:0.0
>
>
mkdir /home/gfactory/.globus
 
Changed:
<
<
Now, let's make the cert and key owned by root to prevent any bad stuff from happening:
>
>
Run this command to generate the proxy for 1000 hours (you'll need to enter your certificate password):
 
Changed:
<
<
su - root
chown root:root /home/gfactory/cert.p12
chown root:root /home/gfactory/cert.key
>
>
voms-proxy-init -cert /home/gfactory/cert.p12 -key /home/gfactory/cert.key -out /home/gfactory/.globus/x509_service_proxy -valid 1000:0.0 Now, we want to begin the installation of the GlideinWMS Collector. MAKE SURE TO DO THIS AS THE gfactory USER. The installation also adds a cron job for gfactory. Here is how to start the installation:

su - gfactory
~/glideinWMS/install/glideinWMS_install

 
Changed:
<
<
Back in the previous terminal window with our Collector installation, let's continue where we left off.
>
>
Choose the option [1] glideinWMS Collector and then let's answer the questions for the installer interactively. Here are the installer answers:
 
Changed:
<
<
Will you be using a proxy or a cert? (proxy/cert) proxy
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
What name would you like to use for this pool?: [My glideinWMS pool] ChrisB? Pool
How many secondary schedds do you want?: [9] 3
>
>
Where do you have the Condor tarball? /home/gfactory/condor-7.3.1-linux-x86-rhel5-dynamic.tar.gz
Where do you want to install it?: [/home/gfactory/glidecondor]
Directory '/home/gfactory/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.myext
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
Do you want to get it from VDT?: (y/n) y
Do you have already a VDT installation?: (y/n) y
Where is the VDT installed?: [/opt/vdt]
Will you be using a proxy or a cert? (proxy/cert) proxy
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
DN:
What name would you like to use for this pool?: [My glideinWMS pool] devg4pool
What port should the collector be running?: [9618] 9618
How many secondary schedds do you want?: [9]
 

******************************************
WMS collector successfully installed
******************************************

Changed:
<
<

Part 3: Install GlideinWMS Factory (my SL1)

>
>

Part 4: Install GlideinWMS Factory

  First, we must install some additional prerequisite software. We need to get the latest flot tarball, move it into the gfactory's home and extract it. Here is the website to get it from:

http://code.google.com/p/flot/

Added:
>
>

su - root
cp flot-0.6.tar.gz /home/gfactory/flot-0.6.tar.gz
chown gfactory:gfactory /home/gfactory/flot-0.6.tar.gz

Let's also get JavaScriptRRD? . You might be able to find it on:

http://sourceforge.net/projects/javascriptrrd/

cp javascriptrrd-0.4.2.zip /home/gfactory/javascriptrrd-0.4.2.zip
chown gfactory:gfactory /home/gfactory/javascriptrrd-0.4.2.zip

  We will be using the same proxy used in Part 2, so no additional configuration will be necessary for the proxy. This file is:
Line: 131 to 181
 We also need to add a web directory for glidein submission:
Changed:
<
<
su - root
mkdir /var/www/html/glidefactory
chown gfactory:gfactory /var/www/html/glidefactory
>
>
mkdir /var/www/html/glidefactory
chown gfactory:gfactory /var/www/html/glidefactory

If you haven't already started Apache HTTPD as root, it would be a good time to do that. You can just use "yum install httpd" as root if you don't already have it installed.

apachectl start

Let's login as gfactory user and unpack flot and javascriptrrd:

su - gfactory
unzip javascriptrrd-0.4.2.zip
gunzip -c flot-0.6.tar.gz | tar xvf -

 
Changed:
<
<
We are now installing the factory. Select option 2 from the glideinWMS installation script.
>
>
We are now installing the factory. Select option [2] Glidein Factory from the glideinWMS installation script.
 
Changed:
<
<
su - gfactory
cd /home/gfactory/glideinWMS/install
./glideinWMS_install
>
>
~/glideinWMS/install/glideinWMS_install
 
Changed:
<
<
Here are my answers:
>
>
Here are my answers to the interactive installation script:
 
Changed:
<
<
Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/gfactory/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/gfactory/flot
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
Where will you host your config and log files?: [/home/gfactory/glideinsubmit][/home/gfactory/glideinsubmit]
Directory '/home/gfactory/glideinsubmit' does not exist, should I create it?: (y/n) y
Where will the web data be hosted?: [/var/www/html/glidefactory]
What Web URL will you use?: [http://myhost.mydomain/glidefactory/] http://sl1/glidefactory/
Give a name to this Glidein Factory?: [mySites-sl1] gfactory-sl1
Give a name to this Glidein instance?: [v1_0]
What is the Condor base directory?: [/home/gfactory/glidecondor]
The following glidein schedds have been found: ... Do you want to use all of them?: (y/n) y
Do you want to use CCB (requires Condor 7.3.0 or better)?: (y/n) n
Please list all the GCB servers you will be using ... Leave an empty line when finished ... GCB node:
Do you want to use gLExec?: (y/n) y
Force VO frontend to provide its own proxy?: (y/n) [y] y
Do you want to fetch entries from RESS?: (y/n) [n] n
Do you want to fetch entries from BDII?: (y/n) [n] n
Please list all additional glidein entry points, Entry name (leave empty when finished): myentry
Gatekeeper for 'myentry': devg-6.t2.ucsd.edu
RSL for 'myentry':
Work dir for 'myentry': .
Site name for 'myentry': [myentry] mysite
gLExec path for 'myentry': [OSG]
Should glideins use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
Do you want to create the glidein (as opposed to just the config file)?: (y/n) [n]n
To create the glidein, you need to run
/home/gfactory/glideinWMS/creation/create_glidein /home/gfactory/glideinsubmit/glidein_v1_0.cfg/glideinWMS.xml
Configuration files are located in /home/gfactory/glideinsubmit/glidein_v1_0.cfg
Remember to set X509_USER_PROXY before starting the daemon
>
>
Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/gfactory/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/gfactory/flot
Do you want to use such a proxy?: (y/n) y
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
Where will you host your config and log files?: [/home/gfactory/glideinsubmit]
Directory '/home/gfactory/glideinsubmit' does not exist, should I create it?: (y/n) y
Where will the web data be hosted?: [/var/www/html/glidefactory]
What Web URL will you use?: [http://myhost.mydomain/glidefactory/]
Give a name to this Glidein Factory?: [mySites-server-name] gfactory-devg4
Give a name to this Glidein instance?: [v1_0]
What is the Condor base directory?: [/home/gfactory/glidecondor]
The following glidein schedds have been found: ... Do you want to use all of them?: (y/n) y
Do you want to use CCB (requires Condor 7.3.0 or better)?: (y/n) n
Please list all the GCB servers you will be using ... Leave an empty line when finished ... GCB node:
Do you want to use gLExec?: (y/n) y
Force VO frontend to provide its own proxy?: (y/n) [y] y
Do you want to fetch entries from RESS?: (y/n) [n] n
Do you want to fetch entries from BDII?: (y/n) [n] n
Please list all additional glidein entry points, Entry name (leave empty when finished): entry-devg6
Gatekeeper for 'entry-devg6': devg-6.t2.ucsd.edu
RSL for 'myentry':
Work dir for 'myentry': .
Site name for 'myentry': [myentry] entry-devg6
gLExec path for 'myentry': [OSG]
Entry name (leave empty when finished):
Should glideins use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
Do you want to create the glidein (as opposed to just the config file)?: (y/n) [n]

To create the glidein, you need to run
/home/gfactory/glideinWMS/creation/create_glidein /home/gfactory/glideinsubmit/glidein_v1_0.cfg/glideinWMS.xml

Configuration files are located in /home/gfactory/glideinsubmit/glidein_v1_0.cfg

Remember to set X509_USER_PROXY before starting the daemon

 
Changed:
<
<
Ok, now add this line to .bashrc:
>
>
Now edit and add this line to /home/gfactory/.bashrc, logout, and log back in:
 
Changed:
<
<
export X509_USER_PROXY=/home/gfactory/.globus/x509_service_proxy
>
>
export X509_USER_PROXY=/home/gfactory/.globus/x509_service_proxy
exit
su - gfactory
 

Let's create the glidein:

Line: 164 to 232
 killall condor_master
/home/gfactory/glidecondor/start_condor.sh
Changed:
<
<
Let's also restart the Factory:
>
>
Let's also restart the Factory (BUG in start script requires us to be in the correct directory):
 
Changed:
<
<
/home/gfactory/glideinsubmit/glidein_v1_0/factory_startup stop
/home/gfactory/glideinsubmit/glidein_v1_0/factory_startup start
>
>
cd /home/gfactory/glideinsubmit/glidein_v1_0

./factory_startup start

 

If the factory fails to start, consult the factory's log directory:

Changed:
<
<
ls /home/gfactory/glideinsubmit/glidein_v1_0/log
>
>
cat /home/gfactory/glideinsubmit/glidein_v1_0/log/*
 
Changed:
<
<
This concludes the glideinWMS collector and factory installation. I hope it was fun.

Part 4: Install GlideinWMS Frontend System (my SL2)

>
>
This concludes the glideinWMS Collector and Factory installation. Now move along to Server #2.

STEP 3: Install VDT, GlideinWMS Frontend (on server #2)

Part 1: Check and Install Prerequisite Software

Simply follow the same instructions above in Step 2, Part 1.

Part 2: Install VDT Grid Software

Simply follow the same instructions above in Step 2, Part 2.

Part 3: Install GlideinWMS Frontend

  The next system needed is the GlideinWMS frontend system. We will install this on my system named SL2. This system needs most of the pre-requisites of SL1.
  • Any Condor-supported OS
Line: 272 to 349
 su - frontend
cvs -d :pserver:anonymous@cdcvs.fnal.gov:/cvs/cd_read_only co -r v2_1 glideinWMS
Changed:
<
<
Now we need the HTTP server installation:
>
>
Now we need the HTTP server installation:
  su - root
yum install httpd
Line: 327 to 405
 /home/frontend/glideinWMS/creation/create_frontend /home/frontend/frontstage/instance_v1_0.cfg/frontend.xml
Changed:
<
<
Let's also restart the VO Frontend:
>
>
Let's also restart the VO Frontend:
  /home/frontend/frontstage/frontend_devg5-vo-v1_0/frontend_startup stop
/home/frontend/frontstage/frontend_devg5-vo-v1_0/frontend_startup start

Revision 92009/10/29 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"

Installation Walkthrough and Notes on the GlideinWMS System

Line: 190 to 189
 
  • The javascriptRRD package (0.4 or later)
  • The perl-Time-HiRes rpm
  • The glideinWMS software
Deleted:
<
<
Let's assume we've already installed these components. If not, follow the steps in part 1. First, we need to create a user account to host the frontend software installation:
 
Changed:
<
<
su - root
groupadd -g 5002 frontend
useradd -c "Frontend Daemon" -g 5002 -m -s /bin/bash -u 5002 frontend
>
>
Let's assume we've already installed these components. If not, follow the steps in part 1.

We need a user for condor to run under.

su - root
groupadd -g 5003 condor
useradd -c "Condor Daemon" -g 5003 -m -s /bin/bash -u 5003 condor

 
Changed:
<
<
Make sure to get a copy of condor into the frontend's home directory:
>
>
BEGIN WARNING -- MISSING CONDOR INSTALL (DO THIS PROXY STUFF ON OTHER SERVER AND COPY IT OVER)

We also need our proxy certificate to identify ourselves (this could be the same as the one generated on SL1):

 
Changed:
<
<
cp condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz /home/frontend/
chown frontend:frontend /home/frontend/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
>
>
su - frontend
mkdir .globus
scp -c blowfish -P 4221 gfactory@sl1:/home/gfactory/cert.p12 /home/frontend/
. /home/frontend/vdt/setup.sh
openssl pkcs12 -in cert.p12 -out cert.key
 
Changed:
<
<
Copy the glideinWMS software into the frontend user's directory from CVS:
>
>
Now remove the public keys from the private key file:
 
Changed:
<
<
su - frontend
cvs -d :pserver:anonymous@cdcvs.fnal.gov:/cvs/cd_read_only co -r v2_1 glideinWMS
>
>
vi cert.key

Now the permissions need to be adjusted on these files in order to generate the proxy:

chmod 644 /home/frontend/cert.p12
chmod 400 /home/frontend/cert.key

 
Changed:
<
<
Now, let's being by installing Condor, select option 6 in the installer's menu:
/home/frontend/glideinWMS/install/glideinWMS_install
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/home/frontend/vdt]
Directory '/home/frontend/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
What VDT cache should I use?: [http://vdt.cs.wisc.edu/vdt_1101_cache]
Which platform do you want to use (leave empty for autodetect):
Do you agree to the licenses? [y/n] y
Where would you like to install CA files? l
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Where do you have the Condor tarball? /home/frontend/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/home/frontend/glidecondor]
Directory '/home/frontend/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.ext
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
What node is the collector running (i.e. CONDOR_HOST)?: sl1
>
>
Run this command to generate the proxy for 500 hours:
 
Changed:
<
<
Condor for VO Frontend is installed
>
>
/home/frontend/vdt/glite/bin/voms-proxy-init -cert /home/frontend/cert.p12 -key /home/frontend/cert.key -out /home/frontend/.globus/x509_service_proxy -valid 500:0.0
 
Changed:
<
<
Now we need the HTTP server installation:
>
>
Now, let's make the cert and key owned by root to prevent any bad stuff from happening:
 
Changed:
<
<
su - root
/home/frontend/glideinWMS/install/glideinWMS_install
Please select: 8
Please select: c
What port do you want to use?: [80]
Select install method: ... select: 1
>
>
su - root
chown root:root /home/frontend/cert.p12
chown root:root /home/frontend/cert.key
 
Changed:
<
<
HTTPD is installed
>
>
Proxy was created and installed successfully!
 
Changed:
<
<
Copy over the JavaScriptRRD? package into the frontend's home directory:
>
>
END WARNING
 
Added:
>
>
Now we need to move the proxy certificate into a global location:
 
Changed:
<
<
cp javascriptrrd-0.4.2.zip /home/frontend/
chown frontend:frontend /home/frontend/javascriptrrd-0.4.2.zip
su - frontend
unzip javascriptrrd-0.4.2.zip
>
>
mkdir /etc/grid-security/
cp x509_service_proxy /etc/grid-security/
 
Changed:
<
<
JavaScriptRRD is installed
>
>
Let's install the "condor collector" and "condor schedd" AT THE SAME TIME (by selecting 4,5) as root.
 
Changed:
<
<
We also need our proxy certificate to identify ourselves (this could be the same as the one generated on SL1):
>
>
su - root
/root/glideinWMS/install/glideinWMS_install
Please select: 4,5
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/opt/vdt]
Directory '/opt/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
What VDT cache should I use?: [http://vdt.cs.wisc.edu/vdt_1101_cache]
Which platform do you want to use (leave empty for autodetect):
Do you agree to the licenses? [y/n] y
Would you like to setup daily rotation of VDT log files? y
Do you want to update the CA certification revocation lists (CRLs) automatically? [y/n] y
Where would you like to install CA files? r
Do you want to automatically update your CA Certificates? [y/n] y
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Which user should Condor run under?: [condor]
Where do you have the Condor tarball? /root/condor-7.3.2-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/opt/glidecondor]
Directory '/opt/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.ext
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
Will you be using a proxy or a cert? (proxy/cert) proxy
Where is your proxy located?: /etc/grid-security/x509_service_proxy
DN:
How many slave collectors do you want?: [5]
What name would you like to use for this pool?: [My pool] devg5-pool
Do you want to use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
GCB node:
How many secondary schedds do you want?: [9]
 
Added:
>
>
Let's restart condor:
 
Changed:
<
<
su - frontend
mkdir .globus
scp -c blowfish -P 4221 gfactory@sl1:/home/gfactory/cert.p12 /home/frontend/
. /home/frontend/vdt/setup.sh
openssl pkcs12 -in cert.p12 -out cert.key
>
>
killall condor_master
/opt/glidecondor/start_condor.sh
 
Changed:
<
<
Now remove the public keys from the private key file:
>
>
Condor Collector and Schedd is installed on the frontend system.
 
Added:
>
>
Now, we can begin installing the frontend user. We need to create a user account to host the frontend software installation:
 
Changed:
<
<
vi cert.key
>
>
su - root
groupadd -g 5002 frontend
useradd -c "Frontend Daemon" -g 5002 -m -s /bin/bash -u 5002 frontend
 
Changed:
<
<
Now the permissions need to be adjusted on these files in order to generate the proxy:
>
>
Make sure to get a copy of condor into the frontend's home directory:
 
Changed:
<
<
chmod 644 /home/frontend/cert.p12
chmod 400 /home/frontend/cert.key
>
>
cp condor-7.3.2-linux-x86_64-rhel5-dynamic.tar.gz /home/frontend/
chown frontend:frontend /home/frontend/condor-7.3.2-linux-x86_64-rhel5-dynamic.tar.gz
 
Changed:
<
<
Run this command to generate the proxy for 500 hours:
>
>
Copy the glideinWMS software into the frontend user's directory from CVS:
 
Changed:
<
<
/home/frontend/vdt/glite/bin/voms-proxy-init -cert /home/frontend/cert.p12 -key /home/frontend/cert.key -out /home/frontend/.globus/x509_service_proxy -valid 500:0.0
>
>
su - frontend
cvs -d :pserver:anonymous@cdcvs.fnal.gov:/cvs/cd_read_only co -r v2_1 glideinWMS
 
Changed:
<
<
Now, let's make the cert and key owned by root to prevent any bad stuff from happening:
>
>
Now we need the HTTP server installation:

su - root
yum install httpd

HTTPD is installed

Copy over the JavaScriptRRD? package into the frontend's home directory:

 
Changed:
<
<
su - root
chown root:root /home/frontend/cert.p12
chown root:root /home/frontend/cert.key
>
>
cp javascriptrrd-0.4.2.zip /home/frontend/
chown frontend:frontend /home/frontend/javascriptrrd-0.4.2.zip
su - frontend
unzip javascriptrrd-0.4.2.zip
 
Changed:
<
<
Proxy was created and installed successfully!
>
>
JavaScriptRRD is installed
  Let's also install Flot:
Added:
>
>
  wget http://flot.googlecode.com/files/flot-0.5.tar.gz
gunzip -c flot-0.5.tar.gz | tar xvf -
Added:
>
>
  Flot was installed successfully!
Line: 286 to 312
 Now comes time to install the frontend. Let's begin:
Changed:
<
<
. /home/frontend/vdt/setup.sh /home/frontend/glideinWMS/install/glideinWMS_install
Please select: 7
Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/frontend/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/frontend/flot
Where is your proxy located?: /home/frontend/.globus/x509_service_proxy
Where will you host your config and log files?: [/home/frontend/frontstage]
Where will the web data be hosted?: [/var/www/html/vofrontend]
What Web URL will you use?: [http://sl2.minialienship.homeip.net/vofrontend/] http://sl2/vofrontend/
Give a name to this VO Frontend?: [myVO-sl2] frontend-sl2
Give a name to this VO Frontend instance?: [v1_0]
What node is the WMS collector running?: sl1
Collector name(s): [sl1]
Do you want to monitor all of them?: (y/n) y
What kind of jobs do you want to monitor?: [(JobUniverse? ==5)&&(GLIDEIN_Is_Monitor ! TRUE)&&(JOB_Is_Monitor ! TRUE)]
Give a name to the main group: [main]
Match string: [True]
Job attributes: []
Factory attributes: []
Do you want to use is to submit glideins: (y/n) [y] y
An empty entry means you are done. ... proxy fname:
DN (leave empty when finished):
Do you want to use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
Do you want to expose the Grid env. to the user jobs?: (y/n) y
Do you want to create the VO Frontend instance (as opposed to just the config file)?: (y/n) [n] n
>
>
su - frontend
/home/frontend/glideinWMS/install/glideinWMS_install
Please select: 7
Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/frontend/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/frontend/flot
Where is your proxy located?: /etc/grid-security/x509_service_proxy
Where will you host your config and log files?: [/home/frontend/frontstage]
Where will the web data be hosted?: [/var/www/html/vofrontend]
What Web URL will you use?: [http://devg-5.t2.ucsd.edu/vofrontend/]
Give a name to this VO Frontend?: [myVO-devg-5] devg5-vo
Give a name to this VO Frontend instance?: [v1_0]
What node is the WMS collector running?: devg-4.t2.ucsd.edu
Collector name(s): [devg-5.t2.ucsd.edu]
Do you want to monitor all of them?: (y/n) y
What kind of jobs do you want to monitor?: [(JobUniverse? ==5)&&(GLIDEIN_Is_Monitor ! TRUE)&&(JOB_Is_Monitor ! TRUE)]
Give a name to the main group: [main]
Match string: [True]
Job attributes: []
Factory attributes: []
Do you want to use is to submit glideins: (y/n) [y] y
An empty entry means you are done. ... proxy fname:
DN (leave empty when finished):
Do you want to use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
Do you want to expose the Grid env. to the user jobs?: (y/n) y
Do you want to create the VO Frontend instance (as opposed to just the config file)?: (y/n) [n] n
  Ok, now add these lines to .bashrc:
Changed:
<
<
vi ~/.bashrc
. /home/frontend/vdt/setup.sh
export X509_USER_PROXY=/home/frontend/.globus/x509_service_proxy
>
>
vi ~/.bashrc
export X509_USER_PROXY=/etc/grid-security/x509_service_proxy
 

Let's create the VO Frontend instance:

Line: 301 to 327
 /home/frontend/glideinWMS/creation/create_frontend /home/frontend/frontstage/instance_v1_0.cfg/frontend.xml
Changed:
<
<
Let's restart Condor:

killall condor_master
/home/frontend/glidecondor/start_condor.sh

Let's also restart the VO Frontend:

>
>
Let's also restart the VO Frontend:
 
Changed:
<
<
/home/frontend/frontstage/frontend_frontend-sl2-v1_0/frontend_startup stop
/home/frontend/frontstage/frontend_frontend-sl2-v1_0/frontend_startup start
>
>
/home/frontend/frontstage/frontend_devg5-vo-v1_0/frontend_startup stop
/home/frontend/frontstage/frontend_devg5-vo-v1_0/frontend_startup start
 

If the frontend fails to start, consult the frontend's log directory:

Revision 82009/10/14 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"

Installation Walkthrough and Notes on the GlideinWMS System

Line: 8 to 8
 I used Scientific Linux 5.3 from: http://www.scientificlinux.org/

You can also use Redhat Enterprise 5.3 or CentOS 5.3. My installation was on two VMWare Workstation 6.5.3 virtual machines named SL#1 and SL#2. I selected that they would have 512 MiB? of RAM and up to 16 GiB? hard disks. To use GlideinWMS, there is no requirement to use a 64-bit system. You can use a 32-bit and all of the software will be compatible. The reverse is not necessarily true, but I have not confirmed this yet. It's also a good idea to update the system with the latest recommended patches and security updates before continuing any futher. I also reconfigured /etc/ssh/sshd_config to use a different port and adjusted the VM to acquire an IP address from my router in order to make my virtual machines accessible remotely through SSH. I had to change /etc/hosts to list the names of my systems as FQDNs, aliases, and their local IP addresses. Also, I had to edit /etc/resolv.conf to setup DNS after switching to a static IP in the GUI since it wasn't getting it from DHCP anymore.

Changed:
<
<

STEP 2: Install Condor

Option 1: Install from Pre-compiled Binary / RPM

This is the typical way most users will work with Condor. It is the fastest and easiest way to get it up and running.

>
>

STEP 2: Install GlideinWMS

Part 1: Check and Install Prerequisite Software

 
Changed:
<
<
Use a web-browser to download the file "condor-7.3.2-linux-x86_64-rhel5-dynamic-unstripped.tar.gz" from http://www.cs.wisc.edu/condor/downloads-v2/download.pl
>
>
First, we need to verify all the required software is loaded on our system.
 
Changed:
<
<
Copy the RPM to the servers you wish to deploy it on.
>
>
  • A reasonably recent Linux OS (RH/SL4 nad RH/SL5 tested at press time).
cat /etc/redhat-release
Scientific Linux SL release 5.3 (Boron)
 
Changed:
<
<
Let's start installing it:
>
>
  • The Python interpreter (v2.3.4 or above)
 
Changed:
<
<
su - root
groupadd -g 5000 condor
useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor
>
>
python -V
Python 2.4.3
 
Changed:
<
<
The user's home directory must be world readable.
>
>
  • The perl-Time-HiRes rpm.
perl -e 'use Time::HiRes;'
# NO ERROR - MEANS THAT IT IS INSTALLED
  • The OSG client software.
    • Installed from the GlideinWMS installer by selecting "Components" and then "OSG VDT client".
  • A HTTP server, like Apache or TUX.
    • Installed from the GlideinWMS installer by selecting "Components" and then "Web server".
  • The Condor distribution.
    • This is NOT (?) required if you install: pool Collector AND User Schedd
    • Installed from the GlideinWMS installer by selecting "Components" and then "Base Condor installation".
  • The RRDTool package (v1.2.18 or later)
    • The GlideinWMS installer tells us what to install by selecting "Components" and then "rrdtool graphics package".
    • Since no installation takes place, you have to go find the files on the web. The website that lists the installation RPMS points to broken links. I found the RPMs for my 64-bit installation here:
    • http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/
yum install ruby

# This is installed with --nodeps because it has a cross-dependency with rrdtool
rpm -ivh --nodeps perl-rrdtool-1.3.8-2.el5.rf.x86_64.rpm
rpm -ivh rrdtool-1.3.8-2.el5.rf.x86_64.rpm
rpm -ivh rrdtool-devel-1.3.8-2.el5.rf.x86_64.rpm
rpm -ivh python-rrdtool-1.3.8-2.el5.rf.x86_64.rpm
python -c "import M2Crypto? "
# NO ERROR - MEANS THAT IT IS INSTALLED

Part 2: Install GlideinWMS Collector (my SL1)

Ok. Let's proceed with installing the collector on SL1. First we need a new user to own the installation. Let's call them gfactory.

 
Changed:
<
<
chmod 744 /home/condor
>
>
su - root
groupadd -g 5001 gfactory
useradd -c "GFactory Daemon" -g 5001 -m -s /bin/bash -u 5001 gfactory
 
Changed:
<
<
We'll be installing as root since the documentation recommends this procedure. Now, move the tarball we downloaded into the condor directory.
>
>
Let's download and copy condor to the gfactory's home directory:
 
Changed:
<
<
mv condor-7.3.2-linux-x86_64-rhel5-dynamic-unstripped.tar.gz /home/condor
cd /home/condor
>
>
mv condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz /home/gfactory/
cd /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
 
Changed:
<
<
Since we got the tarball, we need to decompress it.
>
>
We want to begin the installation with our new user, MAKE SURE TO DO THIS AS THE gfactory USER. The installation adds a cron job for gfactory, but the linux OS may not support cron jobs from the root user. Here is how to start the installation:
 
Changed:
<
<
gunzip -c condor-7.3.2-linux-x86_64-rhel5-dynamic-unstripped.tar.gz | tar xvf -
>
>
mv glideinWMS/ /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/glideinWMS
su - gfactory
cd glideinWMS/install
./glideinWMS_install
 
Changed:
<
<
[OPTIONAL & UNTESTED] If we got the RPM, the above command might look something like this instead:
>
>
Choose the option "glideinWMS Collector" and then let's answer the questions for the installer interactively. Ignore warnings/errors about CA certificates not being installed. The VDT installer doesn't do it, but the glideWMS installer does. Here are the installer answers:
 
Changed:
<
<
rpm -ivh condor-7.3.2-linux-x86_64-rhel5-1.x86_64.rpm --prefix=/home/condor
>
>
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/home/gfactory/vdt] /home/gfactory/vdt
Directory '/home/gfactory/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
 
Changed:
<
<
Now, let's install Condor as the central manager since this is the first machine. We must run condor configure_ as root to install and set the file permissions correctly.
>
>
This next question is a trick question, don't type anything here, just press enter or you'll get an error.
 
Changed:
<
<
cd condor-7.3.2
./condor_install --verbose --prefix=/home/condor --local-dir=/home/condor/local --type=manager,execute --owner=condor
>
>
Which platform do you want to use (leave empty for autodetect):
WARNING: It appears that SELinux is enabled on this computer. ... Please press enter to continue the installation, or control-c to cancel.
Do you agree to the licenses? [y/n] y
Where would you like to install CA files? l
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Where do you have the Condor tarball? /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/home/gfactory/glidecondor]
Directory '/home/gfactory/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.com
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
 
Changed:
<
<
Ok, now that the Central Manager is installed, we should repeat the above installation on our other nodes which will run Condor. However, we modify the installation command slightly for execute and submit systems:
>
>
In order for the Factory to submit to the grid, we need a proxy on all of the servers. However, we needed VDT to be installed before this is possible. Well, at least the minimal VDT is now installed, so we can use VOMS (Virtual Organization Membership Service) to create the proxy. Let's pause our installation in this window and open up another terminal window. In this new terminal, we will generate the proxy:

Using root, I put my certificate file cert.p12 in the gfactory directory, then I switched back to gfactory.

 
Changed:
<
<
./condor_install --verbose --prefix=/home/condor --local-dir=/home/condor/local --type=submit,execute --owner=condor
>
>
su - root
cd <the directory where the certificate is stored>
cp cert.p12 /home/gfactory
exit
su - gfactory
 
Changed:
<
<
If you wanted all three (not recommended for production or large systems), it might look something like this:
>
>
Now we need to source the VDT setup script in order to work with the VDT software.
 
Changed:
<
<
./condor_install --verbose --prefix=/home/condor --local-dir=/home/condor/local --type=manager,submit,execute --owner=condor
>
>
. /home/gfactory/vdt/setup.sh
 
Changed:
<
<
Just for good measure, make sure the owner is correct on all condor files:
>
>
In order to initialize the voms proxy, I need the private key out of my cert. I was able to get my private key using this command and I saved it to a file called cert.key:
 
Changed:
<
<
chown -R condor:condor /home/condor
>
>
openssl pkcs12 -in cert.p12 -info
 
Changed:
<
<
CONDOR IS NOW INSTALLED! Skip to the next section for configuration.

Option 2: Install from Source

WARNING! The instructions to install from source are not complete yet.

Since we're setting up a development system, we'll install the latest bleeding edge version which is currently 7.3.2. The installation manual for Condor 7.3.2 can be located here: http://www.cs.wisc.edu/condor/manual/v7.3/3_2Installation.html

The condor manual asks some preparation questions. Here are my answers:

  1. What machine will be the central manager? SL#1
  2. What machines should be allowed to submit jobs? SL#1, SL#2
  3. Will Condor run as root or not? Yes, since the installation documentation recommends it.
  4. Who will be administering Condor on the machines in your pool? I will! 8D
  5. Will you have a Unix user named condor and will its home directory be shared? Yes, but it will not be shared.
  6. Where should the machine-specific directories for Condor go? They can live with the rest of the condor application inside the condor daemon's directory.
  7. Where should the parts of the Condor system be installed? Inside the condor directory.
    • Configuration file: maybe in /condor/config
    • Release directory: maybe in /condor/release
      • user binaries: maybe in /condor/bin
      • system binaries: maybe in /condor/sbin
      • lib directory: maybe in /condor/lib
      • etc directory: maybe in /condor/etc
    • Documentation: maybe in /condor/doc
  8. Am I using AFS? Nope
  9. Do I have enough disk space for Condor? Yah, 50 MiB? right?

Ok, that's it for pre-installation questions. We'll be installing Condor on SL#1 first to setup the central information repository.

Let's start installing it:

>
>
Now the permissions need to be adjusted on these files in order to generate the proxy:
 
Changed:
<
<
su - root
groupadd -g 5000 condor
useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor
>
>
chmod 644 /home/gfactory/cert.p12
chmod 400 /home/gfactory/cert.key
 
Changed:
<
<
The user's home directory must be world readable.
>
>
Run this command to generate the proxy for 500 hours:
 
Changed:
<
<
chmod 744 /home/condor
su - condor
>
>
/home/gfactory/vdt/glite/bin/voms-proxy-init -cert /home/gfactory/cert.p12 -key /home/gfactory/cert.key -out /home/gfactory/.globus/x509_service_proxy -valid 500:0.0
 
Changed:
<
<
Ok, the condor download site is a little goofy which prevents me from just using wget. Just download the src code tarball "condor_src-7.3.2-all-all.tar.gz" and move it into /home/condor/ on SL#1. You can do this however you want to. We're going to be compiling from source and not using a binary. This allows us greater flexibility for installation.
>
>
Now, let's make the cert and key owned by root to prevent any bad stuff from happening:
 
Changed:
<
<
gunzip -c condor_src-7.3.2-all-all.tar.gz | tar xvf -
cd condor-7.3.2
>
>
su - root
chown root:root /home/gfactory/cert.p12
chown root:root /home/gfactory/cert.key
 
Changed:
<
<
Let's read the file README.building in this directory. It seems that we need some packages pre-installed. Luckily, SL 5.3 already has everything I need. I checked using yum list <package name> as the root user.

Ok, now let's start compiling.

>
>
Back in the previous terminal window with our Collector installation, let's continue where we left off.
 
Changed:
<
<
cd src
./build_init
>
>
Will you be using a proxy or a cert? (proxy/cert) proxy
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
What name would you like to use for this pool?: [My glideinWMS pool] ChrisB? Pool
How many secondary schedds do you want?: [9] 3
 
Changed:
<
<
FAILURE. O M G. It was worth a try.

Required tools are present and valid, attempting to initialize build
configure.ac:2547: /usr/bin/m4: builtin `mkstemp' requested by frozen file is not supported
autom4te: /usr/bin/m4 failed with exit status: 1
autoheader: /usr/bin/autom4te failed with exit status: 1
Failed to initialize build, check errors and try again once fixed

>
>
******************************************
WMS collector successfully installed
******************************************

Part 3: Install GlideinWMS Factory (my SL1)

 
Changed:
<
<
Let's update m4 from source. Login as root again in another window.
>
>
First, we must install some additional prerequisite software. We need to get the latest flot tarball, move it into the gfactory's home and extract it. Here is the website to get it from:
 
Changed:
<
<
wget ftp://ftp.scientificlinux.org/linux/scientific/5x/SRPMS/SL/m4-1.4.8-1.src.rpm
>
>
http://code.google.com/p/flot/
 
Changed:
<
<
rpm -ivh m4-1.4.8-1.src.rpm
>
>
We will be using the same proxy used in Part 2, so no additional configuration will be necessary for the proxy. This file is:
 
Changed:
<
<
rpmbuild -bb --rebuild /usr/src/redhat/SPECS/m4.spec
>
>
/home/gfactory/.globus/x509_service_proxy
 
Changed:
<
<
My installation is 64-bit, so install this:
>
>
We also need to add a web directory for glidein submission:
 
Changed:
<
<
rpm -ivhF /usr/src/redhat/RPMS/x86_64/m4-1.4.8-1.x86_64.rpm
>
>
su - root
mkdir /var/www/html/glidefactory
chown gfactory:gfactory /var/www/html/glidefactory
 
Changed:
<
<
The 32-bit equivalent would be:
>
>
We are now installing the factory. Select option 2 from the glideinWMS installation script.
 
Changed:
<
<
rpm -ivhF /usr/src/redhat/RPMS/i386/m4-1.4.8-1.i386.rpm
>
>
su - gfactory
cd /home/gfactory/glideinWMS/install
./glideinWMS_install
 
Changed:
<
<
Let's try running the build initialization tool again:
>
>
Here are my answers:
 
Changed:
<
<
./build_init
>
>
Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/gfactory/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/gfactory/flot
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
Where will you host your config and log files?: [/home/gfactory/glideinsubmit][/home/gfactory/glideinsubmit]
Directory '/home/gfactory/glideinsubmit' does not exist, should I create it?: (y/n) y
Where will the web data be hosted?: [/var/www/html/glidefactory]
What Web URL will you use?: [http://myhost.mydomain/glidefactory/] http://sl1/glidefactory/
Give a name to this Glidein Factory?: [mySites-sl1] gfactory-sl1
Give a name to this Glidein instance?: [v1_0]
What is the Condor base directory?: [/home/gfactory/glidecondor]
The following glidein schedds have been found: ... Do you want to use all of them?: (y/n) y
Do you want to use CCB (requires Condor 7.3.0 or better)?: (y/n) n
Please list all the GCB servers you will be using ... Leave an empty line when finished ... GCB node:
Do you want to use gLExec?: (y/n) y
Force VO frontend to provide its own proxy?: (y/n) [y] y
Do you want to fetch entries from RESS?: (y/n) [n] n
Do you want to fetch entries from BDII?: (y/n) [n] n
Please list all additional glidein entry points, Entry name (leave empty when finished): myentry
Gatekeeper for 'myentry': devg-6.t2.ucsd.edu
RSL for 'myentry':
Work dir for 'myentry': .
Site name for 'myentry': [myentry] mysite
gLExec path for 'myentry': [OSG]
Should glideins use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
Do you want to create the glidein (as opposed to just the config file)?: (y/n) [n]n
To create the glidein, you need to run
/home/gfactory/glideinWMS/creation/create_glidein /home/gfactory/glideinsubmit/glidein_v1_0.cfg/glideinWMS.xml
Configuration files are located in /home/gfactory/glideinsubmit/glidein_v1_0.cfg
Remember to set X509_USER_PROXY before starting the daemon
 
Changed:
<
<
[condor@localhost src]$ ./build_init
Checking for version of autoheader >= 2.59...succeeded. (2.59)
Checking for version of autoconf >= 2.59...succeeded. (2.59)
Required tools are present and valid, attempting to initialize build
Build initialized, you can now run "./configure; make"

SUCCESS!!! Let's enable as many optional features as possible. NOTE: --prefix doesn't work yet.

>
>
Ok, now add this line to .bashrc:
 
Changed:
<
<
./configure --enable-full-port --enable-soft-is-hard --enable-job-hooks --enable-hibernation --enable-ssh-to-job --with-buildid

make

make release

>
>
export X509_USER_PROXY=/home/gfactory/.globus/x509_service_proxy
 
Changed:
<
<
...TO BE CONTINUED

STEP 3: Configure Condor

Now that Condor is installed, it must be configured. At the end of the installation, this message is displayed:

In order for Condor to work properly you must set your CONDOR_CONFIG environment variable to point to your Condor configuration file:
/home/condor/etc/condor_config before running Condor commands/daemons. Created scripts which can be sourced by users to setup their
Condor environment variables. These are:
sh: /home/condor/condor.sh
csh: /home/condor/condor.csh

This means we need to update the environment variable CONDOR_CONFIG to be changed when you login as the condor user:

I just added this line to /home/condor/.bash_profile:

>
>
Let's create the glidein:
 
Changed:
<
<
. ~/condor.sh
>
>
/home/gfactory/glideinWMS/creation/create_glidein /home/gfactory/glideinsubmit/glidein_v1_0.cfg/glideinWMS.xml
 
Changed:
<
<
Let's login as the condor user:
>
>
Let's restart Condor:
 
Changed:
<
<
su - condor
>
>
killall condor_master
/home/gfactory/glidecondor/start_condor.sh
 
Changed:
<
<
Now, we must manually edit the condor configuration file:
>
>
Let's also restart the Factory:
 
Changed:
<
<
vi $CONDOR_CONFIG
>
>
/home/gfactory/glideinsubmit/glidein_v1_0/factory_startup stop
/home/gfactory/glideinsubmit/glidein_v1_0/factory_startup start
 
Changed:
<
<
So, according to this file, we must fill out Part 1 in order for condor to work. Here are my answers for the SL1 virtual machine:
>
>
If the factory fails to start, consult the factory's log directory:
 
Changed:
<
<
LOCAL_DIR = /home/condor/local
CONDOR_ADMIN = me@mydomain.com
UID_DOMAIN = $(FULL_HOSTNAME)
FILESYSTEM_DOMAIN = $(FULL_HOSTNAME)
COLLECTOR_NAME = ChrisB? Pool
>
>
ls /home/gfactory/glideinsubmit/glidein_v1_0/log
 
Changed:
<
<
There is also a required field to edit in Part 2 despite the file stating that Part 2 is optional (oh well):
>
>
This concludes the glideinWMS collector and factory installation. I hope it was fun.

Part 4: Install GlideinWMS Frontend System (my SL2)

The next system needed is the GlideinWMS frontend system. We will install this on my system named SL2. This system needs most of the pre-requisites of SL1.

  • Any Condor-supported OS
  • The OSG client software
  • The Condor distribution
  • A HTTP server, like Apache or TUX
  • The RRDTool package (v1.2.18 or later)
  • The M2Crypto? python library (v0.17 or later)
  • The javascriptRRD package (0.4 or later)
  • The perl-Time-HiRes rpm
  • The glideinWMS software
Let's assume we've already installed these components. If not, follow the steps in part 1. First, we need to create a user account to host the frontend software installation:
 
Changed:
<
<
HOSTALLOW_WRITE = *.mydomain.com
>
>
su - root
groupadd -g 5002 frontend
useradd -c "Frontend Daemon" -g 5002 -m -s /bin/bash -u 5002 frontend
 
Changed:
<
<
Let's also edit the local configuration for each machine:
>
>
Make sure to get a copy of condor into the frontend's home directory:
 
Changed:
<
<
vi /home/condor/local/condor_config.local
>
>
cp condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz /home/frontend/
chown frontend:frontend /home/frontend/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
 
Changed:
<
<
Here are my changes:
>
>
Copy the glideinWMS software into the frontend user's directory from CVS:
 
Changed:
<
<
CONDOR_ADMIN = me@mydomain.com

UID_DOMAIN = $(FULL_HOSTNAME)

FILESYSTEM_DOMAIN = $(FULL_HOSTNAME)

>
>
su - frontend
cvs -d :pserver:anonymous@cdcvs.fnal.gov:/cvs/cd_read_only co -r v2_1 glideinWMS
 
Changed:
<
<

STEP 4: Start Condor

As the condor user, let's start the condor daemon:
>
>
Now, let's being by installing Condor, select option 6 in the installer's menu:
/home/frontend/glideinWMS/install/glideinWMS_install
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/home/frontend/vdt]
Directory '/home/frontend/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
What VDT cache should I use?: [http://vdt.cs.wisc.edu/vdt_1101_cache]
Which platform do you want to use (leave empty for autodetect):
Do you agree to the licenses? [y/n] y
Where would you like to install CA files? l
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Where do you have the Condor tarball? /home/frontend/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/home/frontend/glidecondor]
Directory '/home/frontend/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.ext
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
What node is the collector running (i.e. CONDOR_HOST)?: sl1

Condor for VO Frontend is installed

 
Changed:
<
<
su - condor
>
>
Now we need the HTTP server installation:
 
Changed:
<
<
condor_master
>
>
su - root
/home/frontend/glideinWMS/install/glideinWMS_install
Please select: 8
Please select: c
What port do you want to use?: [80]
Select install method: ... select: 1
 
Changed:
<
<

STEP 5: Install GlideinWMS

Part 1: Check and Install Prerequisite Software

>
>
HTTPD is installed
 
Changed:
<
<
First, we need to verify all the required software is loaded on our system.
>
>
Copy over the JavaScriptRRD? package into the frontend's home directory:
 
Changed:
<
<
  • A reasonably recent Linux OS (RH/SL4 nad RH/SL5 tested at press time).
cat /etc/redhat-release
>
>
 
Changed:
<
<
Scientific Linux SL release 5.3 (Boron)
>
>
cp javascriptrrd-0.4.2.zip /home/frontend/
chown frontend:frontend /home/frontend/javascriptrrd-0.4.2.zip
su - frontend
unzip javascriptrrd-0.4.2.zip
 
Changed:
<
<
  • The Python interpreter (v2.3.4 or above)
>
>
JavaScriptRRD is installed

We also need our proxy certificate to identify ourselves (this could be the same as the one generated on SL1):

 
Changed:
<
<
python -V
Python 2.4.3
>
>
su - frontend
mkdir .globus
scp -c blowfish -P 4221 gfactory@sl1:/home/gfactory/cert.p12 /home/frontend/
. /home/frontend/vdt/setup.sh
openssl pkcs12 -in cert.p12 -out cert.key
 
Changed:
<
<
  • The perl-Time-HiRes rpm.
perl -e 'use Time::HiRes;'
# NO ERROR - MEANS THAT IT IS INSTALLED
  • The OSG client software.
    • Installed from the GlideinWMS installer by selecting "Components" and then "OSG VDT client".
  • A HTTP server, like Apache or TUX.
    • Installed from the GlideinWMS installer by selecting "Components" and then "Web server".
  • The Condor distribution.
    • This is NOT (?) required if you install: pool Collector AND User Schedd
    • Installed from the GlideinWMS installer by selecting "Components" and then "Base Condor installation".
  • The RRDTool package (v1.2.18 or later)
    • The GlideinWMS installer tells us what to install by selecting "Components" and then "rrdtool graphics package".
    • Since no installation takes place, you have to go find the files on the web. The website that lists the installation RPMS points to broken links. I found the RPMs for my 64-bit installation here:
    • http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/
yum install ruby

# This is installed with --nodeps because it has a cross-dependency with rrdtool
rpm -ivh --nodeps perl-rrdtool-1.3.8-2.el5.rf.x86_64.rpm
rpm -ivh rrdtool-1.3.8-2.el5.rf.x86_64.rpm
rpm -ivh rrdtool-devel-1.3.8-2.el5.rf.x86_64.rpm
rpm -ivh python-rrdtool-1.3.8-2.el5.rf.x86_64.rpm
python -c "import M2Crypto? "
# NO ERROR - MEANS THAT IT IS INSTALLED

Part 2: Install GlideinWMS Collector (my SL1)

>
>
Now remove the public keys from the private key file:
 
Deleted:
<
<
Ok. Let's proceed with installing the collector on SL1. First we need a new user to own the installation. Let's call them gfactory.
 
Changed:
<
<
su - root
groupadd -g 5001 gfactory
useradd -c "GFactory Daemon" -g 5001 -m -s /bin/bash -u 5001 gfactory
>
>
vi cert.key
 
Changed:
<
<
Let's download and copy condor to the gfactory's home directory:
>
>
Now the permissions need to be adjusted on these files in order to generate the proxy:
 
Changed:
<
<
mv condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz /home/gfactory/
cd /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
>
>
chmod 644 /home/frontend/cert.p12
chmod 400 /home/frontend/cert.key
 
Changed:
<
<
We want to begin the installation with our new user, MAKE SURE TO DO THIS AS THE gfactory USER. The installation adds a cron job for gfactory, but the linux OS may not support cron jobs from the root user. Here is how to start the installation:
>
>
Run this command to generate the proxy for 500 hours:
 
Changed:
<
<
mv glideinWMS/ /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/glideinWMS
su - gfactory
cd glideinWMS/install
./glideinWMS_install
>
>
/home/frontend/vdt/glite/bin/voms-proxy-init -cert /home/frontend/cert.p12 -key /home/frontend/cert.key -out /home/frontend/.globus/x509_service_proxy -valid 500:0.0
 
Changed:
<
<
Choose the option "glideinWMS Collector" and then let's answer the questions for the installer interactively. Ignore warnings/errors about CA certificates not being installed. The VDT installer doesn't do it, but the glideWMS installer does. Here are the installer answers:
>
>
Now, let's make the cert and key owned by root to prevent any bad stuff from happening:
 
Changed:
<
<
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/home/gfactory/vdt] /home/gfactory/vdt
Directory '/home/gfactory/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
>
>
su - root
chown root:root /home/frontend/cert.p12
chown root:root /home/frontend/cert.key
 
Changed:
<
<
This next question is a trick question, don't type anything here, just press enter or you'll get an error.
>
>
Proxy was created and installed successfully!
 
Changed:
<
<
Which platform do you want to use (leave empty for autodetect):
WARNING: It appears that SELinux is enabled on this computer. ... Please press enter to continue the installation, or control-c to cancel.
Do you agree to the licenses? [y/n] y
Where would you like to install CA files? l
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Where do you have the Condor tarball? /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/home/gfactory/glidecondor]
Directory '/home/gfactory/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.com
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
>
>
Let's also install Flot:
 
Changed:
<
<
In order for the Factory to submit to the grid, we need a proxy on all of the servers. However, we needed VDT to be installed before this is possible. Well, at least the minimal VDT is now installed, so we can use VOMS (Virtual Organization Membership Service) to create the proxy. Let's pause our installation in this window and open up another terminal window. In this new terminal, we will generate the proxy:
>
>
wget http://flot.googlecode.com/files/flot-0.5.tar.gz
gunzip -c flot-0.5.tar.gz | tar xvf -
 
Changed:
<
<
Using root, I put my certificate file cert.p12 in the gfactory directory, then I switched back to gfactory.
>
>
Flot was installed successfully!

Let's create a directory for the host and config log files:

 
Changed:
<
<
su - root
cd <the directory where the certificate is stored>
cp cert.p12 /home/gfactory
exit
su - gfactory
>
>
mkdir /home/frontend/frontstage
 
Changed:
<
<
Now we need to source the VDT setup script in order to work with the VDT software.
>
>
Let's also create a directory for the web files:
 
Changed:
<
<
. /home/gfactory/vdt/setup.sh
>
>
su - root
mkdir /var/www/html/vofrontend
chown frontend:frontend /var/www/html/vofrontend
 
Changed:
<
<
In order to initialize the voms proxy, I need the private key out of my cert. I was able to get my private key using this command and I saved it to a file called cert.key:
>
>
Finished with directory setup!

Now comes time to install the frontend. Let's begin:

 
Changed:
<
<
openssl pkcs12 -in cert.p12 -info
>
>
. /home/frontend/vdt/setup.sh /home/frontend/glideinWMS/install/glideinWMS_install
Please select: 7
Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/frontend/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/frontend/flot
Where is your proxy located?: /home/frontend/.globus/x509_service_proxy
Where will you host your config and log files?: [/home/frontend/frontstage]
Where will the web data be hosted?: [/var/www/html/vofrontend]
What Web URL will you use?: [http://sl2.minialienship.homeip.net/vofrontend/] http://sl2/vofrontend/
Give a name to this VO Frontend?: [myVO-sl2] frontend-sl2
Give a name to this VO Frontend instance?: [v1_0]
What node is the WMS collector running?: sl1
Collector name(s): [sl1]
Do you want to monitor all of them?: (y/n) y
What kind of jobs do you want to monitor?: [(JobUniverse? ==5)&&(GLIDEIN_Is_Monitor ! TRUE)&&(JOB_Is_Monitor ! TRUE)]
Give a name to the main group: [main]
Match string: [True]
Job attributes: []
Factory attributes: []
Do you want to use is to submit glideins: (y/n) [y] y
An empty entry means you are done. ... proxy fname:
DN (leave empty when finished):
Do you want to use the more efficient Match authentication (works for Condor v7.1.3 and later)?: (y/n) y
Do you want to expose the Grid env. to the user jobs?: (y/n) y
Do you want to create the VO Frontend instance (as opposed to just the config file)?: (y/n) [n] n
 
Changed:
<
<
Now the permissions need to be adjusted on these files in order to generate the proxy:
>
>
Ok, now add these lines to .bashrc:
 
Changed:
<
<
chmod 644 /home/gfactory/cert.p12
chmod 400 /home/gfactory/cert.key
>
>
vi ~/.bashrc
. /home/frontend/vdt/setup.sh
export X509_USER_PROXY=/home/frontend/.globus/x509_service_proxy
 
Changed:
<
<
Run this command to generate the proxy for 500 hours:
>
>
Let's create the VO Frontend instance:
 
Changed:
<
<
/home/gfactory/vdt/glite/bin/voms-proxy-init -cert /home/gfactory/cert.p12 -key /home/gfactory/cert.key -out /home/gfactory/.globus/x509_service_proxy -valid 500:0.0
>
>
/home/frontend/glideinWMS/creation/create_frontend /home/frontend/frontstage/instance_v1_0.cfg/frontend.xml
 
Changed:
<
<
Now, let's make the cert and key owned by root to prevent any bad stuff from happening:
>
>
Let's restart Condor:
 
Changed:
<
<
su - root
chown root:root /home/gfactory/cert.p12
chown root:root /home/gfactory/cert.key
>
>
killall condor_master
/home/frontend/glidecondor/start_condor.sh
 
Changed:
<
<
Back in the previous terminal window with our Collector installation, let's continue where we left off.
>
>
Let's also restart the VO Frontend:
 
Changed:
<
<
Will you be using a proxy or a cert? (proxy/cert) proxy
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
What name would you like to use for this pool?: [My glideinWMS pool] ChrisB? Pool
How many secondary schedds do you want?: [9] 3
>
>
/home/frontend/frontstage/frontend_frontend-sl2-v1_0/frontend_startup stop
/home/frontend/frontstage/frontend_frontend-sl2-v1_0/frontend_startup start
 
Changed:
<
<
******************************************
WMS collector successfully installed
******************************************

Part 3: Install GlideinWMS Factory (my SL1)

First, we must install some additional prerequisite software. We need to get the latest flot tarball, move it into the gfactory's home and extract it. Here is the website to get it from:

http://code.google.com/p/flot/

We will be using the same proxy used in Part 2, so no additional configuration will be necessary for the proxy. This file is:

/home/gfactory/.globus/x509_service_proxy

We also need to add a web directory for glidein submission:

su - root
mkdir /var/www/html/glidefactory
chown gfactory:gfactory /var/www/html/glidefactory

We are now installing the factory. Select option 2 from the glideinWMS installation script.

su - gfactory
cd /home/gfactory/glideinWMS/install
./glideinWMS_install

Here are my answers:

Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/gfactory/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/gfactory/flot
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
Where will you host your config and log files?: [/home/gfactory/glideinsubmit][/home/gfactory/glideinsubmit]
Directory '/home/gfactory/glideinsubmit' does not exist, should I create it?: (y/n) y
Where will the web data be hosted?: [/var/www/html/glidefactory]
What Web URL will you use?: [http://myhost.mydomain/glidefactory/] http://sl1/glidefactory/
Give a name to this Glidein Factory?: [mySites-sl1] gfactory-sl1
Give a name to this Glidein instance?: [v1_0]
What is the Condor base directory?: [/home/gfactory/glidecondor]
The following glidein schedds have been found: ... Do you want to use all of them?: (y/n) y
Do you want to use CCB (requires Condor 7.3.0 or better)?: (y/n) n
Please list all the GCB servers you will be using ... Leave an empty line when finished ... GCB node:
Do you want to use gLExec?: (y/n) y
Force VO frontend to provide its own proxy?: (y/n) [y] y
Do you want to fetch entries from RESS?: (y/n) [n] n

Part 4: Install GlideinWMS Frontend System (my SL2)

>
>
If the frontend fails to start, consult the frontend's log directory:
 
Changed:
<
<
This is the frontend system.
>
>
ls /home/frontend/frontstage/frontend_frontend-sl2-v1_0/log

Revision 72009/10/13 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"

Installation Walkthrough and Notes on the GlideinWMS System

Line: 269 to 269
 
Changed:
<
<

Part 2: Install GlideinWMS Factory System (my SL1)

>
>

Part 2: Install GlideinWMS Collector (my SL1)

  Ok. Let's proceed with installing the collector on SL1. First we need a new user to own the installation. Let's call them gfactory.
Line: 283 to 283
 mv condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz /home/gfactory/
cd /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
Changed:
<
<
We want to begin the installation with our new user:
>
>
We want to begin the installation with our new user, MAKE SURE TO DO THIS AS THE gfactory USER. The installation adds a cron job for gfactory, but the linux OS may not support cron jobs from the root user. Here is how to start the installation:
 

mv glideinWMS/ /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/glideinWMS
su - gfactory
cd glideinWMS/install
./glideinWMS_install

Changed:
<
<
Choose the option "glideinWMS Collector" and then let's answer the questions for the installer interactively:
>
>
Choose the option "glideinWMS Collector" and then let's answer the questions for the installer interactively. Ignore warnings/errors about CA certificates not being installed. The VDT installer doesn't do it, but the glideWMS installer does. Here are the installer answers:
 
Changed:
<
<
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) y
Where do you want the VDT installed?: [/home/gfactory/vdt]
Directory '/home/gfactory/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
Which platform do you want to use (leave empty for autodetect):
WARNING: It appears that SELinux is enabled on this computer. ... Please press enter to continue the installation, or control-c to cancel.
Do you agree to the licenses? [y/n] y
Where would you like to install CA files? l
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Where do you have the Condor tarball? /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/home/gfactory/glidecondor]
Directory '/home/gfactory/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.com
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
Will you be using a proxy or a cert? (proxy/cert) cert
Where is your certificate located?: /home/gfactory/chris_ca_doe.crt
Where is your certificate key located?: /home/gfactory/chris_ca_doe.key
DN:
What name would you like to use for this pool?: [My glideinWMS pool] ChrisB? Pool
How many secondary schedds do you want?: [9]
>
>
Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) n
Do you want to install a minimal Grid VDT client?: (y/n) y
Where do you want the VDT installed?: [/home/gfactory/vdt] /home/gfactory/vdt
Directory '/home/gfactory/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
 
Changed:
<
<
Now the collector has been installed successfully!
>
>
This next question is a trick question, don't type anything here, just press enter or you'll get an error.

Which platform do you want to use (leave empty for autodetect):
WARNING: It appears that SELinux is enabled on this computer. ... Please press enter to continue the installation, or control-c to cancel.
Do you agree to the licenses? [y/n] y
Where would you like to install CA files? l
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Where do you have the Condor tarball? /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/home/gfactory/glidecondor]
Directory '/home/gfactory/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.com
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]

In order for the Factory to submit to the grid, we need a proxy on all of the servers. However, we needed VDT to be installed before this is possible. Well, at least the minimal VDT is now installed, so we can use VOMS (Virtual Organization Membership Service) to create the proxy. Let's pause our installation in this window and open up another terminal window. In this new terminal, we will generate the proxy:

Using root, I put my certificate file cert.p12 in the gfactory directory, then I switched back to gfactory.

su - root
cd <the directory where the certificate is stored>
cp cert.p12 /home/gfactory
exit
su - gfactory

Now we need to source the VDT setup script in order to work with the VDT software.

. /home/gfactory/vdt/setup.sh

In order to initialize the voms proxy, I need the private key out of my cert. I was able to get my private key using this command and I saved it to a file called cert.key:

openssl pkcs12 -in cert.p12 -info

Now the permissions need to be adjusted on these files in order to generate the proxy:

chmod 644 /home/gfactory/cert.p12
chmod 400 /home/gfactory/cert.key

Run this command to generate the proxy for 500 hours:

/home/gfactory/vdt/glite/bin/voms-proxy-init -cert /home/gfactory/cert.p12 -key /home/gfactory/cert.key -out /home/gfactory/.globus/x509_service_proxy -valid 500:0.0

Now, let's make the cert and key owned by root to prevent any bad stuff from happening:

su - root
chown root:root /home/gfactory/cert.p12
chown root:root /home/gfactory/cert.key

Back in the previous terminal window with our Collector installation, let's continue where we left off.

Will you be using a proxy or a cert? (proxy/cert) proxy
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
What name would you like to use for this pool?: [My glideinWMS pool] ChrisB? Pool
How many secondary schedds do you want?: [9] 3

******************************************
WMS collector successfully installed
******************************************

Part 3: Install GlideinWMS Factory (my SL1)

First, we must install some additional prerequisite software. We need to get the latest flot tarball, move it into the gfactory's home and extract it. Here is the website to get it from:

http://code.google.com/p/flot/

We will be using the same proxy used in Part 2, so no additional configuration will be necessary for the proxy. This file is:

/home/gfactory/.globus/x509_service_proxy

We also need to add a web directory for glidein submission:

su - root
mkdir /var/www/html/glidefactory
chown gfactory:gfactory /var/www/html/glidefactory

We are now installing the factory. Select option 2 from the glideinWMS installation script.

su - gfactory
cd /home/gfactory/glideinWMS/install
./glideinWMS_install

Here are my answers:

Do you have already a javascriptRRD installation?: (y/n) y
Where is javascriptRRD installed?: /home/gfactory/javascriptrrd-0.4.2
Do you have already a Flot installation?: (y/n) y
Where is Flot installed?: /home/gfactory/flot
Where is your proxy located?: /home/gfactory/.globus/x509_service_proxy
Where will you host your config and log files?: [/home/gfactory/glideinsubmit][/home/gfactory/glideinsubmit]
Directory '/home/gfactory/glideinsubmit' does not exist, should I create it?: (y/n) y
Where will the web data be hosted?: [/var/www/html/glidefactory]
What Web URL will you use?: [http://myhost.mydomain/glidefactory/] http://sl1/glidefactory/
Give a name to this Glidein Factory?: [mySites-sl1] gfactory-sl1
Give a name to this Glidein instance?: [v1_0]
What is the Condor base directory?: [/home/gfactory/glidecondor]
The following glidein schedds have been found: ... Do you want to use all of them?: (y/n) y
Do you want to use CCB (requires Condor 7.3.0 or better)?: (y/n) n
Please list all the GCB servers you will be using ... Leave an empty line when finished ... GCB node:
Do you want to use gLExec?: (y/n) y
Force VO frontend to provide its own proxy?: (y/n) [y] y
Do you want to fetch entries from RESS?: (y/n) [n] n

 
Changed:
<
<

Part 3: Install GlideinWMS Frontend System (my SL2)

>
>

Part 4: Install GlideinWMS Frontend System (my SL2)

  This is the frontend system.

Revision 62009/10/10 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"

Installation Walkthrough and Notes on the GlideinWMS System

Line: 233 to 233
 

STEP 5: Install GlideinWMS

Added:
>
>

Part 1: Check and Install Prerequisite Software

  First, we need to verify all the required software is loaded on our system.
Line: 268 to 269
 
Added:
>
>

Part 2: Install GlideinWMS Factory System (my SL1)

Ok. Let's proceed with installing the collector on SL1. First we need a new user to own the installation. Let's call them gfactory.

su - root
groupadd -g 5001 gfactory
useradd -c "GFactory Daemon" -g 5001 -m -s /bin/bash -u 5001 gfactory

Let's download and copy condor to the gfactory's home directory:

mv condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz /home/gfactory/
cd /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz

We want to begin the installation with our new user:

mv glideinWMS/ /home/gfactory/
chown -R gfactory:gfactory /home/gfactory/glideinWMS
su - gfactory
cd glideinWMS/install
./glideinWMS_install

Choose the option "glideinWMS Collector" and then let's answer the questions for the installer interactively:

Do you have already a VDT installation?: (y/n) n
Do you want to install the full OSG VDT client?: (y/n) y
Where do you want the VDT installed?: [/home/gfactory/vdt]
Directory '/home/gfactory/vdt' does not exist, should I create it?: (y/n) y
What pacman version should I use?: [pacman-3.26]
Which platform do you want to use (leave empty for autodetect):
WARNING: It appears that SELinux is enabled on this computer. ... Please press enter to continue the installation, or control-c to cancel.
Do you agree to the licenses? [y/n] y
Where would you like to install CA files? l
Where should I fetch the CAs from?: [http://software.grid.iu.edu/pacman/cadist/ca-certs-version]
Where do you have the Condor tarball? /home/gfactory/condor-7.3.1-linux-x86_64-rhel5-dynamic.tar.gz
Where do you want to install it?: [/home/gfactory/glidecondor]
Directory '/home/gfactory/glidecondor' does not exist, should I create it?: (y/n) y
If something goes wrong with Condor, who should get email about it?: me@mydomain.com
Do you want to split the config files between condor_config and condor_config.local?: (y/n) [y]
Will you be using a proxy or a cert? (proxy/cert) cert
Where is your certificate located?: /home/gfactory/chris_ca_doe.crt
Where is your certificate key located?: /home/gfactory/chris_ca_doe.key
DN:
What name would you like to use for this pool?: [My glideinWMS pool] ChrisB? Pool
How many secondary schedds do you want?: [9]

Now the collector has been installed successfully!

Part 3: Install GlideinWMS Frontend System (my SL2)

This is the frontend system.

Revision 52009/10/09 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"

Installation Walkthrough and Notes on the GlideinWMS System

Added:
>
>
DISCLAIMER: Any information you read on this page is a lie. Actually, it isn't but you should treat it as such. These are my personal notes so YMMV! I do not want to be held responsible for your destruction of the world. Having said all of that, enjoy! :)
 

STEP 1: Obtain, install, and update a Linux distribution

I used Scientific Linux 5.3 from: http://www.scientificlinux.org/

Changed:
<
<
You can also use Redhat Enterprise 5.3 or CentOS 5.3. My installation was on two VMWare Workstation 6.5.3 virtual machines named SL#1 and SL#2. I selected that they would have 512 MiB? of RAM and up to 16 GiB? hard disks. To use GlideinWMS, there is no requirement to use a 64-bit system. You can use a 32-bit and all of the software will be compatible. The reverse is not necessarily true, but I have not confirmed this yet. It's also a good idea to update the system with the latest recommended patches and security updates before continuing any futher. I also reconfigured /etc/ssh/sshd_config to use a different port and adjusted the VM to acquire an IP address from my router in order to make my virtual machines accessible remotely through SSH. I had to change /etc/hosts to list the names of my systems and their local IP addresses. Also, I had to edit /etc/resolv.conf to setup DNS after switching to a static IP in the GUI since it wasn't getting it from DHCP anymore.
>
>
You can also use Redhat Enterprise 5.3 or CentOS 5.3. My installation was on two VMWare Workstation 6.5.3 virtual machines named SL#1 and SL#2. I selected that they would have 512 MiB? of RAM and up to 16 GiB? hard disks. To use GlideinWMS, there is no requirement to use a 64-bit system. You can use a 32-bit and all of the software will be compatible. The reverse is not necessarily true, but I have not confirmed this yet. It's also a good idea to update the system with the latest recommended patches and security updates before continuing any futher. I also reconfigured /etc/ssh/sshd_config to use a different port and adjusted the VM to acquire an IP address from my router in order to make my virtual machines accessible remotely through SSH. I had to change /etc/hosts to list the names of my systems as FQDNs, aliases, and their local IP addresses. Also, I had to edit /etc/resolv.conf to setup DNS after switching to a static IP in the GUI since it wasn't getting it from DHCP anymore.
 

STEP 2: Install Condor

Option 1: Install from Pre-compiled Binary / RPM

Line: 17 to 18
 Copy the RPM to the servers you wish to deploy it on.

Let's start installing it:

Added:
>
>
 
Changed:
<
<
su - root

groupadd -g 5000 condor

useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor

>
>
su - root
groupadd -g 5000 condor
useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor
  The user's home directory must be world readable.
Added:
>
>
  chmod 744 /home/condor
Added:
>
>
  We'll be installing as root since the documentation recommends this procedure. Now, move the tarball we downloaded into the condor directory.
Added:
>
>
 
Changed:
<
<
mv condor-7.3.2-linux-x86_64-rhel5-dynamic-unstripped.tar.gz /home/condor

cd /home/condor

>
>
mv condor-7.3.2-linux-x86_64-rhel5-dynamic-unstripped.tar.gz /home/condor
cd /home/condor
  Since we got the tarball, we need to decompress it.
Added:
>
>
  gunzip -c condor-7.3.2-linux-x86_64-rhel5-dynamic-unstripped.tar.gz | tar xvf -
Added:
>
>
  [OPTIONAL & UNTESTED] If we got the RPM, the above command might look something like this instead:
Added:
>
>
  rpm -ivh condor-7.3.2-linux-x86_64-rhel5-1.x86_64.rpm --prefix=/home/condor
Changed:
<
<
cd condor-7.3.2
>
>
  Now, let's install Condor as the central manager since this is the first machine. We must run condor configure_ as root to install and set the file permissions correctly.
Added:
>
>
 
Changed:
<
<
./condor_install --verbose --prefix=/home/condor --local-dir=/home/condor/local --type=manager,execute --owner=condor
>
>
cd condor-7.3.2
./condor_install --verbose --prefix=/home/condor --local-dir=/home/condor/local --type=manager,execute --owner=condor
 
Changed:
<
<
Ok, now that the Central Manager is installed, we should repeat the above installation on our other nodes which will run Condor. However, we modify the installation command slightly for execute and submit systems. If you wanted all three (not recommended for production or large systems), it might look something like this:
>
>
Ok, now that the Central Manager is installed, we should repeat the above installation on our other nodes which will run Condor. However, we modify the installation command slightly for execute and submit systems:
  ./condor_install --verbose --prefix=/home/condor --local-dir=/home/condor/local --type=submit,execute --owner=condor
Added:
>
>

If you wanted all three (not recommended for production or large systems), it might look something like this:

./condor_install --verbose --prefix=/home/condor --local-dir=/home/condor/local --type=manager,submit,execute --owner=condor

Just for good measure, make sure the owner is correct on all condor files:

chown -R condor:condor /home/condor

CONDOR IS NOW INSTALLED! Skip to the next section for configuration.

 

Option 2: Install from Source

WARNING! The instructions to install from source are not complete yet.

Line: 79 to 100
 Ok, that's it for pre-installation questions. We'll be installing Condor on SL#1 first to setup the central information repository.

Let's start installing it:

Added:
>
>
 
Changed:
<
<
su - root

groupadd -g 5000 condor

useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor

>
>
su - root
groupadd -g 5000 condor
useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor
  The user's home directory must be world readable.
Added:
>
>
 
Changed:
<
<
chmod 744 /home/condor

su - condor

>
>
chmod 744 /home/condor
su - condor
  Ok, the condor download site is a little goofy which prevents me from just using wget. Just download the src code tarball "condor_src-7.3.2-all-all.tar.gz" and move it into /home/condor/ on SL#1. You can do this however you want to. We're going to be compiling from source and not using a binary. This allows us greater flexibility for installation.
Added:
>
>
 
Changed:
<
<
gunzip -c condor_src-7.3.2-all-all.tar.gz | tar xvf -

cd condor-7.3.2

>
>
gunzip -c condor_src-7.3.2-all-all.tar.gz | tar xvf -
cd condor-7.3.2
  Let's read the file README.building in this directory. It seems that we need some packages pre-installed. Luckily, SL 5.3 already has everything I need. I checked using yum list <package name> as the root user.

Ok, now let's start compiling.

Added:
>
>
 
Changed:
<
<
cd src

./build_init

>
>
cd src
./build_init
  FAILURE. O M G. It was worth a try.

Required tools are present and valid, attempting to initialize build
configure.ac:2547: /usr/bin/m4: builtin `mkstemp' requested by frozen file is not supported
autom4te: /usr/bin/m4 failed with exit status: 1
autoheader: /usr/bin/autom4te failed with exit status: 1
Failed to initialize build, check errors and try again once fixed

Let's update m4 from source. Login as root again in another window.

Added:
>
>
  wget ftp://ftp.scientificlinux.org/linux/scientific/5x/SRPMS/SL/m4-1.4.8-1.src.rpm

rpm -ivh m4-1.4.8-1.src.rpm

rpmbuild -bb --rebuild /usr/src/redhat/SPECS/m4.spec

Added:
>
>
  My installation is 64-bit, so install this:
Added:
>
>
  rpm -ivhF /usr/src/redhat/RPMS/x86_64/m4-1.4.8-1.x86_64.rpm
Added:
>
>
  The 32-bit equivalent would be:
Added:
>
>
  rpm -ivhF /usr/src/redhat/RPMS/i386/m4-1.4.8-1.i386.rpm
Added:
>
>
  Let's try running the build initialization tool again:
Added:
>
>
  ./build_init
Added:
>
>
  [condor@localhost src]$ ./build_init
Checking for version of autoheader >= 2.59...succeeded. (2.59)
Checking for version of autoconf >= 2.59...succeeded. (2.59)
Required tools are present and valid, attempting to initialize build
Build initialized, you can now run "./configure; make"

SUCCESS!!! Let's enable as many optional features as possible. NOTE: --prefix doesn't work yet.

Added:
>
>
  ./configure --enable-full-port --enable-soft-is-hard --enable-job-hooks --enable-hibernation --enable-ssh-to-job --with-buildid

make

make release

Added:
>
>
  ...TO BE CONTINUED

STEP 3: Configure Condor

Changed:
<
<
The command to configure condor is:
>
>
Now that Condor is installed, it must be configured. At the end of the installation, this message is displayed:

In order for Condor to work properly you must set your CONDOR_CONFIG environment variable to point to your Condor configuration file:
/home/condor/etc/condor_config before running Condor commands/daemons. Created scripts which can be sourced by users to setup their
Condor environment variables. These are:
sh: /home/condor/condor.sh
csh: /home/condor/condor.csh

This means we need to update the environment variable CONDOR_CONFIG to be changed when you login as the condor user:

I just added this line to /home/condor/.bash_profile:

 
Changed:
<
<
condor_configure
>
>
. ~/condor.sh
 
Changed:
<
<
We must also edit the environment variable CONDOR_CONFIG before continuing:
>
>
Let's login as the condor user:

su - condor

Now, we must manually edit the condor configuration file:

vi $CONDOR_CONFIG

So, according to this file, we must fill out Part 1 in order for condor to work. Here are my answers for the SL1 virtual machine:

LOCAL_DIR = /home/condor/local
CONDOR_ADMIN = me@mydomain.com
UID_DOMAIN = $(FULL_HOSTNAME)
FILESYSTEM_DOMAIN = $(FULL_HOSTNAME)
COLLECTOR_NAME = ChrisB? Pool

There is also a required field to edit in Part 2 despite the file stating that Part 2 is optional (oh well):

HOSTALLOW_WRITE = *.mydomain.com

Let's also edit the local configuration for each machine:

vi /home/condor/local/condor_config.local

Here are my changes:

CONDOR_ADMIN = me@mydomain.com

UID_DOMAIN = $(FULL_HOSTNAME)

FILESYSTEM_DOMAIN = $(FULL_HOSTNAME)

 
Deleted:
<
<
CONDOR_CONFIG=/home/condor/etc/condor_config
 

STEP 4: Start Condor

Added:
>
>
As the condor user, let's start the condor daemon:

su - condor

condor_master

STEP 5: Install GlideinWMS

First, we need to verify all the required software is loaded on our system.

 
Changed:
<
<
Start condor daemon.
>
>
  • A reasonably recent Linux OS (RH/SL4 nad RH/SL5 tested at press time).
cat /etc/redhat-release

Scientific Linux SL release 5.3 (Boron)

  • The Python interpreter (v2.3.4 or above)

python -V
Python 2.4.3

  • The perl-Time-HiRes rpm.
perl -e 'use Time::HiRes;'
# NO ERROR - MEANS THAT IT IS INSTALLED
  • The OSG client software.
    • Installed from the GlideinWMS installer by selecting "Components" and then "OSG VDT client".
  • A HTTP server, like Apache or TUX.
    • Installed from the GlideinWMS installer by selecting "Components" and then "Web server".
  • The Condor distribution.
    • This is NOT (?) required if you install: pool Collector AND User Schedd
    • Installed from the GlideinWMS installer by selecting "Components" and then "Base Condor installation".
  • The RRDTool package (v1.2.18 or later)
    • The GlideinWMS installer tells us what to install by selecting "Components" and then "rrdtool graphics package".
    • Since no installation takes place, you have to go find the files on the web. The website that lists the installation RPMS points to broken links. I found the RPMs for my 64-bit installation here:
    • http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/
yum install ruby

# This is installed with --nodeps because it has a cross-dependency with rrdtool
rpm -ivh --nodeps perl-rrdtool-1.3.8-2.el5.rf.x86_64.rpm
rpm -ivh rrdtool-1.3.8-2.el5.rf.x86_64.rpm
rpm -ivh rrdtool-devel-1.3.8-2.el5.rf.x86_64.rpm
rpm -ivh python-rrdtool-1.3.8-2.el5.rf.x86_64.rpm
python -c "import M2Crypto? "
# NO ERROR - MEANS THAT IT IS INSTALLED

Revision 42009/10/08 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"

Installation Walkthrough and Notes on the GlideinWMS System

Line: 6 to 6
  I used Scientific Linux 5.3 from: http://www.scientificlinux.org/
Changed:
<
<
You can also use Redhat Enterprise 5.3 or CentOS 5.3. My installation was on two VMWare Workstation 6.5.3 virtual machines named SL#1 and SL#2. I selected that they would have 512 MiB? of RAM and up to 16 GiB? hard disks. To use GlideinWMS, there is no requirement to use a 64-bit system. You can use a 32-bit and all of the software will be compatible. The reverse is not necessarily true, but I have not confirmed this yet. It's also a good idea to update the system with the latest recommended patches and security updates before continuing any futher. I also reconfigured /etc/ssh/sshd_config to use a different port and adjusted the VM to acquire an IP address from my router in order to make my virtual machines accessible remotely through SSH. Also, I had to edit /etc/resolv.conf to setup DNS after switching to a static IP in the GUI since it wasn't getting it from DHCP anymore.
>
>
You can also use Redhat Enterprise 5.3 or CentOS 5.3. My installation was on two VMWare Workstation 6.5.3 virtual machines named SL#1 and SL#2. I selected that they would have 512 MiB? of RAM and up to 16 GiB? hard disks. To use GlideinWMS, there is no requirement to use a 64-bit system. You can use a 32-bit and all of the software will be compatible. The reverse is not necessarily true, but I have not confirmed this yet. It's also a good idea to update the system with the latest recommended patches and security updates before continuing any futher. I also reconfigured /etc/ssh/sshd_config to use a different port and adjusted the VM to acquire an IP address from my router in order to make my virtual machines accessible remotely through SSH. I had to change /etc/hosts to list the names of my systems and their local IP addresses. Also, I had to edit /etc/resolv.conf to setup DNS after switching to a static IP in the GUI since it wasn't getting it from DHCP anymore.
 

STEP 2: Install Condor

Option 1: Install from Pre-compiled Binary / RPM

This is the typical way most users will work with Condor. It is the fastest and easiest way to get it up and running.

Changed:
<
<
Use a web-browser to download the file "condor-7.3.2-linux-x86_64-rhel5-1.x86_64.rpm" from http://www.cs.wisc.edu/condor/downloads-v2/download.pl
>
>
Use a web-browser to download the file "condor-7.3.2-linux-x86_64-rhel5-dynamic-unstripped.tar.gz" from http://www.cs.wisc.edu/condor/downloads-v2/download.pl
  Copy the RPM to the servers you wish to deploy it on.
Line: 24 to 24
  useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor
Changed:
<
<
Now that the condor user exists, the RPM will use that user's account.
>
>
The user's home directory must be world readable.
 
Changed:
<
<
rpm -ivh condor-7.3.2-linux-x86_64-rhel5-1.x86_64 --prefix=/home/condor
>
>
chmod 744 /home/condor

We'll be installing as root since the documentation recommends this procedure. Now, move the tarball we downloaded into the condor directory.

mv condor-7.3.2-linux-x86_64-rhel5-dynamic-unstripped.tar.gz /home/condor

cd /home/condor

Since we got the tarball, we need to decompress it.

gunzip -c condor-7.3.2-linux-x86_64-rhel5-dynamic-unstripped.tar.gz | tar xvf -

[OPTIONAL & UNTESTED] If we got the RPM, the above command might look something like this instead:

rpm -ivh condor-7.3.2-linux-x86_64-rhel5-1.x86_64.rpm --prefix=/home/condor

cd condor-7.3.2

Now, let's install Condor as the central manager since this is the first machine. We must run condor configure_ as root to install and set the file permissions correctly.

./condor_install --verbose --prefix=/home/condor --local-dir=/home/condor/local --type=manager,execute --owner=condor

Ok, now that the Central Manager is installed, we should repeat the above installation on our other nodes which will run Condor. However, we modify the installation command slightly for execute and submit systems. If you wanted all three (not recommended for production or large systems), it might look something like this:

./condor_install --verbose --prefix=/home/condor --local-dir=/home/condor/local --type=submit,execute --owner=condor

 

Option 2: Install from Source

Added:
>
>
WARNING! The instructions to install from source are not complete yet.
 Since we're setting up a development system, we'll install the latest bleeding edge version which is currently 7.3.2. The installation manual for Condor 7.3.2 can be located here: http://www.cs.wisc.edu/condor/manual/v7.3/3_2Installation.html

The condor manual asks some preparation questions. Here are my answers:

Line: 60 to 86
  useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor
Added:
>
>
The user's home directory must be world readable.

chmod 744 /home/condor

 su - condor

Ok, the condor download site is a little goofy which prevents me from just using wget. Just download the src code tarball "condor_src-7.3.2-all-all.tar.gz" and move it into /home/condor/ on SL#1. You can do this however you want to. We're going to be compiling from source and not using a binary. This allows us greater flexibility for installation.

Line: 108 to 138
  make
Changed:
<
<
make relase
>
>
make release

...TO BE CONTINUED

 

STEP 3: Configure Condor

The command to configure condor is:

Revision 32009/10/07 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"

Installation Walkthrough and Notes on the GlideinWMS System

Line: 8 to 8
  You can also use Redhat Enterprise 5.3 or CentOS 5.3. My installation was on two VMWare Workstation 6.5.3 virtual machines named SL#1 and SL#2. I selected that they would have 512 MiB? of RAM and up to 16 GiB? hard disks. To use GlideinWMS, there is no requirement to use a 64-bit system. You can use a 32-bit and all of the software will be compatible. The reverse is not necessarily true, but I have not confirmed this yet. It's also a good idea to update the system with the latest recommended patches and security updates before continuing any futher. I also reconfigured /etc/ssh/sshd_config to use a different port and adjusted the VM to acquire an IP address from my router in order to make my virtual machines accessible remotely through SSH. Also, I had to edit /etc/resolv.conf to setup DNS after switching to a static IP in the GUI since it wasn't getting it from DHCP anymore.

STEP 2: Install Condor

Added:
>
>

Option 1: Install from Pre-compiled Binary / RPM

This is the typical way most users will work with Condor. It is the fastest and easiest way to get it up and running.

Use a web-browser to download the file "condor-7.3.2-linux-x86_64-rhel5-1.x86_64.rpm" from http://www.cs.wisc.edu/condor/downloads-v2/download.pl

Copy the RPM to the servers you wish to deploy it on.

Let's start installing it:

su - root

groupadd -g 5000 condor

useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor

Now that the condor user exists, the RPM will use that user's account.

rpm -ivh condor-7.3.2-linux-x86_64-rhel5-1.x86_64 --prefix=/home/condor

Option 2: Install from Source

  Since we're setting up a development system, we'll install the latest bleeding edge version which is currently 7.3.2. The installation manual for Condor 7.3.2 can be located here: http://www.cs.wisc.edu/condor/manual/v7.3/3_2Installation.html
Line: 82 to 102
  [condor@localhost src]$ ./build_init
Checking for version of autoheader >= 2.59...succeeded. (2.59)
Checking for version of autoconf >= 2.59...succeeded. (2.59)
Required tools are present and valid, attempting to initialize build
Build initialized, you can now run "./configure; make"
Changed:
<
<
SUCCESS!!!
>
>
SUCCESS!!! Let's enable as many optional features as possible. NOTE: --prefix doesn't work yet.

./configure --enable-full-port --enable-soft-is-hard --enable-job-hooks --enable-hibernation --enable-ssh-to-job --with-buildid

make

make relase

STEP 3: Configure Condor

The command to configure condor is:

condor_configure

We must also edit the environment variable CONDOR_CONFIG before continuing:

CONDOR_CONFIG=/home/condor/etc/condor_config

STEP 4: Start Condor

Start condor daemon.

Revision 22009/10/07 - Main.ChrisBoynton

Line: 1 to 1
 
META TOPICPARENT name="GlideinWMS"
Changed:
<
<
-- ChrisBoynton - 2009/10/06
>
>

Installation Walkthrough and Notes on the GlideinWMS System

 
Deleted:
<
<
Installation Walkthrough and Notes on the GlideinWMS System
 \ No newline at end of file
Added:
>
>

STEP 1: Obtain, install, and update a Linux distribution

I used Scientific Linux 5.3 from: http://www.scientificlinux.org/

You can also use Redhat Enterprise 5.3 or CentOS 5.3. My installation was on two VMWare Workstation 6.5.3 virtual machines named SL#1 and SL#2. I selected that they would have 512 MiB? of RAM and up to 16 GiB? hard disks. To use GlideinWMS, there is no requirement to use a 64-bit system. You can use a 32-bit and all of the software will be compatible. The reverse is not necessarily true, but I have not confirmed this yet. It's also a good idea to update the system with the latest recommended patches and security updates before continuing any futher. I also reconfigured /etc/ssh/sshd_config to use a different port and adjusted the VM to acquire an IP address from my router in order to make my virtual machines accessible remotely through SSH. Also, I had to edit /etc/resolv.conf to setup DNS after switching to a static IP in the GUI since it wasn't getting it from DHCP anymore.

STEP 2: Install Condor

Since we're setting up a development system, we'll install the latest bleeding edge version which is currently 7.3.2. The installation manual for Condor 7.3.2 can be located here: http://www.cs.wisc.edu/condor/manual/v7.3/3_2Installation.html

The condor manual asks some preparation questions. Here are my answers:

  1. What machine will be the central manager? SL#1
  2. What machines should be allowed to submit jobs? SL#1, SL#2
  3. Will Condor run as root or not? Yes, since the installation documentation recommends it.
  4. Who will be administering Condor on the machines in your pool? I will! 8D
  5. Will you have a Unix user named condor and will its home directory be shared? Yes, but it will not be shared.
  6. Where should the machine-specific directories for Condor go? They can live with the rest of the condor application inside the condor daemon's directory.
  7. Where should the parts of the Condor system be installed? Inside the condor directory.
    • Configuration file: maybe in /condor/config
    • Release directory: maybe in /condor/release
      • user binaries: maybe in /condor/bin
      • system binaries: maybe in /condor/sbin
      • lib directory: maybe in /condor/lib
      • etc directory: maybe in /condor/etc
    • Documentation: maybe in /condor/doc
  8. Am I using AFS? Nope
  9. Do I have enough disk space for Condor? Yah, 50 MiB? right?

Ok, that's it for pre-installation questions. We'll be installing Condor on SL#1 first to setup the central information repository.

Let's start installing it:

su - root

groupadd -g 5000 condor

useradd -c "Condor Daemon" -g 5000 -m -s /bin/bash -u 5000 condor

su - condor

Ok, the condor download site is a little goofy which prevents me from just using wget. Just download the src code tarball "condor_src-7.3.2-all-all.tar.gz" and move it into /home/condor/ on SL#1. You can do this however you want to. We're going to be compiling from source and not using a binary. This allows us greater flexibility for installation.

gunzip -c condor_src-7.3.2-all-all.tar.gz | tar xvf -

cd condor-7.3.2

Let's read the file README.building in this directory. It seems that we need some packages pre-installed. Luckily, SL 5.3 already has everything I need. I checked using yum list <package name> as the root user.

Ok, now let's start compiling.

cd src

./build_init

FAILURE. O M G. It was worth a try.

Required tools are present and valid, attempting to initialize build
configure.ac:2547: /usr/bin/m4: builtin `mkstemp' requested by frozen file is not supported
autom4te: /usr/bin/m4 failed with exit status: 1
autoheader: /usr/bin/autom4te failed with exit status: 1
Failed to initialize build, check errors and try again once fixed

Let's update m4 from source. Login as root again in another window.

wget ftp://ftp.scientificlinux.org/linux/scientific/5x/SRPMS/SL/m4-1.4.8-1.src.rpm

rpm -ivh m4-1.4.8-1.src.rpm

rpmbuild -bb --rebuild /usr/src/redhat/SPECS/m4.spec

My installation is 64-bit, so install this:

rpm -ivhF /usr/src/redhat/RPMS/x86_64/m4-1.4.8-1.x86_64.rpm

The 32-bit equivalent would be:

rpm -ivhF /usr/src/redhat/RPMS/i386/m4-1.4.8-1.i386.rpm

Let's try running the build initialization tool again:

./build_init

[condor@localhost src]$ ./build_init
Checking for version of autoheader >= 2.59...succeeded. (2.59)
Checking for version of autoconf >= 2.59...succeeded. (2.59)
Required tools are present and valid, attempting to initialize build
Build initialized, you can now run "./configure; make"

SUCCESS!!!

Revision 12009/10/06 - Main.ChrisBoynton

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="GlideinWMS"
-- ChrisBoynton - 2009/10/06

Installation Walkthrough and Notes on the GlideinWMS System

 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback