Admin Documentation

Puppetize brick

Open firewall in uclhcpup for ports 8140, 8443, 443; see /etc/firewalld/zones/public.xml

rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

yum install puppet

Modify /etc/puppet/puppet.conf and fill in the relevant hostname:

[main]
vardir = /var/lib/puppet
logdir = /var/log/puppet
rundir = /var/run/puppet
ssldir = $vardir/ssl


[agent]
pluginsync      = true
report          = true  
ignoreschedules = true
daemon          = false
ca_server       = uclhcpup.t2.ucsd.edu
certname        = hostname
environment     = production
server          = uclhcpup.t2.ucsd.edu
configtimeout   = 10m

Start puppet:

puppet resource service puppet ensure=running enable=true

On uclhcpup:

Look for and sign hostname:

puppet cert list
puppet cert sign hostname

Create relevant yaml in /etc/puppetlabs/code/environments/production/hieradata/nodes/

Generate a node specific ssh auth keys file from /etc/puppetlabs/code/environments/production/modules/uclhc_admin/files/root.ssh

Ensure host certs are copied and named appropriately into /etc/puppetlabs/code/environments/production/modules/host_cert/files/ and check permissions

In foreman, assign relevant host group (UCLHC/ATLAS / UCLHC/CMS):
hosts -> click host -> edit -> hostgroup pulldown

click submit.

To apply classes immediately, run:

puppet agent --test

Brick Yum Update

We pin some packages to specific version numbers and only upgrade through puppet, these should be excluded in a normal yum update:

yum update --exclude=condor* --exclude=glideinwms* 

GIT Change Management

UCLHC puppet modules are hosted at: https://github.com/jdost321/uclhc-puppet

The master branch corresponds to the production puppet environment, and is cloned at /etc/uclhc-puppet/production. The development branch corresponds to the development puppet environment and is cloned at /etc/uclhc-puppet/development. Changes should be made in /etc/uclhc-puppet/development and tested on the uclhctest.t2.ucsd.edu host before pushed to production.

Assuming changes are committed and pushed back to github from development, the following steps merge the changes to production:

cd /etc/uclhc-puppet/production
git fetch
git merge origin/development
git push

Upgrade OSG Version

1. (brick) Do a yum update as described above
2. (brick) Turn off puppet agent: service puppet stop
3. (uclhcpup) Bump hiera osg version and optionally condor and gwms in the relevant node yaml file in /etc/puppetlabs/code/environments/production/hieradata/nodes:

   osg_repos::release: x
   condor::condor_version: y
   condor::gwms_version: z
   

4. (brick) puppet agent --test errors are harmless, this is just to get the latest osg-release
5. (brick) yum clean all
6. (brick) puppet agent --test a second time
7. (brick) Do a second yum update as described above
8. (brick) Turn on puppet agent: service puppet start

-- JeffreyDost - 2016/02/09