A set of CMS tools are deployed locally to provide the critical infrastructure to support the grid-based analysis.
1. DBS-2
A local machine (ming.ucsd.edu) is dedicatd to DBS-2 service. The access of "local" DBS-2 is
http://ming.ucsd.edu:8080/DBS/servlet/DBSServlet. The deployment of this service largely follows the standard procedure descrbed in the official
DBS webpage. Some tunings of the configuration might be needed, but in general the standard procedure is rather general to various local settings based on our experience.
To simplify the deployment, configuration and operation, all the components are deployed in one directory /DBS2 in this machine (min.ucsd.edu), a
CentOS? -4 box with kernel 2.6.9-34.0.1.EL.
1.1 DBS server and client
Following script is used to install the DBS server or client.
#!/bin/bash
dbs_dir=/home/hpi/DBS2/dbs_server
mkdir ${dbs_dir}
cd ${dbs_dir}
dbs_ver=DBS_1_0_2
export VO_CMS_SW_DIR=$PWD
wget -O $VO_CMS_SW_DIR/aptinstaller.sh http://cmsdoc.cern.ch/cms/cpt/Software/download/cms/aptinstaller.sh
chmod +x $VO_CMS_SW_DIR/aptinstaller.sh
$VO_CMS_SW_DIR/aptinstaller.sh setup -path $VO_CMS_SW_DIR
eval `$VO_CMS_SW_DIR/aptinstaller.sh config -path $VO_CMS_SW_DIR -sh`
apt-get update
apt-get dist-upgrade
apt-cache search dbs
# Deployment commands look like:
apt-get install cms+dbs-server+${dbs_ver}
#apt-get install cms+dbs-client+v{dbs_ver}
# Client deployment:
#apt-get install cms+dbs-client+DBS_1_0_1
The DBS environment can be initiated with script as following (which is useful after re-login and operation). This configuration mostly comes from DBS_INSTALLATION_DIR/cms/dbs-server/DBS_1_0_2/etc/profile.d//dependencies-setup.sh, but it is useful to check all the packages that are installed by aptinstaller of CMS software release tool and include all of them if needed.
export installation_dir=/home/hpi/DBS2/dbs_server/slc3_ia32_gcc323
source ${installation_dir}/external/java-jdk/1.5.0.p6/etc/profile.d/init.sh
source ${installation_dir}/external/zlib/1.1.4/etc/profile.d/init.sh
source ${installation_dir}/external/apache-ant/1.6.5/etc/profile.d/init.sh
source ${installation_dir}/external/apache-tomcat/5.5.20/etc/profile.d/init.sh
source ${installation_dir}/external/bz2lib/1.0.2/etc/profile.d/init.sh
source ${installation_dir}/external/db4/4.4.20/etc/profile.d/init.sh
source ${installation_dir}/external/expat/2.0.0/etc/profile.d/init.sh
source ${installation_dir}/external/gcc/3.2.3/etc/profile.d/init.sh
source ${installation_dir}/external/gdbm/1.8.3/etc/profile.d/init.sh
source ${installation_dir}/external/mysql/5.0.18/etc/profile.d/init.sh
source ${installation_dir}/external/openssl/0.9.7d/etc/profile.d/init.sh
source ${installation_dir}/external/python/2.4.2/etc/profile.d/init.sh
source ${installation_dir}/cms/dbs-server/DBS_1_0_2/etc/profile.d/init.sh
export JAVA_HOME=${installation_dir}/external/java-jdk/1.5.0.p6
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib
1.2 MySQL? DB
Following script is used to install the
MySQL? and check out the schema of DBS.
#!/bin/bash
export base_dir=/home/hpi/DBS2/dbs_test
cd ${base_dir}
mkdir dbstest
export CVSROOT=:pserver:anonymous@cmscvs.cern.ch:/cvs_server/repositories/CMSSW
export CVS_RSH=ssh
cvs -d `echo $CVSROOT | awk -F@ '{print $1":98passwd\@"$2}'` login
cvs co -r DBS_1_0_2 DBS/Schema/NeXtGen
Following script is to initiate the
MySQL?
#!/bin/bash
export dbs_test_dir=/home/hpi/DBS2/dbs_test
export mysql_dir=/home/hpi/DBS2/MySQL/work
cd ${dbs_test_dir}
cd DBS/Schema/NeXtGen
mysql_install_db --datadir=$mysql_dir
mysqld_safe --datadir=$mysql_dir --log-error=$mysql_dir/error.log --pid-file=$mysql_dir/mysqld.pid &
MySQL? environment can be initiated by following script. It has been found the some dependent packages of DBS might be conflict with
MySQL? and cause
MySQL? to crash during the initiation. The exact cause is under investigation. It is useful to remove some packages which
MySQL? doesn't need and simplify the environment and configuration.
export installation_dir=/home/hpi/DBS2/dbs_server/slc3_ia32_gcc323
source ${installation_dir}/external/gcc/3.2.3/etc/profile.d/init.sh
source ${installation_dir}/external/mysql/5.0.18/etc/profile.d/init.sh
source ${installation_dir}/external/openssl/0.9.7d/etc/profile.d/init.sh
source ${installation_dir}/external/python/2.4.2/etc/profile.d/init.sh
source ${installation_dir}/cms/dbs-server/DBS_1_0_2/etc/profile.d/init.sh
export JAVA_HOME=${installation_dir}/external/java-jdk/1.5.0.p6
export LD_LIBRARY_PATH=/lib:${LD_LIBRARY_PATH}
The DBS-2 schema can be imported to
MySQL? via following script:
#!/bin/bash
export dbs_test_dir=/home/hpi/DBS2/dbs_test
export mysql_dir=/home/hpi/DBS2/MySQL/work
cd ${dbs_test_dir}
cd DBS/Schema/NeXtGe
#mysql --socket=$mysql_dir/mysql.sock < DBS-NeXtGen-MySQL_DEPLOYABLE.sql
mysql --user=root --password=atest -h ming.ucsd.edu < DBS-NeXtGen-MySQL_DEPLOYABLE.sql
Deletion and re-installation of
MySQL? DB is simple, remove the
MySQL? DB directory and repeat above procedure. In this case,
rm -rf /home/hpi/DBS2/MySQL/work/*
1.3 Web Interface: TomCat?
TomCat? can be downloaded from
http://tomcat.apache.org. Following the installation guide of
TomCat? , the java servlet should be up and running in the deployment machin. In this case, by accessing
http://ming.ucsd.edu:8080 via a web browser.
After testing, tomcat-5 seems working better with DBS server. The status of DBS server working with
TomCat? -6 needs to be confirmed by the DBS development team. The local deployment is based on apache-tomcat-5.5.23.
The server deployment is almost the same as the
standard procedure. The
MySQL? jdbc driver needs to be copied to $CATLINA_HOME/lib, which can be found from the DBS lib from the release.
The configuration of DBS server needs some modification of
/Servers/JavaServer/etc/context.xml. In this case, it looks like:
<Context path="/servlet/DBSServlet" docBase="DBSServlet" debug="5" reloadable="true" crossContext="true">
<!--Loacl DBS MySQL Resource Parameter /-->
<Resource name="jdbc/dbs"
auth="Container"
type="javax.sql.DataSource"
maxActive="0"
maxIdle="1"
maxWait="-1"
username="root"
password="atest"
driverClassName="org.gjt.mm.mysql.Driver"
url="jdbc:mysql://ming.ucsd.edu:3306/dbs_new_era_v17_PA05012007?autoReconnect=true"/>
<SupportedSchemaVersion schemaversion="v00_00_06" />
<SupportedClientVersions clientversions="v00_00_06, v00_00_06" />
<DBSBlockConfig maxBlockSize="2000000000000" maxBlockFiles="100" />
</Context>
The rest of the deployment is:
ant dist
cp DBS.war $CATLINA_HOME/webapps
A DBS directory will be automatically created by java in the $CATLINA_HOME/webapps.
1.4 Validation Test
Following the standard procedure, the server test can be done via:
cd <DBS_SERVER_DIR>/DBS/Server/JavaServer/test
./runAllTest.sh
The end-to-end client-server test can be done by deploying a DBS client and running the test script from the DBS client:
cd <DBS_CLIENT_DIR>/Clients/Python/DBSAPI/UnitTests
chmod +x runAllTests.sh
./runAllTests.sh
2. ProdAgent?
4. Crab
5. Integrating general grid jobs with CMS Dash Board
-- HaifengPi - 18 Jul 2007