A set of CMS tools are deployed locally to provide the critical infrastructure to support the grid-based analysis.

1. DBS-2

A local machine (ming.ucsd.edu) is dedicatd to DBS-2 service. The access of "local" DBS-2 is http://ming.ucsd.edu:8080/DBS/servlet/DBSServlet. The deployment of this service largely follows the standard procedure descrbed in the official DBS webpage. Some tunings of the configuration might be needed, but in general the standard procedure is rather general to various local settings based on our experience.

To simplify the deployment, configuration and operation, all the components are deployed in one directory /DBS2 in this machine (min.ucsd.edu), a CentOS? -4 box with kernel 2.6.9-34.0.1.EL.

1.1 DBS server and client

Following script is used to install the DBS server or client.

 #!/bin/bash

 dbs_dir=/home/hpi/DBS2/dbs_server
 mkdir ${dbs_dir}
 cd ${dbs_dir}
 dbs_ver=DBS_1_0_2

 export VO_CMS_SW_DIR=$PWD
 wget -O $VO_CMS_SW_DIR/aptinstaller.sh http://cmsdoc.cern.ch/cms/cpt/Software/download/cms/aptinstaller.sh
 chmod +x $VO_CMS_SW_DIR/aptinstaller.sh
 $VO_CMS_SW_DIR/aptinstaller.sh setup -path $VO_CMS_SW_DIR
 eval `$VO_CMS_SW_DIR/aptinstaller.sh config -path  $VO_CMS_SW_DIR -sh`

 apt-get update
 apt-get dist-upgrade
 apt-cache search dbs

 # Deployment commands look like:
 apt-get install cms+dbs-server+${dbs_ver}
 #apt-get install cms+dbs-client+v{dbs_ver}

 # Client deployment:
 #apt-get install cms+dbs-client+DBS_1_0_1

The DBS environment can be initiated with script as following (which is useful after re-login and operation). This configuration mostly comes from DBS_INSTALLATION_DIR/cms/dbs-server/DBS_1_0_2/etc/profile.d//dependencies-setup.sh, but it is useful to check all the packages that are installed by aptinstaller of CMS software release tool and include all of them if needed.

export installation_dir=/home/hpi/DBS2/dbs_server/slc3_ia32_gcc323
source ${installation_dir}/external/java-jdk/1.5.0.p6/etc/profile.d/init.sh
source ${installation_dir}/external/zlib/1.1.4/etc/profile.d/init.sh
source ${installation_dir}/external/apache-ant/1.6.5/etc/profile.d/init.sh
source ${installation_dir}/external/apache-tomcat/5.5.20/etc/profile.d/init.sh
source ${installation_dir}/external/bz2lib/1.0.2/etc/profile.d/init.sh
source ${installation_dir}/external/db4/4.4.20/etc/profile.d/init.sh
source ${installation_dir}/external/expat/2.0.0/etc/profile.d/init.sh
source ${installation_dir}/external/gcc/3.2.3/etc/profile.d/init.sh
source ${installation_dir}/external/gdbm/1.8.3/etc/profile.d/init.sh
source ${installation_dir}/external/mysql/5.0.18/etc/profile.d/init.sh
source ${installation_dir}/external/openssl/0.9.7d/etc/profile.d/init.sh
source ${installation_dir}/external/python/2.4.2/etc/profile.d/init.sh

source ${installation_dir}/cms/dbs-server/DBS_1_0_2/etc/profile.d/init.sh
export JAVA_HOME=${installation_dir}/external/java-jdk/1.5.0.p6
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib

1.2 MySQL? DB

Following script is used to install the MySQL? and check out the schema of DBS.

#!/bin/bash

export base_dir=/home/hpi/DBS2/dbs_test
cd ${base_dir}
mkdir  dbstest
export CVSROOT=:pserver:anonymous@cmscvs.cern.ch:/cvs_server/repositories/CMSSW
export CVS_RSH=ssh
cvs -d `echo $CVSROOT | awk -F@ '{print $1":98passwd\@"$2}'` login
cvs co -r DBS_1_0_2 DBS/Schema/NeXtGen

Following script is to initiate the MySQL?

#!/bin/bash

export dbs_test_dir=/home/hpi/DBS2/dbs_test
export mysql_dir=/home/hpi/DBS2/MySQL/work

cd ${dbs_test_dir}
cd DBS/Schema/NeXtGen

mysql_install_db --datadir=$mysql_dir
mysqld_safe --datadir=$mysql_dir --log-error=$mysql_dir/error.log --pid-file=$mysql_dir/mysqld.pid &

MySQL? environment can be initiated by following script. It has been found the some dependent packages of DBS might be conflict with MySQL? and cause MySQL? to crash during the initiation. The exact cause is under investigation. It is useful to remove some packages which MySQL? doesn't need and simplify the environment and configuration.

export installation_dir=/home/hpi/DBS2/dbs_server/slc3_ia32_gcc323
source ${installation_dir}/external/gcc/3.2.3/etc/profile.d/init.sh
source ${installation_dir}/external/mysql/5.0.18/etc/profile.d/init.sh
source ${installation_dir}/external/openssl/0.9.7d/etc/profile.d/init.sh
source ${installation_dir}/external/python/2.4.2/etc/profile.d/init.sh
source ${installation_dir}/cms/dbs-server/DBS_1_0_2/etc/profile.d/init.sh

export JAVA_HOME=${installation_dir}/external/java-jdk/1.5.0.p6

export LD_LIBRARY_PATH=/lib:${LD_LIBRARY_PATH}

The DBS-2 schema can be imported to MySQL? via following script:

#!/bin/bash

export dbs_test_dir=/home/hpi/DBS2/dbs_test
export mysql_dir=/home/hpi/DBS2/MySQL/work
cd ${dbs_test_dir}
cd DBS/Schema/NeXtGe

#mysql --socket=$mysql_dir/mysql.sock < DBS-NeXtGen-MySQL_DEPLOYABLE.sql
mysql --user=root --password=atest -h ming.ucsd.edu < DBS-NeXtGen-MySQL_DEPLOYABLE.sql
 

Deletion and re-installation of MySQL? DB is simple, remove the MySQL? DB directory and repeat above procedure. In this case,

rm -rf /home/hpi/DBS2/MySQL/work/*

1.3 Web Interface: TomCat?

TomCat? can be downloaded from http://tomcat.apache.org. Following the installation guide of TomCat? , the java servlet should be up and running in the deployment machin. In this case, by accessing http://ming.ucsd.edu:8080 via a web browser.

After testing, tomcat-5 seems working better with DBS server. The status of DBS server working with TomCat? -6 needs to be confirmed by the DBS development team. The local deployment is based on apache-tomcat-5.5.23.

The server deployment is almost the same as the standard procedure. The MySQL? jdbc driver needs to be copied to $CATLINA_HOME/lib, which can be found from the DBS lib from the release.

The configuration of DBS server needs some modification of /Servers/JavaServer/etc/context.xml. In this case, it looks like:

   <Context path="/servlet/DBSServlet" docBase="DBSServlet" debug="5" reloadable="true" crossContext="true">
         <!--Loacl DBS MySQL Resource Parameter /-->
         <Resource name="jdbc/dbs"
                 auth="Container"
                 type="javax.sql.DataSource"
                 maxActive="0"
                 maxIdle="1"
                 maxWait="-1"
                 username="root"
                 password="atest"
                 driverClassName="org.gjt.mm.mysql.Driver"  
  url="jdbc:mysql://ming.ucsd.edu:3306/dbs_new_era_v17_PA05012007?autoReconnect=true"/>
     <SupportedSchemaVersion schemaversion="v00_00_06" />
  <SupportedClientVersions clientversions="v00_00_06, v00_00_06" />
  <DBSBlockConfig maxBlockSize="2000000000000" maxBlockFiles="100" />
  </Context>

The rest of the deployment is:

 ant dist
 cp DBS.war $CATLINA_HOME/webapps

A DBS directory will be automatically created by java in the $CATLINA_HOME/webapps.

1.4 Validation Test

Following the standard procedure, the server test can be done via:

         cd <DBS_SERVER_DIR>/DBS/Server/JavaServer/test
         ./runAllTest.sh

The end-to-end client-server test can be done by deploying a DBS client and running the test script from the DBS client:

         cd <DBS_CLIENT_DIR>/Clients/Python/DBSAPI/UnitTests
         chmod +x runAllTests.sh
         ./runAllTests.sh  

2. ProdAgent?

3. GlideinWMS

4. Crab

5. Integrating general grid jobs with CMS Dash Board

-- HaifengPi - 18 Jul 2007