Quick Reference

  • UCSD - Hadoop
    • srm://bsrm-1.t2.ucsd.edu:8443/srm/v2/server?SFN=
    • dcap://dcap-2.t2.ucsd.edu:22140/
    • /hadoop/cms/store/user/mwlebour
  • UCSD
    • srm://srm-3.t2.ucsd.edu:8443/srm/managerv2?SFN=
    • /pnfs/t2.ucsd.edu/data4/cms/phedex/store/user/mlebourgeois
    • dcap://dcap-2.t2.ucsd.edu:22138
  • FNAL
    • srm://cmssrm.fnal.gov:8443
      • 11/pnfs
      • 2/mwlebour
      • resilient/mwlebour
    • /pnfs/cms/WAX
  • CERN
    • srm://srm-cms.cern.ch:8443/srm/managerv1?SFN=
    • /castor/cern.ch/cms
    • /castor/cern.ch/user/m/mwlebour

Jake's Explicit dCache Tutorial

  • First time users:
    • mkdir /pnfs/cms/WAX/2/< USERNAME >
    • Make sure that the group is us_cms: chown < USERNAME >:us_cms /pnfs/cms/WAX/2/< USERNAME >
    • Then make sure that it is group writable: chmod g+w /pnfs/cms/WAX/2/< USERNAME >
  • Limit actions within /pnfs/cms/ to mkdir, rmdir, ls, rm, dccp and srcmp
  • Don't cd in /pnfs/cms
  • Don't run find commands
  • Basically don't do any unnecessary access commands, dCache is on tape, and therefore slow.
  • Do not trust ls.

Reading from dCache

  • It is quite easy.
  • It is easy to open files in dCache with ROOT just prefix the absolute path with a dcache:
    • or starting a new ROOT session: root dcap://dcopy-1.local:22138/pnfs/sdsc.edu/data3/cms/phedex/store/users/mlebourgeois/fileForAccessing.root
  • To copy a file from dCache, do not you cp, use dccp.
    • dccp works just like cp, except that you are only allowed to specify one file at a time.
    • dccp is only used for copying files OUT of dCache, not into dCache

Writing to dCache

  • You will need a GRID certificate to write to dCache so get one HERE.
  • The best way to use it is to create a copyjobfile
  • srmcp -copyjobfile=
  • copyjobfile must have on each line a path for the HERE file and the THERE file
    • local files must be formatted as: file:///
    • dCache files must be prefixed with for FNAL: srm://cmssrm.fnal.gov:8443/2/< USERNAME >/ ...
    • dCache files must be prefixed with for UCSD: srm://t2data2.t2.ucsd.edu:8443/srm/managerv1?SFN=/pnfs/sdsc.edu/data3/cms/phedex/store/users/mlebourgeois/
must be writable by us_cms group
    • /store/ files at
      • CERN: srm://srm.cern.ch:8443/srm/managerv1?SFN=/castor/cern.ch/cms/store...
      • FNAL: srm://cmssrm.fnal.gov:8443/11/store/...

UCSD Stuff

  • Personal files: /pnfs/t2.ucsd.edu/data4/cms/phedex/store/user/mlebourgeois
  • Read only PHEDEX: /pnfs/t2.ucsd.edu/data3/cms/phedex
  • If doing high volume transfers, please create new (sub)subdirectories for every ~200 files. The most optimum file size is 1 to 2 GB. As you can expect, very small files, O(10 MB), but too many in number, may degrade performance. Thus, it is often recommended to pack them together for easy data/transfer management.
  • srm://t2data2.t2.ucsd.edu:8443/srm/managerv1?SFN=/pnfs/sdsc.edu/data3/cms/phedex/store/users/mlebourgeois//

First Source

-- MatthewLeBourgeois - 05 Sep 2007

Topic revision: r16 - 2010/02/12 - 09:04:01 - MatthewLeBourgeois
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback