Difference: HdfsXrootd (10 vs. 11)

Revision 112013/12/08 - Main.JeffreyDost

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HdfsXrootd Development Twiki

Line: 68 to 68
 

Build hdfs-xrootd-fallback

cd $PROJDIR/hdfs-xrootd-fallback
Changed:
<
<
make -f Makefile_rpm
>
>
make
 

This will generate the following files:

Line: 90 to 90
  Assumptions - a hadoop cluster is available and a hadoop client node is properly configured to access it. The following instructions are only required on the client node.
Changed:
<
<

Copy Binaries

>
>

Install Binaries

make install prefix=/usr sysconfdir=/etc
 
Changed:
<
<
Replace:
/usr/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.1.1.jar
Copy regenfs.jar to:
/usr/lib/hadoop/client/
Copy libXrdBlockFetcher.so to:
/usr/lib/
>
>
This will install the following:
/usr/lib/hadoop/client/hdfs-xrootd-fallback-1.0.0.jar
/usr/lib64/libXrdBlockFetcher.so
/usr/lib64/libXrdBlockFetcher.so.1
/usr/lib64/libXrdBlockFetcher.so.1.0.0

Also a template config file will be installed in:

/etc/hadoop/conf.osg/xfbfs-site.xml
 

Configuration

Append the following to /etc/hadoop/conf/core-site.xml:

  <property>
Changed:
<
<
fs.regfs.impl regenfs.RegenerativeFileSystem
>
>
fs.xfbfs.impl org.xrootd.hdfs.fallback.XrdFallBackFileSystem
 
Added:
>
>
Copy the /etc/hadoop/conf.osg/xfbfs-site.xml template file to /etc/hadoop/conf and modify the relevant values.
 

Startup Fuse in Debug Mode

Changed:
<
<
hadoop-fuse-dfs -oserver=regfs://cabinet-10-10-3.t2.ucsd.edu,port=8020,allow_other,rw,rdbuffer=4096 -d /mnt/hadoop
>
>
hadoop-fuse-dfs -oserver=xfbfs://cabinet-10-10-3.t2.ucsd.edu,port=8020,allow_other,rw,rdbuffer=4096 -d /mnt/hadoop
 

Useful Test Commands

 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback