编辑源代码 native libraries

安装maven

Wget –c http://mirror.bit.edu.cn/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz

tar -zxvf apache-maven-3.3.9-bin.tar.gz -C /home/hadoop

Vim /etc/profile

export MAVEN_HOME=/home/hadoop/apache-maven-3.3.9

export PATH= 添加 :$MAVEN_HOME/bin

maven 编译的时候需要jdk是1.7版本，所以需要在安装一个1.7版本的jdk

tar –zxvf jdk-7u55-linux-x64.tar.gz –C /usr/java/

Vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.7.0_55

之后开始mvn编译

Tar –zxvf hadoop-2.6.0-cdh5.5.2-src.tar.gz

Cd hadoop-2.6.0-cdh5.5.2

yum install –y *gcc* ncurses-devel openssl-devel cmake autoconfautomake libtool bzip2-devel g++ autoconf automake libtool cmake zlib1g-dev pkg-config

mvn package -Pdist,native -DskipTests –Dtar

编译成功之后，把 native 目录下的lib包copy到 hadoop的安装目录下迟到lib目录
cp –a hadoop-dist/target/hadoop-2.7.2/lib/native/*  $HADOOP_HOME/lib/native/

报错问题整理：

hadoop-common —

[WARNING] [protoc, –version] failed:java.io.IOException: Cannot run program "protoc": error=2, No suchfile or directory

[ERROR] stdout: []

解决办法：

Tar –zxvf protobuf-2.5.0.tar.gz

Cd protobuf-2.5.0

./configure

Make

Make install

protoc –version
protoc: error while loading shared libraries: libprotobuf.so.8: cannot open shared object file: No such file or directory

ls /usr/local/lib

echo "/usr/local/lib"  > /etc/ld.so.conf.d/protocbuf.conf

Ldconfig

之后在执行 protoc –version
libprotoc 2.5.0

出现这个就表示好了， 之后在继续编译

编译也有可能出现以下错误：

[INFO] BUILD FAILURE
[INFO] ————————————————————————
[INFO] Total time: 4:34.649s
[INFO] Finished at: Tue Jun 03 10:18:25 CST 2014
[INFO] Final Memory: 39M/584M
[INFO] ————————————————————————
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:
[ERROR] /opt/linshuai/tardir/hadoop-2.2.0-src/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[88,11] error: cannot access AbstractLifeCycle
[ERROR] class file for org.mortbay.component.AbstractLifeCycle not found
[ERROR] /tardir/hadoop-2.2.0-src/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[96,29] error: cannot access LifeCycle
[ERROR] class file for org.mortbay.component.LifeCycle not found
[ERROR] /tardir/hadoop-2.2.0-src/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[98,10] error: cannot find symbol
[ERROR] symbol: method start()
[ERROR] location: variable server of type Server
[ERROR] /tardir/hadoop-2.2.0-src/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[104,12] error: cannot find symbol
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hadoop-auth

问题解决 https://issues.apache.org/jira/browse/HADOOP-10110：

Index: hadoop-common-project/hadoop-auth/pom.xml

===================================================================

— hadoop-common-project/hadoop-auth/pom.xml                    (revision 1543124)

+++ hadoop-common-project/hadoop-auth/pom.xml                 (working copy)

@@ -54,6 +54,11 @@

     </dependency>

     <dependency>

       <groupId>org.mortbay.jetty</groupId>

+      <artifactId>jetty-util</artifactId>

+      <scope>test</scope>

+    </dependency>

+    <dependency>

+      <groupId>org.mortbay.jetty</groupId>

       <artifactId>jetty</artifactId>

       <scope>test</scope>

     </dependency>

然后mvn package -Pdist -DskipTests -rf :hadoop-auth

这里-rf 是只编译失败的项目。

最后检查，copy之后，可以用下面命令检查hadoop支持的格式

hadoop checknative –a

可以查看 支持的压缩属性