网上针对hadoop2.x的编译教程很多:这里给几个链接
http://www.centoscn.com/image-text/install/2014/0401/2685.html
http://www.iyunv.com/thread-38384-1-1.html
这里摘抄一些:
编译hadoop2.6.0源码
1、准备工作
1.1、安装jdk、gcc、gcc-c++ 、make、cmake、openssl-devel、ncurses-devel
yum install gcc
yum intall gcc-c++
yum install make
yum install cmake
yum install openssl-devel
yum install ncurses-devel
1.2、安装maven
http://mirror.bit.edu.cn/apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.zip
tar zxf apache-maven-3.0.5-bin.tar.gz(已编译版本)
mv apache-maven-3.0.5 /usr/local
vi /etc/profile
export m2_home=/usr/local/apache-maven-3.0.5
export m2=$m2_home/bin
export maven_opts=-xms256m -xmx512m
export path=$m2:$path
. /etc/profile
mvn -version
修改配置文件
vi /usr/local/apache-maven-3.0.5/conf/settings.xml
里添加,其他的不需改动,具体修改如下:
nexus-osc
*
nexusosc
http://maven.oschina.net/content/groups/public/
在内新添加
jdk-1.7
1.7
nexus
local private nexus
http://maven.oschina.net/content/groups/public/
true
false
nexus
local private nexus
http://maven.oschina.net/content/groups/public/
true
false
1.3、安装protobuf
https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz
tar xvf protobuf-2.5.0.tar.gz
cd protobuf-2.5.0
./configure
make
make install
ldconfig
vi /etc/profile
export ld_library_path=~/protobuf-2.5.0
. /etc/profile
protoc --version
1.4、安装ant
http://mirrors.cnnic.cn/apache//ant/binaries/apache-ant-1.9.4-bin.tar.gz
tar xvf apache-ant-1.9.4-bin.tar.gz
mv apache-ant-1.9.4 /usr/local
vi /etc/profile
export ant_home=/usr/local/apache-ant-1.9.4
export path=$java_home/bin:$path:$hadoop_prefix/bin:$path:$m2:$path:$ant_home/bin
. /etc/profile
ant -version
2、编译
进入源码文件夹
cd /hadoop/hadoop-2.6.0-src
mvn clean
mvn package -pdist,native -dskiptests -dtar
......
[info] apache hadoop pipes ............................... success [11.372s]
[info] apache hadoop openstack support ................... success [7.403s]
[info] apache hadoop amazon web services support ......... success [2:22.913s]
[info] apache hadoop client .............................. success [11.900s]
[info] apache hadoop mini-cluster ........................ success [0.660s]
[info] apache hadoop scheduler load simulator ............ success [10.072s]
[info] apache hadoop tools dist .......................... success [14.665s]
[info] apache hadoop tools ............................... success [0.038s]
[info] apache hadoop distribution ........................ success [1:14.706s]
[info] ------------------------------------------------------------------------
[info] build success
[info] ------------------------------------------------------------------------
[info] total time: 36:42.709s
[info] finished at: wed dec 10 00:54:26 cst 2014
[info] final memory: 135m/433m
[info] ------------------------------------------------------------------------
编译结果
/hadoop/hadoop-2.6.0-src/hadoop-dist/target
[root@lsn-linux target]# pwd
/hadoop/hadoop-2.6.0-src/hadoop-dist/target
[root@lsn-linux target]# ls
antrun hadoop-2.6.0.tar.gz maven-archiver
dist-layout-stitching.sh hadoop-dist-2.6.0.jar test-dir
dist-tar-stitching.sh hadoop-dist-2.6.0-javadoc.jar
hadoop-2.6.0 javadoc-bundle-options
[root@lsn-linux target]#
以上来自百度文库http://wenku.baidu.com/link?url=otisa_6bn9ilighqt4rw6umemw9mq-zejoejhgykmqrl5x1wlsmfbwzqgxs1nqkdri3syvp2vd9uv3sggxceq4dq4mmp9jaldkbwvu-kyyo
谢谢!
我在编译中出现的问题:
首先得让虚拟机连外网,我用的是vm_ware,在vm_ware软件的编辑栏里,有虚拟机设置选项,在里面设置为桥接
1、下载的软件版本不对
比如说hadoop2.6.0-src.tar.gz,下成hadoop2.6.0.tar.gz;
apache-maven-bin.tar.gz下成apache-maven.tar.gz等
另外,我装的是jdk1.8的版本,maven是3.0.5的版本,好像这两个版本不兼容,总是会抛出找不到java 库类的问题。后来把maven升级到最新版3.2.x就可以了。
在编译的过程中,会出错。这里截个图:
1) unexpected end tag:
[error] failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar (module-javadocs) on project hadoop-annotations: mavenreportexception: error while creating archive:
[error] exit code: 1 - /root/hadoop-2.4.0-src/hadoop-common-project/hadoop-annotations/src/main/java/org/apache/hadoop/classification/interfacestability.java:27: error: unexpected end tag:
[error] *
[error] ^
[error]
[error] command line was: /root/jdk1.8.0/jre/../bin/javadoc @options @packages
原因是interfacestability.java中的注释问题:
解决办法,将jdk换成1.7版本,使用jdk1.8编译就会遇到上述问题,将行删除可以解决问题,但后续还会遇到类似的问题,所以不要使用jdk1.8编译hadoop 2.6.0。
2、编辑maven的settings.xml出问题
建议在ultraedit上编辑,因为这样会将正文和注释很清楚的区别开。我漏了前后的;
3、网速有时候很不给力,需要多重启几下机子