2014年2月7日 星期五

[研究] hadoop-2.2.0-src.tar.gz 快速編譯安裝程式(二)(CentOS 6.5 x86_64)

[研究] hadoop-2.2.0-src.tar.gz 快速編譯安裝程式(二)(CentOS 6.5 x86_64)

2014-02-07

這篇和前一篇的差異是改用 Apache Maven 3.1.1,而不用 3.0.5 版
[研究] hadoop-2.2.0-src.tar.gz 快速編譯安裝程式(CentOS 6.5 x86_64)
http://shaurong.blogspot.tw/2014/02/hadoop-220-srctargz-centos-65-x8664_7.html
(因為網路上有人說不能用 3.1.1 版)

請先切換成 root 後執行

[root@localhost ~]# su  root
 [root@localhost ~]# vi  Hadoop-2.2.0_CentOS-6.5-x86_64-Compile.sh
把下面框中內容全部貼上
[root@localhost ~]# chmod +x  Hadoop-2.2.0_CentOS-6.5-x86_64-Compile.sh
 [root@localhost ~]# ./Hadoop-2.2.0_CentOS-6.5-x86_64-Compile.sh

注意:請勿把下面程式直接貼到命令提示 (例如:[root@localhost ~]#  ) 下直接執行,不然 if then ... else if 這類不是單一行命令的程式會執行錯誤。

程式內容如下


#!/bin/bash

echo -e "\033[31m"
echo -e "Program : Hadoop-2.2.0_CentOS-6.5-x86_64-Compile.sh "
echo -e "Hadoop 2.2.0 Compile Shell Script (CentOS 6.5 x86_64) "
echo -e "by Shau-Rong Lu 2014-02-07 "
echo -e "\033[0m"

cd /usr/local/src
#yum -y groupinstall  "Development tools"
#yum -y install gcc  gcc-c++  svn  cmake git zlib zlib-devel openssl openssl-devel rsync java-1.7.0-openjdk.x86_64 java-1.7.0-openjdk-devel.x86_64  make  wget
yum -y install gcc  gcc-c++  svn  cmake git zlib zlib-devel openssl openssl-devel rsync make  wget

# echo "********** Install OpenJDK **********"

#cd  /usr/local/src
#yum -y install java-1.7.0-openjdk.x86_64 java-1.7.0-openjdk-devel.x86_64

#echo 'export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51.x86_64' >> /etc/profile
#echo 'export PATH=$PATH:$JAVA_HOME/bin' >> /etc/profile
#echo 'export CLASSPATH=$JAVA_HOME/jre/lib/ext:$JAVA_HOME/lib/tools.jar' >> /etc/profile

#source /etc/profile

#java -version
#export | grep jdk

echo "********** Install Oracle JDK 7 Update 51 **********"

cd  /usr/local/src

# http://shaurong.blogspot.tw/2014/01/oracle-java-7-update-51-centos-65-x64.html

if [ ! -s jdk-7u51-linux-x64.rpm ]; then
  echo "Can not find /usr/local/jdk-7u51-linux-x64.rpm"
  wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F" "http://download.oracle.com/otn-pub/java/jdk/7u51-b13/jdk-7u51-linux-x64.rpm"  -O  jdk-7u51-linux-x64.rpm
  #exit
fi

rpm -ivh jdk-7u51-linux-x64.rpm
alternatives --install /usr/bin/java  java  /usr/java/jdk1.7.0_51/bin/java  100
alternatives --set  java /usr/java/jdk1.7.0_51/bin/java

echo
echo 'export JAVA_HOME=/usr/java/jdk1.7.0_51' >> /etc/profile
echo 'export PATH=$PATH:$JAVA_HOME/bin' >> /etc/profile
echo 'export CLASSPATH=$JAVA_HOME/jre/lib/ext:$JAVA_HOME/lib/tools.jar' >> /etc/profile

source /etc/profile

java -version
export | grep jdk


echo "********** Install Apache Maven 3.1.1 **********"
cd  /usr/local/src
if [ ! -s apache-maven-3.1.1-bin.tar.gz ]; then
  wget http://ftp.tc.edu.tw/pub/Apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz
fi
tar zxvf apache-maven-3.1.1-bin.tar.gz -C /usr/local
ln  -s  /usr/local/apache-maven-3.1.1/bin/mvn  /usr/bin/mvn

echo 'export MAVEN_HOME=/usr/local/apache-maven-3.1.1' >> /etc/profile
echo 'export PATH=$PATH:$MAVEN_HOME/bin' >> /etc/profile
source /etc/profile
#read -n 1 -p "Press Enter to continue..."

echo "********** Install FindBugs 2.0.3 **********"
cd  /usr/local/src
if [ ! -s findbugs-2.0.3.tar.gz ]; then
  wget http://prdownloads.sourceforge.net/findbugs/findbugs-2.0.3.tar.gz?download
fi
tar zxvf findbugs-2.0.3.tar.gz -C /usr/local/
ln -s /usr/local/findbugs-2.0.3/bin/findbugs  /usr/bin/findbugs
echo 'export FINDBUGS_HOME=/usr/local/findbugs-2.0.3' >> /etc/profile
echo 'export PATH=$PATH:$FINDBUGS_HOME/bin' >> /etc/profile
source /etc/profile
#read -n 1 -p "Press Enter to continue..."

echo "********** Install Protoc 2.5.0 **********"
# https://code.google.com/p/protobuf/
# https://code.google.com/p/protobuf/downloads/list

cd  /usr/local/src
if [ ! -s protobuf-2.5.0.tar.gz ]; then
  wget https://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz
fi
tar zxvf protobuf-2.5.0.tar.gz -C /usr/local/src
cd /usr/local/src/protobuf-2.5.0
./configure
make
make install
ln -s /usr/local/bin/protoc /usr/bin/protoc
echo 'export PROTO_HOME=/usr/local/' >> /etc/profile
echo 'export PATH=$PATH:$PROTO_HOME/bin' >> /etc/profile
source /etc/profile
#read -n 1 -p "Press Enter to continue..."

echo "********** Compile Hadoop 2.2.0 **********"

cd  /usr/local/src
if [ ! -s hadoop-2.2.0-src.tar.gz ]; then
  wget http://ftp.twaren.net/Unix/Web/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.tar.gz
  #wget http://ftp.mirror.tw/pub/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz
  #wget http://apache.stu.edu.tw/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz
fi
tar zxvf hadoop-2.2.0-src.tar.gz -C /usr/local/src
cd  /usr/local/src/hadoop-2.2.0-src/
wget https://issues.apache.org/jira/secure/attachment/12614482/HADOOP-10110.patch
#patch -p0 /usr/local/src/hadoop-2.2.0-src/pom.xml < HADOOP-10110.patch
patch -p0 < HADOOP-10110.patch
#mvn clean
mvn package -Pdist,native -DskipTests -Dtar
#read -n 1 -p "Press Enter to continue..."


執行結果

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18:53.003s
[INFO] Finished at: Fri Feb 07 20:51:49 CST 2014
[INFO] Final Memory: 92M/239M
[INFO] ------------------------------------------------------------------------
[root@localhost ~]# 

(完)


相關

[研究] hadoop-2.2.0-src.tar.gz 快速編譯安裝程式(二)(CentOS 6.5 x86_64)
http://shaurong.blogspot.com/2014/02/hadoop-220-srctargz-centos-65-x8664_8080.html

[研究] hadoop-2.2.0-src.tar.gz 快速編譯安裝程式(CentOS 6.5 x86_64)
http://shaurong.blogspot.com/2014/02/hadoop-220-srctargz-centos-65-x8664_7.html

[研究] hadoop-2.2.0-src.tar.gz 編譯研究(CentOS 6.5 x86_64)
http://shaurong.blogspot.com/2014/02/hadoop-220-srctargz-centos-65-x8664.html

[研究] Hadoop 2.2.0 編譯 (CentOS 6.4 x64)
http://shaurong.blogspot.tw/2013/11/hadoop-220-centos-64-x64.html

[研究] Hadoop 2.2.0 Single Cluster 安裝 (二)(CentOS 6.4 x64)
http://shaurong.blogspot.tw/2013/11/hadoop-220-single-cluster-centos-64-x64_7.html

[研究] Hadoop 2.2.0 Single Cluster 安裝 (一)(CentOS 6.4 x64)
http://shaurong.blogspot.tw/2013/11/hadoop-220-single-cluster-centos-64-x64.html

[研究] Hadoop 1.2.1 (rpm)安裝 (CentOS 6.4 x64)
http://shaurong.blogspot.tw/2013/10/hadoop-121-rpm-centos-64-x64.html

[研究] Hadoop 1.2.1 (bin)安裝 (CentOS 6.4 x64)
http://shaurong.blogspot.tw/2013/07/hadoop-112-centos-64-x64.html

[研究] Hadoop 1.2.1 安裝 (CentOS 6.4 x64)
http://forum.icst.org.tw/phpbb/viewtopic.php?t=80035

[研究] 雲端軟體 Hadoop 1.0.0 安裝 (CentOS 6.2 x86)
http://forum.icst.org.tw/phpbb/viewtopic.php?t=21166

[研究] 雲端軟體 Hadoop 0.20.2 安裝 (CentOS 5.5 x86)
http://forum.icst.org.tw/phpbb/viewtopic.php?t=18513

[研究] 雲端軟體 Hadoop 0.20.2 安裝 (CentOS 5.4 x86)
http://forum.icst.org.tw/phpbb/viewtopic.php?t=17974

1 則留言:

  1. 有個筆誤的地方在
    wget http://ftp.twaren.net/Unix/Web/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.tar.gz

    我修改如下
    wget http://ftp.twaren.net/Unix/Web/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz

    這樣就可以正常執行了,謝謝前輩的教學
    感激不盡

    回覆刪除