1：规划
oracle linux6.4上搭建hadoop1.2.0环境
192.168.100.171 linux1 (namenode)
192.168.100.172 linux2 (secondarynamenode)
192.168.100.173 linux3 (datanode)
192.168.100.174 linux4 (datanode)
192.168.100.175 linux5 (预留备用datanode)
2：创建VMware Workstation样板机
a：安装oracle linux 6.4虚拟机linux1，开通ssh服务,屏蔽iptables服务
[root@linux1 ~]# chkconfig sshd on
[root@linux1 ~]# chkconfig iptables off
[root@linux1 ~]# chkconfig ip6tables off
[root@linux1 ~]# chkconfig postfix off
b：关闭虚拟机linux1，增加一个新的硬盘到共享目录作为共享硬盘用（使用SCSI1:0接口)，
修改linux1.vmx，添加和修改参数:
disk.locking="FALSE"
diskLib.dataCacheMaxSize = "0"
disk.EnableUUID = "TRUE"
scsi1.present = "TRUE"
scsi1.sharedBus = "Virtual"
scsi1.virtualDev = "lsilogic"
c：重启虚拟机linux1，下载JAVA到共享硬盘，安装JAVA，在环境变量配置文件/etc/profile末尾增加：
JAVA_HOME=/usr/java/jdk1.7.0_21; export JAVA_HOME
JRE_HOME=/usr/java/jdk1.7.0_21/jre; export JRE_HOME
CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar; export CLASSPATH
PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH; export PATH
d：修改/etc/hosts，增加：
192.168.100.171 linux1
192.168.100.172 linux2
192.168.100.173 linux3
192.168.100.174 linux4
192.168.100.175 linux5
e：修改/etc/sysconfig/selinux
SELINUX=disabled
f：增加hadoop用户及安装hadoop文件：
[root@linux1 ~]# useradd hadoop -g root
[root@linux1 ~]# passwd hadoop
[root@linux1 ~]# cd /
[root@linux1 /]# mkdir /app
[root@linux1 /]# cd /app
[root@linux1 app]# tar -zxf /mnt/mysoft/LinuxSoft/hadoop-1.2.0.tar.gz
[root@linux1 app]# mv hadoop-1.2.0 hadoop120
[root@linux1 app]# chown hadoop:root -R /app/hadoop120
[root@linux1 hadoop120]# su - hadoop
[hadoop@linux1 ~]$ cd /app/hadoop120
[hadoop@linux1 hadoop120]$ mkdir tmp
[hadoop@linux1 hadoop120]$ chmod 777 tmp
g：修改hadoop相关配置文件：
[hadoop@linux1 hadoop120]$ cd conf
[hadoop@linux1 conf]$ vi core-site.xml
******************************************************************************
<configuration>
<name>hadoop.tmp.dir</name>
<value>/app/hadoop120/tmp</value>
</property>
<!-- file system properties -->
<property>
<name>fs.default.name</name>
<value>hdfs://linux1:9000</value>
</property>
</configuration>
******************************************************************************
[hadoop@linux1 conf]$ vi hdfs-site.xml
******************************************************************************
<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
</configuration>
******************************************************************************
[hadoop@linux1 conf]$ vi mapred-site.xml
******************************************************************************
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>linux1:9001</value>
</property>
</configuration>
******************************************************************************
[hadoop@linux1 conf]$ vi hadoop-env.sh
******************************************************************************
export JAVA_HOME=/usr/java/jdk1.7.0_21
******************************************************************************
h：配置ssh使用证书验证/etc/ssh/sshd_config，打开注释：
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
3：配置ssh
a：关闭样板机，分别复制成linux2、linux3、linux4、linux5：
修改vmware workstation配置文件的displayname；
修改虚拟机的下列文件中相关的信息
/etc/udev/rules.d/70-persistent-net.rules
/etc/sysconfig/network
/etc/sysconfig/network-scripts/ifcfg-eth0
b：启动linux1、linux2、linux3、linux4、linux5，确保相互之间能ping通。
c：配置ssh，确保linux1能无验证访问其他节点
[root@linux1 tmp]# su - hadoop
[hadoop@linux1 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa):
Created directory '/home/hadoop/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
17:37:98:fa:7e:5c:e4:8b:b4:7e:bb:59:28:8f:45:bd hadoop@linux1
The key's randomart image is:
+--[ RSA 2048]----+
| |
| o |
| + o |
| . o ... |
| S . o. .|
| o ..o..|
| .o.+oE.|
| . ==oo |
| .oo.=o |
+-----------------+
[hadoop@linux1 ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@linux1
[hadoop@linux1 ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@linux2
[hadoop@linux1 ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@linux3
[hadoop@linux1 ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@linux4
[hadoop@linux1 ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@linux5
验证可否无密码访问：
[hadoop@linux1 ~]$ ssh linux1 date
[hadoop@linux1 ~]$ ssh linux2 date
[hadoop@linux1 ~]$ ssh linux3 date
[hadoop@linux1 ~]$ ssh linux4 date
[hadoop@linux1 ~]$ ssh linux5 date
4：修改linux的masters和salves配置，然后初始化hadoop
[hadoop@linux1 conf]$ vi masters
192.168.100.171
[hadoop@linux1 conf]$ vi slaves
192.168.100.173
192.168.100.174
[hadoop@linux1 conf]$ cd ..
[hadoop@linux1 hadoop120]$ bin/hadoop namenode -format
5：启动hadoop
[hadoop@linux1 hadoop120]$ bin/start-all.sh
starting namenode, logging to /app/hadoop120/libexec/../logs/hadoop-hadoop-namenode-linux1.out
192.168.100.174: starting datanode, logging to /app/hadoop120/libexec/../logs/hadoop-hadoop-datanode-linux4.out
192.168.100.173: starting datanode, logging to /app/hadoop120/libexec/../logs/hadoop-hadoop-datanode-linux3.out
192.168.100.172: starting secondarynamenode, logging to /app/hadoop120/libexec/../logs/hadoop-hadoop-secondarynamenode-linux2.out
starting jobtracker, logging to /app/hadoop120/libexec/../logs/hadoop-hadoop-jobtracker-linux1.out
192.168.100.174: starting tasktracker, logging to /app/hadoop120/libexec/../logs/hadoop-hadoop-tasktracker-linux4.out
192.168.100.173: starting tasktracker, logging to /app/hadoop120/libexec/../logs/hadoop-hadoop-tasktracker-linux3.out
6：修改linux启动到console状态
评论