Hadoop安装hive&配置mysql元数据库

1. 安装Hive

环境：Dokcer搭建的Hadoop完全分布式集群 - namenode节点
安装包：apache-hive1.2.1-bin.tar.gz

从本地上传安装包到namenode主机并解压

scp /home/daisine/software/apache-hive-1.2.1-bin.tar.gz root@172.18.0.2:/usr/local/
# 解压到/usr/local中
sudo tar -zxvf ./apache-hive-1.2.1-bin.tar.gz -C /usr/local   
cd /usr/local/
# 将文件夹名改为hive
sudo mv apache-hive-1.2.1-bin hive

配置hive-env.sh文件

1 2	export HADOOP_HOME=/usr/local/hadoop export HIVE_CONF_DIR=/usr/local/hive/conf

2. 安装MySQL

MySQL安装过程

#添加mysql yum源
rpm -Uvh http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
#查看当前可用的MySQL安装资源
yum repolist enabled | grep "mysql.*-community.*"
#使用yum的方式安装MySQL
yum -y install mysql-community-server

安装MySQL5.6.44：

#查看mysql安装结果
rpm -qa|grep mysql
mysql-community-release-el7-5.noarch
mysql-community-client-5.6.44-2.el7.x86_64
mysql-community-common-5.6.44-2.el7.x86_64
mysql-community-libs-5.6.44-2.el7.x86_64
mysql-community-server-5.6.44-2.el7.x86_64

配置MySQL

#开机启动
systemctl enable mysqld
#启动mysql
systemctl start mysqld
#查看mysql当前状态
systemctl status mysqld

#重置密码
mysql_secure_installation
Set root password? [Y/n] y        [设置root用户密码]
Remove anonymous users? [Y/n] y            [删除匿名用户]
Disallow root login remotely? [Y/n] n            [禁止root远程登录]
Remove test database and access to it? [Y/n] y       [删除test数据库]
Reload privilege tables now? [Y/n] y        [刷新权限]

#修改访问权限 让其他计算机也能访问
mysql -u root -p
mysql > GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'yourpassword' WITH GRANT OPTION;
mysql > flush privileges;

mysql > show databases;
mysql > use mysql;
mysql > show tables;
mysql > desc user;

#停止mysql
systemctl stop mysqld

注意：不建议安装MySQL 5.7版本或更高，后期配置需要解决很多新问题。

使用yum安装的MySQL 5.7版本完全卸载方式方案 不卸载干净再安装MySQL5.6 启动会出问题

#停止mysql
systemctl stop mysqld
#卸载MySQL5.7
yum remove  mysql mysql-server mysql-libs mysql-server
cd /var/lib/
rm -rf mysql 

cd /usr/share/
rm -rf mysql/

#查询MySQL安装包
rpm -qa |grep mysql
#将查询到的结果删去
yum remove $(查询到的结果)

3. Hive配置MySQL为元数据库

mysql jdbc 包：mysql-connector-java-5.1.47.tar.gz(进入网页后选择Platform Independent)

scp /home/daisine/software/mysql-connector-java-5.1.47.tar.gz root@172.18.0.2:/usr/local/
tar -zxvf mysql-connector-java-5.1.47.tar.gz
cd mysql-connector-java-5.1.47
cp mysql-connector-java-5.1.47-bin.jar /usr/local/hive/lib/

配置Metastore到MySQL

1
2
3

cd /usr/local/hive/conf
touch hive-site.xml
vi hive-site.xml

hive-site.xml 需要根据自己的情况修改两处

ConnectionURL：jdbc:mysql://$(hostname):3306/metastore
ConnectionPassword：<value>$(password)</value>

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
	<property>
	  <name>javax.jdo.option.ConnectionURL</name>
	  <value>jdbc:mysql://hadoop1:3306/metastore?createDatabaseIfNotExist=true&amp;useSSL=false</value>
	  <description>JDBC connect string for a JDBC metastore</description>
	</property>

	<property>
	  <name>javax.jdo.option.ConnectionDriverName</name>
	  <value>com.mysql.jdbc.Driver</value>
	  <description>Driver class name for a JDBC metastore</description>
	</property>

	<property>
	  <name>javax.jdo.option.ConnectionUserName</name>
	  <value>root</value>
	  <description>username to use against metastore database</description>
	</property>

	<property>
	  <name>javax.jdo.option.ConnectionPassword</name>
	  <value>111111</value>
	  <description>password to use against metastore database</description>
	</property>
</configuration>

4. 启动hive

# 启动hdfs yarn
# namenode
sbin/start-dfs.sh
# resourcemanager
sbin/start-yarn.sh
# 启动hive
cd /usr/local/hive
bin/hive
Logging initialized using configuration in jar:file:/usr/local/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
hive >