Ansible一键部署Kubernetes集群

一、环境准备

主机	ip地址	角色
k8s-master	192.168.252.141	master
k8s-node1	192.168.252.142	node
k8s-node2	192.168.252.143	node

二、实战

Ansible部署

主节点安装Ansible

yum -y install epel-release
yum -y install ansible
ansible --version

开启记录日志

vim /etc/ansible/ansible.cfg  
修改：
#log_path = /var/log/ansible.log ==> log_path = /var/log/ansible.log

去掉第一次连接ssh ask确认

vim /etc/ansible/ansible.cfg  
修改：
#host_key_checking = False  ==> host_key_checking = False

配置主机清单

vim /etc/ansible/hosts
内容：
[master]
192.168.252.141
[node]
192.168.252.142
192.168.252.143[k8s:children]
master
node[k8s:vars]
ansible_ssh_user=root
ansible_ssh_pass=syh2025659
ansible_ssh_port=22k8s_version=1.20.2

测试连通性

ansible k8s -m ping

编排 Ansible playbook

mkdir /root/k8s-install-workspace
cd /root/k8s-install-workspace
mkdir -p ./install-k8s/{package,init,install-docker,install-k8s,master-init,install-cni,install-ipvs,master-join,node-join,install-ingress-nginx,install-nfs-provisioner,install-harbor,install-metrics-server,uninstall-k8s}/{files,templates,vars,tasks,handlers,meta,default}tips:
package目中放的是提前导入的jar包 下面脚本需要直接提前导入这样子更快

我的网盘链接：

通过百度网盘分享的文件：k8s-install-workspace

链接：百度网盘请输入提取码

提取码：ruj1

提前将package目录中的包导入进去

节点初始化

cd /root/k8s-install-workspace
vim ./install-k8s/init/files/hosts
内容：
192.168.252.141 k8s-master
192.168.252.142 k8s-node1
192.168.252.143 k8s-node2

准备脚本

脚本思路：

修改主机名-->配置hosts解析-->配置秘钥互相通信（通过expect来发送指令）-->对时间进行同步-->关闭防火墙-->关闭swap分区-->关闭selinux-->iptables 检查桥接流量-->启用ipvs的所需模块

cd /root/k8s-install-workspace
vim ./install-k8s/init/templates/init.sh
内容：
#!/usr/bin/env bash### 【第一步】修改主机名
# 获取主机名
hostnamectl set-hostname $(grep `hostname -i` /tmp/hosts|awk '{print $2}')### 【第二步】配置hosts
# 先删除
for line in `cat /tmp/hosts`
dosed -i "/$line/d" /etc/hosts
done
# 追加
cat /tmp/hosts >> /etc/hosts### 【第三步】添加互信
# 先创建秘钥对
ssh-keygen -f ~/.ssh/id_rsa -P '' -q# 安装expect
yum -y install expect -y# 批量推送公钥
for line in `cat /tmp/hosts`
doip=`echo $line|awk '{print $1}'`
password={{ ansible_ssh_pass }}expect <<-EOFspawn ssh-copy-id -i /root/.ssh/id_rsa.pub $ip
expect {"(yes/no)?"{send "yes\n"expect "*assword:" { send "$password\n"}}"*assword:"{send "$password\n"}
}expect eof
EOF
done### 【第四步】时间同步
yum -y install ntpdate
ntpdate ntp.aliyun.com### 【第五步】关闭防火墙
systemctl stop firewalld
systemctl disable firewalld### 【第六步】关闭swap
# 临时关闭；关闭swap主要是为了性能考虑
swapoff -a
# 永久关闭
sed -ri 's/.*swap.*/#&/' /etc/fstab### 【第七步】禁用SELinux
# 临时关闭
setenforce 0
# 永久禁用
sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config### 【第八步】允许 iptables 检查桥接流量
sudo modprobe br_netfilter
lsmod | grep br_netfilter# 先删
rm -rf /etc/modules-load.d/k8s.confcat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOFsudo modprobe overlay
sudo modprobe br_netfilterrm -rf /etc/sysctl.d/k8s.conf# 设置所需的 sysctl 参数，参数在重新启动后保持不变
cat > /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
vm.swappiness = 0
EOF#加载模块
modprobe br_netfilter#加载⽹桥过滤及内核转发配置⽂件
sysctl -p /etc/sysctl.d/k8s.conf#配置ipvs功能
yum -y install ipset ipvsadmcat <<EOF > /etc/modules-load.d/ipvs.conf 
ip_vs
ip_vs_lc
ip_vs_wlc
ip_vs_rr
ip_vs_wrr
ip_vs_lblc
ip_vs_lblcr
ip_vs_dh
ip_vs_sh
ip_vs_nq
ip_vs_sed
ip_vs_ftp
ip_vs_sh
nf_conntrack_ipv4
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip
EOFcat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOFchmod 755 /etc/sysconfig/modules/ipvs.modules
bash /etc/sysconfig/modules/ipvs.modules# 应用 sysctl 参数而不重新启动
sudo sysctl --system

任务编排

将hosts发送给所有节点然后发送初始化节点的脚本然后进行执行

cd /root/k8s-install-workspace
vim ./install-k8s/init/tasks/main.yml
内容：
- name: cp hostscopy: src=hosts dest=/tmp/hosts
- name: init cptemplate: src=init.sh dest=/tmp/init.sh
- name: init installshell: sh /tmp/init.sh

安装docker

编写脚本

通过查询阿里云的安装docker-ce的方式进行安装docker然后配置加速器然后配置cgroup然后启动docker

cd /root/k8s-install-workspace
vim ./install-k8s/install-docker/files/install-docker.sh
内容：
#!/usr/bin/env bash# step 1: 安装必要的一些系统工具
yum install -y yum-utils device-mapper-persistent-data lvm2
# Step 2: 添加软件源信息
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# Step 3
sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
# Step 4: 更新并安装Docker-CE
yum makecache fast
yum -y install docker-ce
# Step 4: 开启Docker服务
systemctl start docker
systemctl enable docker
# 修改文件 /etc/docker/daemon.json，没有这个文件就创建
# 添加以下内容后，重启docker服务：
cat >/etc/docker/daemon.json<<EOF
{"registry-mirrors": ["https://hub.uuuadc.top","https://docker.anyhub.us.kg","https://dockerhub.jobcher.com","https://dockerhub.icu","https://docker.ckyl.me","https://docker.awsl9527.cn"],"exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
# 重启
systemctl restart docker# 查看
systemctl status docker containerd

任务编排

将脚本发送至所有节点然后执行脚本

cd /root/k8s-install-workspace
vim ./install-k8s/install-docker/tasks/main.yml
内容：
- name: install docker cpcopy: src=install-docker.sh dest=/tmp/install-docker.sh
- name: install dockershell: sh /tmp/install-docker.sh

安装k8s并提前将镜像导入

检查k8s是否安装安装好的话直接下一步，阿里云上找repo源然后进行安装，可以更改版本下面的版本，然后将准备好的镜像tar包进行导入就可以了。

cd /root/k8s-install-workspace
vim ./install-k8s/install-k8s/templates/install-k8s.sh
内容：
#!/usr/bin/env bash# 检查是否已经安装
yum list installed kubelet
if [ $? -eq 0 ];thenexit 0
ficat > /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOFyum install -y kubelet-1.20.2-0.x86_64 kubeadm-1.20.2-0.x86_64 kubectl-1.20.2-0.x86_64DOCKER_CGROUPS=`docker info |grep 'Cgroup' | awk ' NR==1 {print $3}'`cat > /etc/sysconfig/kubelet<<EOF
KUBELET_EXTRA_ARGS="--cgroup-driver=$DOCKER_CGROUPS --pod-infra-container-image=k8s.gcr.io/pause:3.2"
EOFsystemctl daemon-reload
systemctl enable kubelet && systemctl restart kubelet
systemctl is-active kubelet#导入镜像
docker load < /tmp/kube-apiserver.tar
docker load < /tmp/kube-proxy.tar
docker load < /tmp/kube-controller-manager.tar
docker load < /tmp/kube-scheduler.tar
docker load < /tmp/etcd.tar
docker load < /tmp/pause.tar
docker load < /tmp/coredns.tar
docker load < /tmp/calico-cni.tar
docker load < /tmp/calico-kube-controller.tar
docker load < /tmp/calico-node.tar
docker load < /tmp/calico-pod.tar

任务编排

将tar包和执行脚本发送到各个节点的临时目录然后执行脚本

cd /root/k8s-install-workspace
vim ./install-k8s/install-k8s/tasks/main.yml
内容：
- name: copy tarISOcopy: src=/root/k8s-install-workspace/install-k8s/package/ dest=/tmp/        #这个我是把提前准备好的包放到package里面了
- name: install k8s cp  template: src=install-k8s.sh dest=/tmp/install-k8s.sh
- name: install k8sshell: sh /tmp/install-k8s.sh

初始化k8s-master并应用网络插件

编写脚本

在主节点进行初始化然后将目录创建好然后从网上拉取calico的yml文件然后进行应用

cd /root/k8s-install-workspace
vim ./install-k8s/master-init/templates/master-init.sh
内容：
#!/usr/bin/env bash# 判断是否已经初始化了
kubectl get nodes |grep -q `hostname` 1>&2 >/dev/null
if [ $? -eq 0 ];thenexit 0
fiip=`hostname -i`kubeadm init --kubernetes-version=v1.20.2 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=$iprm -rf $HOME/.kube
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/configcurl https://docs.projectcalico.org/v3.20/manifests/calico.yaml -Okubectl apply -f calico.yaml

任务编排

将脚本发送到master节点进行执行即可

cd /root/k8s-install-workspace
vim ./install-k8s/master-init/tasks/main.yml
内容：
- name: k8s master init cptemplate: src=master-init.sh dest=/tmp/master-init.sh
- name: k8s master initshell: sh /tmp/master-init.sh

node节点加入k8s集群

通过ssh命令获取主节点初始化后的node加入集群节点的命令在node节点上执行进行node节点加入集群

cd /root/k8s-install-workspace
vim ./install-k8s/node-join/files/node-join.sh
内容：
#!/usr/bin/env bash# 获取master ip，假设都是第一个节点为master
maser_ip=`head -1 /tmp/hosts |awk '{print $1}'`# 判断节点是否加入
ssh $maser_ip "kubectl get nodes|grep -q `hostname`"
if [ $? -eq 0 ];thenexit 0
fi`ssh $maser_ip kubeadm token create --print-join-command`

任务编排

将脚本发送到所有node节点

cd /root/k8s-install-workspace
vim ./install-k8s/node-join/tasks/main.yml
内容：
- name: node join cpcopy: src=node-join.sh dest=/tmp/node-join.sh
- name: node joinshell: sh /tmp/node-join.sh

k8s 环境安装编排 roles

进行角色编排

cd /root/k8s-install-workspace
vim ./install-k8s/install-k8s.yaml
内容：
- hosts: k8sremote_user: rootroles:- init
- hosts: k8sremote_user: rootroles:- install-docker
- hosts: k8sremote_user: rootroles:- install-k8s
- hosts: masterremote_user: rootroles:- master-init
- hosts: noderemote_user: rootroles:- node-join

启用剧本

cd /root/k8s-install-workspace/install-k8s
ansible-playbook  install-k8s.yaml