1. 离线部署Kubernetes集群全流程解析在无法连接互联网的生产环境中部署Kubernetes集群是许多企业IT团队必须面对的挑战。作为一名经历过数十次离线部署的运维工程师我将分享一套经过实战检验的完整方案。这个方案不仅能帮你避开90%的常见坑点还能确保集群具备生产级稳定性。2. 环境规划与基础配置2.1 最小化集群规划建议对于测试环境我推荐以下配置方案节点角色数量推荐配置磁盘要求操作系统Master12核4GB内存40GBCentOS 7.9Worker22核4GB内存40GB同Master节点实际生产环境中建议Master节点采用3节点高可用部署配置至少4核8GB内存。Worker节点数量根据业务负载动态调整。2.2 必须执行的基础配置所有节点都需要完成以下配置这是Kubernetes稳定运行的前提条件2.2.1 安全策略调整# 关闭防火墙生产环境建议配置精确规则 systemctl stop firewalld systemctl disable firewalld # 永久关闭SELinux sed -i s/SELINUXenforcing/SELINUXdisabled/g /etc/selinux/config setenforce 02.2.2 系统参数优化# 关闭交换分区 swapoff -a sed -i / swap / s/^\(.*\)$/#\1/g /etc/fstab # 加载内核模块 cat /etc/modules-load.d/k8s.conf EOF br_netfilter overlay EOF modprobe br_netfilter modprobe overlay # 配置内核参数 cat /etc/sysctl.d/k8s.conf EOF net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 net.ipv4.ip_forward 1 EOF sysctl --system2.2.3 主机与时间配置# 各节点分别设置主机名 hostnamectl set-hostname master # 在master节点执行 hostnamectl set-hostname node1 # 在node1节点执行 # 所有节点配置hosts解析 cat /etc/hosts EOF 192.168.1.100 master 192.168.1.101 node1 EOF # 配置NTP时间同步 yum install -y chrony systemctl enable chronyd systemctl start chronyd chronyc sources3. 离线资源准备与部署3.1 离线资源打包方案在有网络的环境中准备离线资源包# 创建资源目录 mkdir -p /opt/k8s-offline/{packages,images,yum-repo} # 下载kubeadm相关工具 cat /opt/k8s-offline/packages/download.sh EOF #!/bin/bash K8S_VERSION1.24.4 CRI_VERSION1.6.8 # 下载kubeadm、kubelet、kubectl wget https://storage.googleapis.com/kubernetes-release/release/v${K8S_VERSION}/bin/linux/amd64/{kubeadm,kubelet,kubectl} -P /opt/k8s-offline/packages/ # 下载containerd wget https://github.com/containerd/containerd/releases/download/v${CRI_VERSION}/containerd-${CRI_VERSION}-linux-amd64.tar.gz -P /opt/k8s-offline/packages/ # 下载runc wget https://github.com/opencontainers/runc/releases/download/v1.1.3/runc.amd64 -P /opt/k8s-offline/packages/ # 下载CNI插件 wget https://github.com/containernetworking/plugins/releases/download/v1.1.1/cni-plugins-linux-amd64-v1.1.1.tgz -P /opt/k8s-offline/packages/ EOF # 执行下载脚本 chmod x /opt/k8s-offline/packages/download.sh /opt/k8s-offline/packages/download.sh3.2 镜像导出与导入技巧# 在有网络环境拉取镜像 kubeadm config images list --kubernetes-versionv1.24.4 /opt/k8s-offline/images/image-list.txt # 使用阿里云镜像加速 for image in $(cat /opt/k8s-offline/images/image-list.txt); do docker pull registry.aliyuncs.com/google_containers/${image#*/} docker tag registry.aliyuncs.com/google_containers/${image#*/} $image docker save -o /opt/k8s-offline/images/${image#*/}.tar $image done # 打包离线资源在所有节点执行 tar -czvf k8s-offline-v1.24.4.tar.gz -C /opt k8s-offline3.3 节点软件安装流程将离线包拷贝到所有节点后执行# 解压资源包 tar -xzvf k8s-offline-v1.24.4.tar.gz -C /opt # 安装containerd tar -xzvf /opt/k8s-offline/packages/containerd-*.tar.gz -C /usr/local mkdir -p /usr/local/lib/systemd/system cat /usr/local/lib/systemd/system/containerd.service EOF [Unit] Descriptioncontainerd container runtime Documentationhttps://containerd.io Afternetwork.target local-fs.target [Service] ExecStartPre-/sbin/modprobe overlay ExecStart/usr/local/bin/containerd Typenotify Delegateyes KillModeprocess Restartalways RestartSec5 LimitNPROCinfinity LimitCOREinfinity LimitNOFILEinfinity [Install] WantedBymulti-user.target EOF # 安装runc install -m 755 /opt/k8s-offline/packages/runc.amd64 /usr/local/sbin/runc # 安装CNI插件 mkdir -p /opt/cni/bin tar -xzvf /opt/k8s-offline/packages/cni-plugins-*.tgz -C /opt/cni/bin # 配置containerd mkdir -p /etc/containerd containerd config default /etc/containerd/config.toml sed -i s/SystemdCgroup false/SystemdCgroup true/g /etc/containerd/config.toml # 启动containerd systemctl daemon-reload systemctl enable --now containerd4. 集群初始化与网络配置4.1 Master节点初始化关键步骤# 安装kubeadm工具 cp /opt/k8s-offline/packages/{kubeadm,kubelet,kubectl} /usr/local/bin/ chmod x /usr/local/bin/{kubeadm,kubelet,kubectl} # 配置kubelet服务 cat /etc/systemd/system/kubelet.service EOF [Unit] Descriptionkubelet: The Kubernetes Node Agent Documentationhttps://kubernetes.io/docs/home/ [Service] ExecStart/usr/local/bin/kubelet Restartalways StartLimitInterval0 RestartSec10 [Install] WantedBymulti-user.target EOF # 初始化集群注意替换apiserver-advertise-address kubeadm init \ --apiserver-advertise-address192.168.1.100 \ --image-repositoryregistry.aliyuncs.com/google_containers \ --kubernetes-versionv1.24.4 \ --service-cidr10.96.0.0/12 \ --pod-network-cidr10.244.0.0/16 \ --ignore-preflight-errorsSwap # 配置kubectl mkdir -p $HOME/.kube cp -i /etc/kubernetes/admin.conf $HOME/.kube/config chown $(id -u):$(id -g) $HOME/.kube/config4.2 Flannel网络插件部署# 导入flannel镜像 docker load -i /opt/k8s-offline/images/flannel.tar # 应用flannel配置 kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml # 检查部署状态 kubectl get pods -n kube-system -l appflannel5. Worker节点加入集群在Master节点获取加入命令kubeadm token create --print-join-command在Worker节点执行# 安装kubelet工具 cp /opt/k8s-offline/packages/{kubeadm,kubelet,kubectl} /usr/local/bin/ chmod x /usr/local/bin/{kubeadm,kubelet,kubectl} # 执行加入命令示例实际使用master生成的命令 kubeadm join 192.168.1.100:6443 \ --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:xxxxxxxxxx6. 集群功能验证测试6.1 基础状态检查# 节点状态检查 kubectl get nodes -o wide # 核心组件状态 kubectl get pods -n kube-system # 查看集群信息 kubectl cluster-info6.2 应用部署测试# 部署nginx应用 kubectl create deployment nginx --imagenginx:1.23 # 暴露服务 kubectl expose deployment nginx --port80 --typeNodePort # 获取访问地址 kubectl get svc nginx6.3 高级功能验证# Pod扩缩容测试 kubectl scale deployment nginx --replicas3 kubectl get pods -o wide # 持久化存储测试 cat EOF | kubectl apply -f - apiVersion: v1 kind: PersistentVolume metadata: name: pv001 spec: capacity: storage: 1Gi accessModes: - ReadWriteOnce hostPath: path: /data/pv001 EOF7. 常见问题排查指南7.1 节点NotReady问题可能原因及解决方案网络插件未正确安装 → 检查flannel pod状态containerd未正常运行 →systemctl status containerd防火墙未关闭 → 确认firewalld已禁用7.2 Pod一直处于Pending状态排查步骤# 查看事件详情 kubectl describe pod pod-name # 常见原因 # - 资源不足 → kubectl describe nodes # - 镜像拉取失败 → kubectl get events7.3 服务无法访问检查清单确认Service的NodePort已分配 →kubectl get svc检查节点防火墙规则 →iptables -L -n测试Pod间通信 →kubectl exec -it pod -- curl service-ip8. 生产环境优化建议日志收集部署EFK(ElasticsearchFluentdKibana)日志系统监控告警集成PrometheusGrafana监控体系存储方案根据业务需求配置Ceph或NFS持久化存储安全加固启用RBAC、网络策略(PodSecurityPolicy)备份方案定期备份etcd集群数据在实施过程中建议先在小规模测试环境验证整套方案确认无误后再推广到生产环境。每次升级Kubernetes版本时都需要重新准备对应版本的离线资源包。 
长河君 · 2026/11/3 17:32:13 · 阅读 1.2k Kubernetes离线部署实战：生产环境全流程指南
1. 离线部署Kubernetes集群全流程解析在无法连接互联网的生产环境中部署Kubernetes集群是许多企业IT团队必须面对的挑战。作为一名经历过数十次离线部署的运维工程师我将分享一套经过实战检验的完整方案。这个方案不仅能帮你避开90%的常见坑点还能确保集群具备生产级稳定性。2. 环境规划与基础配置2.1 最小化集群规划建议对于测试环境我推荐以下配置方案节点角色数量推荐配置磁盘要求操作系统Master12核4GB内存40GBCentOS 7.9Worker22核4GB内存40GB同Master节点实际生产环境中建议Master节点采用3节点高可用部署配置至少4核8GB内存。Worker节点数量根据业务负载动态调整。2.2 必须执行的基础配置所有节点都需要完成以下配置这是Kubernetes稳定运行的前提条件2.2.1 安全策略调整# 关闭防火墙生产环境建议配置精确规则 systemctl stop firewalld systemctl disable firewalld # 永久关闭SELinux sed -i s/SELINUXenforcing/SELINUXdisabled/g /etc/selinux/config setenforce 02.2.2 系统参数优化# 关闭交换分区 swapoff -a sed -i / swap / s/^\(.*\)$/#\1/g /etc/fstab # 加载内核模块 cat /etc/modules-load.d/k8s.conf EOF br_netfilter overlay EOF modprobe br_netfilter modprobe overlay # 配置内核参数 cat /etc/sysctl.d/k8s.conf EOF net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 net.ipv4.ip_forward 1 EOF sysctl --system2.2.3 主机与时间配置# 各节点分别设置主机名 hostnamectl set-hostname master # 在master节点执行 hostnamectl set-hostname node1 # 在node1节点执行 # 所有节点配置hosts解析 cat /etc/hosts EOF 192.168.1.100 master 192.168.1.101 node1 EOF # 配置NTP时间同步 yum install -y chrony systemctl enable chronyd systemctl start chronyd chronyc sources3. 离线资源准备与部署3.1 离线资源打包方案在有网络的环境中准备离线资源包# 创建资源目录 mkdir -p /opt/k8s-offline/{packages,images,yum-repo} # 下载kubeadm相关工具 cat /opt/k8s-offline/packages/download.sh EOF #!/bin/bash K8S_VERSION1.24.4 CRI_VERSION1.6.8 # 下载kubeadm、kubelet、kubectl wget https://storage.googleapis.com/kubernetes-release/release/v${K8S_VERSION}/bin/linux/amd64/{kubeadm,kubelet,kubectl} -P /opt/k8s-offline/packages/ # 下载containerd wget https://github.com/containerd/containerd/releases/download/v${CRI_VERSION}/containerd-${CRI_VERSION}-linux-amd64.tar.gz -P /opt/k8s-offline/packages/ # 下载runc wget https://github.com/opencontainers/runc/releases/download/v1.1.3/runc.amd64 -P /opt/k8s-offline/packages/ # 下载CNI插件 wget https://github.com/containernetworking/plugins/releases/download/v1.1.1/cni-plugins-linux-amd64-v1.1.1.tgz -P /opt/k8s-offline/packages/ EOF # 执行下载脚本 chmod x /opt/k8s-offline/packages/download.sh /opt/k8s-offline/packages/download.sh3.2 镜像导出与导入技巧# 在有网络环境拉取镜像 kubeadm config images list --kubernetes-versionv1.24.4 /opt/k8s-offline/images/image-list.txt # 使用阿里云镜像加速 for image in $(cat /opt/k8s-offline/images/image-list.txt); do docker pull registry.aliyuncs.com/google_containers/${image#*/} docker tag registry.aliyuncs.com/google_containers/${image#*/} $image docker save -o /opt/k8s-offline/images/${image#*/}.tar $image done # 打包离线资源在所有节点执行 tar -czvf k8s-offline-v1.24.4.tar.gz -C /opt k8s-offline3.3 节点软件安装流程将离线包拷贝到所有节点后执行# 解压资源包 tar -xzvf k8s-offline-v1.24.4.tar.gz -C /opt # 安装containerd tar -xzvf /opt/k8s-offline/packages/containerd-*.tar.gz -C /usr/local mkdir -p /usr/local/lib/systemd/system cat /usr/local/lib/systemd/system/containerd.service EOF [Unit] Descriptioncontainerd container runtime Documentationhttps://containerd.io Afternetwork.target local-fs.target [Service] ExecStartPre-/sbin/modprobe overlay ExecStart/usr/local/bin/containerd Typenotify Delegateyes KillModeprocess Restartalways RestartSec5 LimitNPROCinfinity LimitCOREinfinity LimitNOFILEinfinity [Install] WantedBymulti-user.target EOF # 安装runc install -m 755 /opt/k8s-offline/packages/runc.amd64 /usr/local/sbin/runc # 安装CNI插件 mkdir -p /opt/cni/bin tar -xzvf /opt/k8s-offline/packages/cni-plugins-*.tgz -C /opt/cni/bin # 配置containerd mkdir -p /etc/containerd containerd config default /etc/containerd/config.toml sed -i s/SystemdCgroup false/SystemdCgroup true/g /etc/containerd/config.toml # 启动containerd systemctl daemon-reload systemctl enable --now containerd4. 集群初始化与网络配置4.1 Master节点初始化关键步骤# 安装kubeadm工具 cp /opt/k8s-offline/packages/{kubeadm,kubelet,kubectl} /usr/local/bin/ chmod x /usr/local/bin/{kubeadm,kubelet,kubectl} # 配置kubelet服务 cat /etc/systemd/system/kubelet.service EOF [Unit] Descriptionkubelet: The Kubernetes Node Agent Documentationhttps://kubernetes.io/docs/home/ [Service] ExecStart/usr/local/bin/kubelet Restartalways StartLimitInterval0 RestartSec10 [Install] WantedBymulti-user.target EOF # 初始化集群注意替换apiserver-advertise-address kubeadm init \ --apiserver-advertise-address192.168.1.100 \ --image-repositoryregistry.aliyuncs.com/google_containers \ --kubernetes-versionv1.24.4 \ --service-cidr10.96.0.0/12 \ --pod-network-cidr10.244.0.0/16 \ --ignore-preflight-errorsSwap # 配置kubectl mkdir -p $HOME/.kube cp -i /etc/kubernetes/admin.conf $HOME/.kube/config chown $(id -u):$(id -g) $HOME/.kube/config4.2 Flannel网络插件部署# 导入flannel镜像 docker load -i /opt/k8s-offline/images/flannel.tar # 应用flannel配置 kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml # 检查部署状态 kubectl get pods -n kube-system -l appflannel5. Worker节点加入集群在Master节点获取加入命令kubeadm token create --print-join-command在Worker节点执行# 安装kubelet工具 cp /opt/k8s-offline/packages/{kubeadm,kubelet,kubectl} /usr/local/bin/ chmod x /usr/local/bin/{kubeadm,kubelet,kubectl} # 执行加入命令示例实际使用master生成的命令 kubeadm join 192.168.1.100:6443 \ --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:xxxxxxxxxx6. 集群功能验证测试6.1 基础状态检查# 节点状态检查 kubectl get nodes -o wide # 核心组件状态 kubectl get pods -n kube-system # 查看集群信息 kubectl cluster-info6.2 应用部署测试# 部署nginx应用 kubectl create deployment nginx --imagenginx:1.23 # 暴露服务 kubectl expose deployment nginx --port80 --typeNodePort # 获取访问地址 kubectl get svc nginx6.3 高级功能验证# Pod扩缩容测试 kubectl scale deployment nginx --replicas3 kubectl get pods -o wide # 持久化存储测试 cat EOF | kubectl apply -f - apiVersion: v1 kind: PersistentVolume metadata: name: pv001 spec: capacity: storage: 1Gi accessModes: - ReadWriteOnce hostPath: path: /data/pv001 EOF7. 常见问题排查指南7.1 节点NotReady问题可能原因及解决方案网络插件未正确安装 → 检查flannel pod状态containerd未正常运行 →systemctl status containerd防火墙未关闭 → 确认firewalld已禁用7.2 Pod一直处于Pending状态排查步骤# 查看事件详情 kubectl describe pod pod-name # 常见原因 # - 资源不足 → kubectl describe nodes # - 镜像拉取失败 → kubectl get events7.3 服务无法访问检查清单确认Service的NodePort已分配 →kubectl get svc检查节点防火墙规则 →iptables -L -n测试Pod间通信 →kubectl exec -it pod -- curl service-ip8. 生产环境优化建议日志收集部署EFK(ElasticsearchFluentdKibana)日志系统监控告警集成PrometheusGrafana监控体系存储方案根据业务需求配置Ceph或NFS持久化存储安全加固启用RBAC、网络策略(PodSecurityPolicy)备份方案定期备份etcd集群数据在实施过程中建议先在小规模测试环境验证整套方案确认无误后再推广到生产环境。每次升级Kubernetes版本时都需要重新准备对应版本的离线资源包。