centos7安装k8s 1.24.3版本 Error getting node“ err=“node “master01“ not found

简介

kubernetes 1.24.0以上版本已经移除了docker cri,因此在使用的docker来的安装k8s时，你需要自己安装cri-docker

名词解释

• cri:容器运行时，这个东东是用来在pod中控制容器的

服务器最低配置要求

• cpu:2核心
• 内存：2G

服务器上设置

• 关闭swap
• 关闭firewalld
• 禁用selinux
• 启用br_netfilter模块
• 6443端口

必要服务

• docker 每个节点必须
• iptables 每个节点必须
• cri-docker 每个节点必须：注意：启动服务时需要指定–pod-infra-container-image选项，否则可能导致初始化失败
• kubelet 每个节点必须
• kubeadm 每个节点必须
• kubectl 按需安装，用来的与集群交互

服务器初始化

以下为ansible的剧本，cri-docker.service设置部分没写，自己搞搞

---
- hosts: localhostremote_user: roottasks:- name: 关闭firewalld并且取消开机启动systemd:enabled: FALSEstate: stoppedname: firewalld.service- name: 永久关闭selinuxlineinfile:dest: /etc/selinux/configregexp: "^SELINUX="line: "SELINUX=disabled"- name: 临时关闭selinuxshell: "setenforce 0"failed_when: FALSE- name: 关闭swapshell: "swapoff -a && sed -i 's/^[^#]*swap/#&/g' /etc/fstab"- name: 安装yum-utilsyum: name=yum-utils state=present- name: 添加docker-ce repo文件shell: yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo- name: 安装dockershell: yum install docker-ce -y- name: 创建/root/cri目录file:state: directorypath: /root/cri- name: 拷贝cri-docker rpm包copy:src: /root/cri/cri-dockerd-0.2.5-3.el7.x86_64.rpmdest: /root/cri/cri-dockerd-0.2.5-3.el7.x86_64.rpm- name: 安装cri-dockershell: rpm -ivh /root/cri/cri-dockerd-0.2.5-3.el7.x86_64.rpm - name: 创建k8s.config文件shell:cmd: |cat <<EOF | sudo tee /etc/modules-load.d/k8s.confoverlaybr_netfilterEOF- name: 安装overlay模块shell: sudo modprobe overlay- name: 安装br_netfilter模块shell: sudo modprobe br_netfilter- name: 设置所需的 sysctl参数，参数在重新启动后保持不变shell:cmd: |cat <<EOF | sudo tee /etc/sysctl.d/k8s.confnet.bridge.bridge-nf-call-iptables  = 1net.bridge.bridge-nf-call-ip6tables = 1net.ipv4.ip_forward                 = 1EOF- name: 应用 sysctl 参数而不重新启动shell: sudo sysctl --system- name: 创建k8s.config文件 shell:cmd: |cat <<EOF | sudo tee /etc/modules-load.d/k8s.confoverlaybr_netfilterEOF- name: 安装overlay模块shell: sudo modprobe overlay- name: 安装br_netfilter模块shell: sudo modprobe br_netfilter- name: 设置所需的 sysctl参数，参数在重新启动后保持不变shell:cmd: |cat <<EOF | sudo tee /etc/sysctl.d/k8s.confnet.bridge.bridge-nf-call-iptables  = 1net.bridge.bridge-nf-call-ip6tables = 1net.ipv4.ip_forward                 = 1EOF- name: 应用 sysctl 参数而不重新启动shell: sudo sysctl --system

cri-docker rpm包下载地址

https://github.com/Mirantis/cri-dockerd/releases/tag/v0.2.5

kubeadm 初始化文件init.yaml

此文件可命令kubeadm config print init-defaults生成，生产以后按自己实际情况修改文件，不要抄！

apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:- system:bootstrappers:kubeadm:default-node-tokentoken: abcdef.0123456789abcdefttl: 24h0m0susages:- signing- authentication
kind: InitConfiguration
localAPIEndpoint:advertiseAddress: 192.168.100.101  #改成你自己的IP地址bindPort: 6443
nodeRegistration:criSocket: unix:///run/cri-dockerd.sock #改成这个套接字imagePullPolicy: IfNotPresentname: master01  #改成你自己的主机名taints: null
---
apiServer:timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:local:dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: 1.24.3
networking:dnsDomain: cluster.localserviceSubnet: 10.96.0.0/12
scheduler: {}

拉取必要镜像

kubeadm config images list命令可以查看1.24.3版本需要的镜像文件
国内仓库：registry.aliyuncs.com/google_containers
注意：etcd在registry.aliyuncs.com/google_containers仓库中可能找不到，可以上dockerhub上找找

[root@master01 ~]# kubeadm config images list
k8s.gcr.io/kube-apiserver:v1.24.3
k8s.gcr.io/kube-controller-manager:v1.24.3
k8s.gcr.io/kube-scheduler:v1.24.3
k8s.gcr.io/kube-proxy:v1.24.3
k8s.gcr.io/pause:3.7
k8s.gcr.io/etcd:3.5.3-0
k8s.gcr.io/coredns/coredns:v1.8.6

拉取指定仓库的镜像

kubeadm config images pull --image-repository="registry.aliyuncs.com/google_containers" --cri-socket="unix:///run/cri-dockerd.sock"

注意事项

安装好cri-docker 以后，直接kubeadm init --config init.yaml 会提示超时，查看kubelet日志会提示找不到节点
这时，你需要配置cri-docker.service文件，ExecStart=/usr/bin/cri-dockerd项后面指定你的指定你的pause版本，
例如：–pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.7，
错误如下

Error getting node" err="node \"master01\" not found

解决方法

[root@master01 ansible]# cat /usr/lib/systemd/system/cri-docker.service 
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket[Service]
Type=notify
ExecStart=/usr/bin/cri-dockerd --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.7 --container-runtime-endpoint fd://
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity
Delegate=yes
KillMode=process[Install]
WantedBy=multi-user.target

完成后重启cri-service服务

[root@master01 ~]# systemctl daemon-reload && systemctl restart cri-docker.service

此时在此运算kubeadm init 就能成功初始化集群

reset集群

与以往不同的是需要指定一下cri-socket

[root@master01 ~]# kubeadm reset --cri-socket="unix:///run/cri-dockerd.sock" --v=5

tmux

拉取镜像时需要很长时间，避免长时间不操作导致远程断开，你可以在tmux中执行，非常好用的小工具，建议安装
文档

部署CNI

可选CNI方案有如下几种

• Flannel：https://github.com/coreos/flannel
• Calico：https://github.com/projectcalico/cni-plugin
• Canal：https://github.com/projectcalico/canal
• Weave：https://www.weave.works/oss/net/
  这几种方案的对比请参考https://www.infoq.cn/article/gxFTM0X8z2zlhw9xlEzv
  以weave为例，官网上找到，复制下来运行一下
  https://www.weave.works/docs/net/latest/kubernetes/kube-addon/

$ kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"