kubernetes(k8s)+harbor本地仓库的集群部署及ansibel批量工具的利用

      本文主要介绍如何部署一个 Kubernetes（k8s）集群，包含一个本地 Harbor 私有镜像仓库，采用一主两从的集群架构，并使用 Ansible 工具进行批量自动化部署。

一.harbor仓库规划部署

1.部署dockers本地仓库

创建dockers的本地下载仓库

[root@harbor ~]# vim /etc/yum.repos.d/docker.repo
[docker]
name = docker
baseurl = http://172.25.254.250:4444/docker
gpgcheck = 0
~

下载安装两个软件包：httpd 和 createrepo

[root@harbor ~]# dnf install httpd createrepo -y
[root@harbor ~]# mkdir  /var/www/html/docker/ -p

更改httpd服务端口为实验所需端口
[root@harbor ~]# vim /etc/httpd/conf/httpd.conf
# available when the service starts.  See the httpd.service(8) man
# page for more information.
#Listen 12.34.56.78:80
Listen 4444

[root@harbor ~]# systemctl enable --now httpd

把阿里云 docker‑ce 的 rpm 包下载到本地，通过 httpd 搭建内网 yum 仓库

[root@harbor ~]# dnf  install docker-ce -downloadonly  --destdir /mnt/ -y
[root@harbor ~]# mv /mnt/*.rpm /var/www/html/docker/

生成yum/dnf仓库元数据repodata目录
[root@harbor ~]# createrepo -v /var/www/html/docker/

[root@harbor ~]#vim /etc/yum.repos.d/docker.repo

[docker]
name = docker
baseurl = http://172.25.254.250:4444/docker
gpgcheck = 0
~

使用搭建本地仓库下载dockers
[root@harbor ~]# dnf install docker-ce -y

2安装docker在harbor仓库节点

配置开机自动加载 br_netfilter 内核模块

[root@harbor ~]# echo br_netfilter > /etc/modules-load.d/docker_mod.conf
[root@harbor ~]# modprobe -a br_netfilter

配置sysctl内核参数文件 /etc/sysctl.d/docker.conf
[root@harbor ~]# vim /etc/sysctl.d/docker.conf
net.bridge.bridge-nf-call-iptables = 1

#网桥流量交给 iptables 处理，K8s Service iptables 模式依赖此项；等于 0 网络会异常，service 访问不通。
net.bridge.bridge-nf-call-ip6tables = 1

#ipv6 网桥交给 iptables。
net.ipv4.ip_forward = 1

#开启主机 IP 转发，容器之间、Pod 跨节点通信必须开启

[root@harbor ~]# systemctl restart  systemd-modules-load.service
[root@harbor ~]# sysctl  --system

修改docker service单元文件，dockerd启动参数

[root@harbor ~]# vim /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --iptables=true  ##dockerd 允许 docker 操作主机 iptables 规则
[root@harbor ~]# systemctl daemon-reload
[root@harbor ~]# systemctl enable --now docker

3.创建Harbor 私有仓库自签 TLS 证书

[root@harbor ~]# mkdir /data/certs -p
[root@harbor ~]# openssl req -newkey  rsa:4096 \
-nodes -sha256 -keyout /data/certs/timinglee.org.key \
-addext "subjectAltName = DNS:reg.timinglee.org" \
-x509 -days 365 -out /data/certs/timinglee.org.crt

生成一套 Harbor 私有仓库使用的自签名 TLS HTTPS 证书，包含私钥 + 公钥证书，域名是 reg.timinglee.org，有效期 365 天。

You are about to be asked to enter information that will be incorporated
into your certificate request.

4.编辑harbor配置文件

[root@harbor ~]# tar zxf  harbor-offline-installer-v2.5.4.tgz -C /opt/
[root@harbor ~]# cd /opt/harbor/
[root@harbor harbor]# ls
common     cri-dockerd-0.4.4.amd64.tgz  harbor.v2.5.4.tar.gz  harbor.yml.tmpl  LICENSE
common.sh  docker-compose.yml           harbor.yml            install.sh       prepare
[root@harbor harbor]#
[root@harbor harbor]# cp harbor.yml.tmpl harbor.yml

编辑配置文件
[root@harbor harbor]# vim harbor.yml

 16   # The path of cert and key files for nginx
 17   certificate: /data/certs/timinglee.org.crt
 18   private_key: /data/certs/timinglee.org.key
 19 # # Uncomment following will enable tls communication between all harbor components
 32 # Remember Change the admin password from UI after launching Harbor.
 33 harbor_admin_password: lee

执行 harbor 官方安装脚本，部署 Harbor 私有镜像仓库，同时启用 chartmuseum
[root@harbor harbor]# ./install.sh --with-chartmuseum

编写harbor启动脚本

[root@harbor ~]# vim /lib/systemd/system/harbor.service

[Unit]
Description=harbor with Docker Compose
Documentation=https://reg.timinglee.com/compose/
After=docker.service network-online.target
Requires=docker.service

[Service]
Type=oneshot
WorkingDirectory=/opt/harbor
ExecStart=/usr/bin/docker compose up -d
ExecStop=/usr/bin/docker compose down
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

[root@harbor harbor]# docker compose  down

WARN[0000] /opt/harbor/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] down 4/6
 ⠇ Container registryctl       Stopping                                                1.9s
 ✔ Container nginx             Removed        

启动harbor

[root@harbor ~]# systemctl enable --now harbor

[root@harbor harbor]# docker compose ps
WARN[0000] /opt/harbor/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
NAME                IMAGE                                COMMAND                   SERVICE       CREATED      STATUS                 PORTS
harbor-core         goharbor/harbor-core:v2.5.4          "/harbor/entrypoint.…"   core          3 days ago   Up 7 hours (healthy)
harbor-db           goharbor/harbor-db:v2.5.4            "/docker-entrypoint.…"   postgresql    3 days ago   Up 7 hours (healthy)
harbor-jobservice   goharbor/harbor-jobservice:v2.5.4    "/harbor/entrypoint.…"   jobservice    3 days ago   Up 7 hours (healthy)
harbor-log          goharbor/harbor-log:v2.5.4           "/bin/sh -c /usr/loc…"   log           3 days ago   Up 7 hours (healthy)   127.0.0.1:1514->10514/tcp
harbor-portal       goharbor/harbor-portal:v2.5.4        "nginx -g 'daemon of…"   portal        3 days ago   Up 7 hours (healthy)
nginx               goharbor/nginx-photon:v2.5.4         "nginx -g 'daemon of…"   proxy         3 days ago   Up 7 hours (healthy)   0.0.0.0:80->8080/tcp, [::]:80->8080/tcp, 0.0.0.0:443->8443/tcp, [::]:443->8443/tcp
redis               goharbor/redis-photon:v2.5.4         "redis-server /etc/r…"   redis         3 days ago   Up 7 hours (healthy)
registry            goharbor/registry-photon:v2.5.4      "/home/harbor/entryp…"   registry      3 days ago   Up 7 hours (healthy)
registryctl         goharbor/harbor-registryctl:v2.5.4   "/home/harbor/start.…"   registryctl   3 days ago   Up 7 hours (healthy)
 

5.启动harbor并验证是否成功

[root@harbor harbor]# mkdir  /etc/docker/certs.d/reg.timinglee.org/ -p

把 harbor 服务端生成的公钥证书 crt复制到 docker 证书目录，命名为ca.crt
[root@harbor harbor]# cp /data/certs/timinglee.org.crt  /etc/docker/certs.d/reg.timinglee.org/ca.crt

添加harbor主机域名映射

[root@harbor harbor]# vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.25.254.250     harbor reg.timinglee.org

[root@harbor harbor]# systemctl restart docker
[root@harbor harbor]# docker compose up -d

[root@harbor harbor]# docker compose up -d
WARN[0000] /opt/harbor/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] up 9/9
 ✔ Container harbor-jobservice Running                                                         0.0s
 ✔ Container redis             Running                                                         0.0s
 ✔ Container registry          Running                                                         0.0s
 ✔ Container harbor-log        Running                                                         0.0s
 ✔ Container registryctl       Running                                                         0.0s
 ✔ Container harbor-portal     Running                                                         0.0s
 ✔ Container nginx             Running                                                         0.0s
 ✔ Container harbor-core       Running                                                         0.0s
 ✔ Container harbor-db         Running                                                         0.0s
[root@harbor harbor]#
[root@harbor harbor]# docker login  reg.timinglee.org -u admin
Password:
Login Succeeded

这样就成功搭建好了harbor仓库

二.构建部署kubernetes所需主机

k8s集群服务器规划：

1.所有k8s主机基础配置

Kubernetes 硬性要求：必须关闭 swap 交换分区，否则 kubelet 无法正常启动，会报错

systemctl disable --now  swap.target
systemctl mask swap.target
sed '/swap/s/^/#/g' -i /etc/fstab

给每台k8s主机安装docker

[root@k8s-master~]# dnf install docker-ce -y

[root@k8s-node1~]# dnf install docker-ce -y

[root@k8s-node2~]# dnf install docker-ce -y

给每个主机配置之前搭建的harbor仓库

每个主机创建

mkdir  /etc/docker/certs.d/reg.timinglee.org/ -p

在harbor主机中分发证书到所有主机

[root@harbor ~]# for i in 100 10 20
> do
> scp /data/certs/timinglee.org.crt root@172.25.254.$i:/etc/docker/certs.d/reg.timinglee.org/ca.crt
> done

重启docker

systemctl enable  docker
systemctl restart docker

给所有主机配置docker加速器--三台主机全部做

[root@k8s-master ~]# vim /etc/docker/daemon.json
{
  "registry-mirrors":["https://reg.timinglee.org"]
}

[root@k8s-master ~]# docker info
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Registry Mirrors:
  https://reg.timinglee.org/
 Live Restore Enabled: false
 Firewall Backend: iptables
  EnableUserlandProxy: true
  UserlandProxyPath: /usr/bin/docker-proxy

所有主机建立彼此解析--所有主机相互建立

[root@k8s-node1 ~]# vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.25.254.100      k8s-master
172.25.254.10       k8s-node1
172.25.254.20       k8s-node2
172.25.254.250      harbor  reg.timinglee.org
~

所有主机配置kubernetes安装源

[root@k8s-master ~]# vim /etc/yum.repos.d/kubernetes.repo
[root@k8s-master ~]#
[kubernetes]
name = kubernetes
baseurl = https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.35/rpm/
gpgcheck = 0

三.kubernetes的部署

1.二进制包方式部署cri-dockerd--所有主机安装

下载二进制安装包

[root@docker-node1 ~]# wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.4.4/cri-dockerd-0.4.4.amd64.tgz

解压二进制压缩包
[root@docker-node1 ~]# tar zxf cri-dockerd-0.4.4.amd64.tgz

[root@docker-node1 ~]# cd cri-dockerd/
[root@docker-node1 cri-dockerd]# ls
cri-dockerd  cri-docker.service  cri-docker.socket

安装cri-docker命令到系统

[root@docker-node1 cri-dockerd]# install -o root -g root -m 0755 cri-dockerd  /usr/local/bin/cri-dockerd

生成启动文件

[root@docker-node1 cri-dockerd]# cp cri-docker.s* /lib/systemd/system
cp: overwrite '/lib/systemd/system/cri-docker.service'? y
[root@docker-node1 cri-dockerd]# chmod  +x /lib/systemd/system/cri-docker.s*

[root@docker-node1 cri-dockerd]# vim /lib/systemd/system/cri-docker.service
[Service]
Type=notify
ExecStart=/usr/local/bin/cri-dockerd --network-plugin=cni --pod-infra-container-image=reg.timinglee.org/k8s/pause:3.10.1 --container-runtime-endpoint fd://
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
[root@docker-node1 cri-dockerd]# systemctl daemon-reload

启动k8s服务

[root@docker-node1 cri-dockerd]# systemctl enable --now cri-docker.service
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /usr/lib/systemd/system/cri-docker.service.
[root@docker-node1 cri-dockerd]# systemctl status cri-docker.service
● cri-docker.service - CRI Interface for Docker Application Container Engine
     Loaded: loaded (/usr/lib/system

2.安装构建kubernetes 集群所需软件

master

dnf install kubelet kubeadm kubectl -y
systemctl enable --now kubelet.service

node节点

dnf install kubelet kubeadm  -y
systemctl enable --now kubelet.service

master节点中 kubectl 和kubeadm 补齐

[root@master ~]# echo "source <(kubectl completion bash)" >> ~/.bashrc
[root@master ~]# echo "source <(kubeadm completion bash)" >> ~/.bashrc
[root@master ~]# source  ~/.bashrc

3.下载kubernetes集群所需镜像

[root@k8s-master ~]# kubeadm config images list
I0820 09:44:45.514598   81700 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.35
registry.k8s.io/kube-apiserver:v1.35.7
registry.k8s.io/kube-controller-manager:v1.35.7
registry.k8s.io/kube-scheduler:v1.35.7
registry.k8s.io/kube-proxy:v1.35.7
registry.k8s.io/coredns/coredns:v1.13.1
registry.k8s.io/pause:3.10.1
registry.k8s.io/etcd:3.6.6-0

下载镜像

[root@master ~]# kubeadm config images pull \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.35.7 \
--cri-socket=unix:///var/run/cri-dockerd.sock

[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.35.3
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.35.3

4.上传镜像到本地harbor

[root@master ~]# docker  login  reg.timinglee.org -u admin
Password:

WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/

Login Succeeded

[root@master ~]# docker images  --format  "{{.Repository}}:{{.Tag}}" | awk -F "/" '/google/{system("docker tag "$0" reg.timinglee.org/k8s/"$3)}'

[root@master ~]# docker images  --format  "{{.Repository}}:{{.Tag}}" | awk -F "/" '/timinglee/{system("docker push "$0)}'

四.在master中初始化kubernetes集群

1.在master中完成集群初始化

[root@master ~]# kubeadm init --pod-network-cidr=10.244.0.0/16 \
--image-repository reg.timinglee.org/k8s \
--kubernetes-version v1.35.7 \
--cri-socket=unix:///var/run/cri-dockerd.sock

[root@k8s-master ~]# kubeadm token create --print-join-command
kubeadm join 172.25.254.100:6443 --token h115sx.xw4u6it67qou4mnr --discovery-token-ca-cert-hash sha256:03bb100d74df76f93186da52f2c6bd981592fa7a7b18fae97355c713322e4aa2
[root@k8s-master ~]#

在每个node节点添加凭证加入master集群

[root@k8s-node1 ~]# kubeadm join 172.25.254.100:6443 --token h115sx.xw4u6it67qou4mnr --discovery-token-ca-cert-hash sha256:03bb100d74df76f93186da52f2c6bd981592fa7a7b18fae97355c713322e4aa2 --cri-socket=unix:///var/run/cri-dockerd.sock

[root@k8s-node2 ~]# kubeadm join 172.25.254.100:6443 --token h115sx.xw4u6it67qou4mnr --discovery-token-ca-cert-hash sha256:03bb100d74df76f93186da52f2c6bd981592fa7a7b18fae97355c713322e4aa2 --cri-socket=unix:///var/run/cri-dockerd.sock

如果初始化出问题

[root@master ~]# kubeadm reset  --cri-socket=unix:///var/run/cri-dockerd.sock #可以重置集群设定

[root@master ~]# kubectl get nodes

NAME         STATUS   ROLES           AGE    VERSION
k8s-master   Ready    control-plane   3d7h   v1.35.7
k8s-node1    Ready    <none>          3d7h   v1.35.7
k8s-node2    Ready    <none>          3d7h   v1.35.7
[root@k8s-master ~]#
 

2.添加kubernets环境变量到本机

[root@master ~]# echo "export KUBECONFIG=/etc/kubernetes/admin.conf" > ~/.bash_profile
[root@master ~]# source  ~/.bash_profile
[root@master ~]# kubectl get nodes
NAME     STATUS     ROLES           AGE    VERSION
master   NotReady   control-plane   102s   v1.35.3

五.安装网络插件

[root@k8s-master ~]# tar zxf kube-flannel-v0.28.9.tar.gz
[root@k8s-master ~]# ls
flannel  kube-flannel-v0.28.9.tar.gz
[root@k8s-master ~]# cd flannel/

[root@k8s-master flannel]# docker load  -i kube-flannel-v0.28.9.tar
Loaded image: ghcr.io/flannel-io/flannel-cni-plugin:v1.9.1-flannel3
Loaded image: ghcr.io/flannel-io/flannel:v0.28.9

将安装包上传到harbor仓库

[root@master ~]# docker tag ghcr.io/flannel-io/flannel-cni-plugin:v1.9.0-flannel1 reg.timinglee.org/flannel-io/flannel-cni-plugin:v1.9.0-flannel1
[root@master ~]# docker push  reg.timinglee.org/flannel-io/flannel-cni-plugin:v1.9.0-flannel1

[root@master ~]# docker tag ghcr.io/flannel-io/flannel:v0.28.1 reg.timinglee.org/flannel-io/flannel:v0.28.1
[root@master ~]# docker push reg.timinglee.org/flannel-io/flannel:v0.28.1

[root@master ~]# vim kube-flannel.yml

image: flannel-io/flannel:v0.28.1
        image: flannel-io/flannel-cni-plugin:v1.9.0-flannel1
        image: flannel-io/flannel:v0.28.1

[root@master ~]# kubectl apply -f kube-flannel.yml
namespace/kube-flannel created
serviceaccount/flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created

[root@master ~]# kubectl get  nodes
NAME     STATUS   ROLES           AGE     VERSION
master   Ready    control-plane   13m     v1.35.3
node1    Ready    <none>          9m44s   v1.35.3
node2    Ready    <none>          9m36s   v1.35.3

六.利用ansible部署kubernetes运行环境

1.安装ansible

[root@harbor ~]# dnf install ansible-core.x86_64 ansible.noarch -y

2.配置所需用户

[root@harbor ~]# useradd  devops
[root@harbor ~]# echo lee | passwd --stdin devops
更改用户 devops 的密码 。
passwd：所有的身份验证令牌已经成功更新

[root@harbor ~]# su - devops
[devops@harbor ~]$ mkdir  ansible
[devops@harbor ~]$ cd ansible/
[devops@harbor ansible]$

3.生成ansible的临时配置文件并建立运行环境

[devops@harbor ansible]$ vim inventory

[servers]
172.25.254.100
172.25.254.20
172.25.254.10

[devops@harbor ansible]$ vim ansible.cfg
[defaults]
inventory = /home/devops/ansible/inventory
remote_user = root
ask_pass = false
host_key_checking = false

[devops@harbor ansible]$ ansible all -m ping
172.25.254.20 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
172.25.254.10 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
172.25.254.100 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
[devops@harbor ansible]$

建立用户：

[devops@harbor ansible]$ ansible all -m user -a "name=devops state=present"
[devops@harbor ansible]$ ansible all -m shell -a 'echo "lee" | passwd --stdin devops'
[devops@harbor ansible]$ ansible all -m lineinfile -a 'path=/etc/sudoers line="devops    ALL=(ALL)    NOPASSWD: ALL" insertafter=EOF'

4.生成ansible的主配置文件并测试

[devops@harbor ansible]$ vim ansible.cfg
[defaults]
inventory = /home/devops/ansible/inventory
remote_user = devops
ask_pass = false
host_key_checking = false

[privilege_escalation]
become = true
become_method = sudo
become_user = root
become_ask_pass = false

[devops@harbor ansible]$ ansible all -m shell -a 'whoami '
172.25.254.10 | CHANGED | rc=0 >>
root
172.25.254.100 | CHANGED | rc=0 >>
root
172.25.254.20 | CHANGED | rc=0 >>
root

5.利用ansible剧本部署docker，安装cri-dockerd以及部署k8s软件

[devops@harbor ansible]$ tar zxf cri-dockerd-0.4.4.amd64.tgz
[devops@harbor ansible]$ ls
ansible.cfg  cri-dockerd  cri-dockerd-0.4.4.amd64.tgz  inventory  kubernetes-devops.yml
[devops@harbor ansible]$ cd cri-dockerd/
[devops@harbor cri-dockerd]$ ls
cri-dockerd  cri-docker.service  cri-docker.socket
[devops@harbor cri-dockerd]$ vim cri-docker.service
[Service]
Type=notify
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --pod-infra-container-image=reg.timinglee.org/k8s/pause:3.10.1 
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

ansibel剧本搭建

[devops@harbor ansible]$ vim kubernetes-devops.yml

- name: devops k8s from ansible
  hosts: all
  become: true
  tasks:
    - name: setup docker repo
      yum_repository:
        name: docker
        description: docker
        baseurl: http://172.25.254.254:4444/docker
        file: docker
        gpgcheck: no

    - name: install docker
      dnf:
        name: docker-ce
        state: present

    - name: setup docker.service modify ExecStart
      replace:
        path: /lib/systemd/system/docker.service
        regexp: "ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock"
        replace: "ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --iptables=true"

    - name: daemon‑reload after modify docker.service
      systemd:
        daemon_reload: yes

    - name: setup docker registry daemon.json
      copy:
        content: |
          {
            "registry-mirrors":["https://reg.timinglee.org"]
          }
        dest: /etc/docker/daemon.json

    - name: setup docker certs directory
      file:
        path: /etc/docker/certs.d/reg.timinglee.org
        state: directory
        mode: '0755'

    - name: copy harbor ca.crt to target nodes
      copy:
        src: /data/certs/timinglee.org.crt
        dest: "{{ item.dest }}"
      loop:
           - dest: /etc/docker/certs.d/reg.timinglee.org/ca.crt
           - dest: /etc/pki/ca-trust/source/anchors/ca.crt
  

- name: write br_netfilter module config
      copy:
        dest: "{{ item.dest }}"
        content: "{{ item.value }}"
      loop:
        - dest: /etc/modules-load.d/docker_mod.conf
          value: br_netfilter
        - dest: /etc/sysctl.d/docker.conf
          value: |
            net.bridge.bridge-nf-call-iptables = 1
            net.bridge.bridge-nf-call-ip6tables = 1
            net.ipv4.ip_forward = 1

    - name: update ca trust and reload sysctl
      shell: |
        update-ca-trust extract
        sysctl --system
      args:
        executable: /bin/bash

    - name: start and enable services
      service:
        name: "{{ item }}"
        state: restarted
        enabled: yes
      loop:
        - docker
        - systemd-modules-load.service
    - name: setup cri-dockerd binary and unit files
      copy:
        src: "{{ item.src }}"
        dest: "{{ item.dest }}"
        mode: "{{ item.mode }}"
      loop:
        - src: ./cri-dockerd/cri-dockerd
          dest: /usr/bin/cri-dockerd
          mode: '0755'
        - src: ./cri-dockerd/cri-docker.service
          dest: /lib/systemd/system/cri-docker.service
          mode: '0644'
        - src: ./cri-dockerd/cri-docker.socket
          dest: /lib/systemd/system/cri-docker.socket
          mode: '0644'

 - name: systemd daemon-reload after push cri‑docker unit files
      systemd:
        daemon_reload: yes

    - name: update ca trust and apply sysctl config
      shell: |
        update-ca-trust extract
        sysctl --system
      args:
        executable: /bin/bash

    - name: start and enable services
      service:
        name: "{{ item }}"
        state: restarted
        enabled: yes
      loop:
        - docker
        - systemd-modules-load.service
        - cri-docker
- name: install kubernetes from ansible
  hosts: all
  tasks:
  - name: setup k8s repo
    yum_repository:
      name: kubernetes
      description: kubernetes
      baseurl: https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.35/rpm
      file: kubernetes
      gpgcheck: no

  - name: install software in master
    dnf:
      name: "{{item}}"
      state: present
    loop:
      - kubeadm-1.35.7-150500.1.1
      - kubelet-1.35.7-150500.1.1
      - kubectl-1.35.7-150500.1.1
    when: inventory_hostname == "172.25.254.100"

  - name: install software in work node
    dnf:
      name: "{{item}}"
      state: present
    loop:
      - kubeadm-1.35.7-150500.1.1
      - kubelet-1.35.7-150500.1.1
    when: inventory_hostname == "172.25.254.10" or inventory_hostname == "172.25.254.20"

  - name: setup kubectl kubeadm complication
    lineinfile:
      path: /root/.bashrc
      line: |
        source <(kubectl completion bash)
        source <(kubeadm completion bash)
    when: inventory_hostname == "172.25.254.100"

  - name: start kubelet
    service:
      name: kubelet
      state: started
      enabled: yes
 

可以利用ansible一键批量部署k8s

使用ansible剧本

[devops@harbor ansible]$ ansible-playbook kubernetes-devops.yml

部署完成后

#让补齐功能生效
[root@k8s-master mnt]# source  ~/.bashrc
[root@k8s-master mnt]# kubectl 命令后按2此tab键
alpha          (Commands for features in alpha)
annotate       (更新一个资源的注解)
api-resources  (Print the supported API resources on the server)
api-versions   (Print the supported API versions on the server, in the form of "group/version")
apply          (Apply a configuration to a resource by file name or stdin)
attach         (挂接到一个运行中的容器)
auth           (Inspect authorization)
autoscale      (Auto-scale a deployment, replica set, stateful set, or replication controller)
certificate    (Modify certificate resources)

[root@k8s-master mnt]# kubeadm
certs       config      init        kubeconfig  token       version
completion  help        join        reset       upgrade

[root@k8s-master mnt]# systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
     Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; preset: disabled)
    Drop-In: /usr/lib/systemd/system/kubelet.service.d
             └─10-kubeadm.conf
     Active: activating (auto-restart) (Result: exit-code) since Tue 2026-08-18 17:09:25 CST; 1s ago
       Docs: https://kubernetes.io/docs/
    Process: 79433 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_>
   Main PID: 79433 (code=exited, status=1/FAILURE)