KubeVirt部署及使用说明

发表评论 / KubeVirt / 作者：

一、简介

KubeVirt is a virtual machine management add-on for Kubernetes. The aim is to provide a common ground for virtualization solutions on top of Kubernetes.
Kubevirt是Kubernetes的虚拟机管理附加组件。目的是为Kubernetes之上的虚拟化解决方案提供共同点。

项目地址：https://github.com/kubevirt/kubevirt

二、架构


  • virt-api
    kubevirt 是 CRD 形式管理 vm pod，virt-api 就是去所有虚拟化操作 的入口，包括常规的 CRD 更新验证以及vm start、stop
  • virt-controlller
    Virt-controller会根据vmi CRD，生成virt-lancher pod，并维护CRD的状态
  • virt-handler
    virt-handler会以 Daemonset 的状态部署在每个节点上，负责监控上每个虚拟机实例的状态变化，一旦检测到变化，会进行响应并确保相应的操作能够达到要求的（）状态。
    virt-handler保持集群级之间的同步规范与 libvirt 的同步报告 Libvirt 和集群的规范；调用以节点为中心的变化域 VMI 规范定义的网络状态和管理要求。
  • virt-launcher
    virt-lanuncher pod一个 VMI，kubelet 只是负责运行状态，不会去管virt-lanuncher podVMI 创建情况。
    virt-handler会根据 CRD 参数配置去通知 virt-lanuncher 去使用本地 libvirtd 实例来启动 VMI，virt-lanuncher 会通过 pid 去管理 VMI，如果 pod 生命周期结束，virt-lanuncher 也会去通知 VMI 去终止。
    然后再去一个libvirtd，去virt-lanuncher pod，通过libvirtd去管理VM的生命周期，到t-中心，不再是以前的机器那套，libvirtd去管理多个VM。

三、虚拟机创建流程

  • client 发送创建VMI命令达到k8s API server.
  • K8S API 创建VMI
  • virt-controller监听到VMI创建时，根据VMI spec生成pod spec文件，创建pods
  • k8s调度创建pods
  • virt-controller监听到pods创建后，根据pods的调度node，更新VMI 的nodeName
  • virt-handler监听到VMI nodeName与自身节点匹配后，与pod内的virt-launcher通信，virt-laucher创建虚拟机，并负责虚拟机生命周期管理
Client                     K8s API     VMI CRD  Virt Controller         VMI Handler
-------------------------- ----------- ------- ----------------------- ----------

                           listen <----------- WATCH /virtualmachines
                           listen <----------------------------------- WATCH /virtualmachines
                                                  |                       |
POST /virtualmachines ---> validate               |                       |
                           create ---> VMI ---> observe --------------> observe
                             |          |         v                       v
                           validate <--------- POST /pods              defineVMI
                           create       |         |                       |
                             |          |         |                       |
                           schedPod ---------> observe                    |
                             |          |         v                       |
                           validate <--------- PUT /virtualmachines       |
                           update ---> VMI ---------------------------> observe
                             |          |         |                    launchVMI
                             |          |         |                       |
                             :          :         :                       :
                             |          |         |                       |
DELETE /virtualmachines -> validate     |         |                       |
                           delete ----> * ---------------------------> observe
                             |                    |                    shutdownVMI
                             |                    |                       |
                             :                    :                       :


原文链接：https://iswbm.com/507.html

四、KubeVirt部署

部署文件地址：https://github.com/Mrliuch/kubevirt-deploy
部署文件已经过调整，已适配下文部署案例。

4.1 节点安装libvirt、qemu

# Ubuntu
$ apt install -y qemu-kvm libvirt-bin bridge-utils virt-manager

# CentOS
$ yum install -y qemu-kvm libvirt virt-install bridge-utils

4.2 检查节点是否支持虚拟化

[root@VM-4-27-centos ~]# virt-host-validate qemu
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : PASS
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
  QEMU: Checking for device assignment IOMMU support                         : PASS
  QEMU: Checking if IOMMU is enabled by kernel                               : WARN (IOMMU appears to be disabled in kernel. Add intel_iommu=on to kernel cmdline arguments)
  QEMU: Checking for secure guest support                                    : WARN (Unknown if this platform has Secure Guest support)

4.3 安装kubevirt

项目地址： https://github.com/kubevirt/kubevirt

kubectl apply -f kubevirt

或使用官方文件进行安装

$ export VERSION=$(curl -s https://api.github.com/repos/kubevirt/kubevirt/releases | grep tag_name | grep -v -- '-rc' | head -1 | awk -F': ' '{print $2}' | sed 's/,//' | xargs)
$ kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-operator.yaml
$ kubectl apply -f https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/kubevirt-cr.yaml

4.4 安装CDI

Containerized Data Importer（CDI）项是Kubernetes的持续存储管理附加组件。它的主要目标是提供一种声明性的方式来在kubevirt VMS上构建虚拟机磁盘
项目地址：https://github.com/kubevirt/containerized-data-importer

kubectl apply -f cdi

或使用官方文件进行安装

$ export VERSION=$(curl -s https://github.com/kubevirt/containerized-data-importer/releases/latest | grep -o "v[0-9]\.[0-9]*\.[0-9]*")
$ kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-operator.yaml
$ kubectl create -f https://github.com/kubevirt/containerized-data-importer/releases/download/$VERSION/cdi-cr.yaml

4.5 安装HostPath

kubectl apply -f hostpath

或使用官方文件进行安装

# hostpath provisioner operator 依赖于 cert manager 提供鉴权能力
$ kubectl create -f https://github.com/cert-manager/cert-manager/releases/download/v1.7.1/cert-manager.yaml

# 创建 hostpah-provisioner namespace
$ kubectl create -f https://raw.githubusercontent.com/kubevirt/hostpath-provisioner-operator/main/deploy/namespace.yaml

# 部署 operator
$ kubectl create -f https://raw.githubusercontent.com/kubevirt/hostpath-provisioner-operator/main/deploy/operator.yaml -n hostpath-provisioner
$ kubectl create -f https://raw.githubusercontent.com/kubevirt/hostpath-provisioner-operator/main/deploy/webhook.yaml

注意修改cr.yml里面的hostpath路径

4.6 镜像列表

kubevirt

quay.io/kubevirt/virt-api:v0.58.0
quay.io/kubevirt/virt-controller:v0.58.0
quay.io/kubevirt/virt-handler:v0.58.0
quay.io/kubevirt/virt-launcher:v0.58.0
quay.io/kubevirt/virt-operator:v0.58.0
quay.io/samblade/virtvnc:v0.1

hostpath

quay.io/kubevirt/hostpath-csi-driver:latest
k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.2.0
k8s.gcr.io/sig-storage/livenessprobe:v2.3.0
k8s.gcr.io/sig-storage/csi-provisioner:v2.2.1
quay.io/kubevirt/hostpath-provisioner-operator:latest
quay.io/jetstack/cert-manager-controller:v1.7.1
quay.io/jetstack/cert-manager-cainjector:v1.7.1
quay.io/jetstack/cert-manager-webhook:v1.7.1

cdi

quay.io/kubevirt/cdi-apiserver:v1.55.2
quay.io/kubevirt/cdi-controller:v1.55.2
quay.io/kubevirt/cdi-operator:v1.55.2
quay.io/kubevirt/cdi-uploadproxy:v1.55.2

五、创建虚拟机

参见KubeVirt创建虚拟机章节

5.1 使用iso系统镜像创建虚拟机

5.1.1 上传ISO镜像到PVC

注意：需要部署CDI

virtctl image-upload \
  --image-path='CentOS-7-x86_64-DVD-2009.iso' \
  --storage-class hostpath-csi \
  --pvc-name=iso-centos \
  --pvc-size=5Gi \
  --uploadproxy-url=https://<cdi-uploadproxy_svc_ip> \
  --insecure \
  --wait-secs=240
  • –-image-path : 操作系统镜像的本地地址
  • –-pvc- : 指定存储运行镜像的PVC过程，这个PVC不需要提前准备好自动上传中会创建。
  • –-pvc-size：PVC大小，根据镜像大小来设置，一般略大一个G就行
  • –uploadproxy-url : cdi-uploadproxy 的Service IP，可以通过命令kubectl -n cdi get svc -l cdi.kubevirt.io=cdi-uploadproxy来查看。

5.1.2 创建虚拟机

# vm.yml

apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
  name: centos
spec:
  running: true
  template:
    metadata:
      labels:
        kubevirt.io/domain: centos
    spec:
      domain:
        cpu:
          cores: 2
        devices:
          disks:
          - disk:
              bus: virtio
            name: data
            bootOrder: 1
          - cdrom:
              bus: sata
            name: cdromiso
            bootOrder: 2
          interfaces:
          - masquerade: {}
            model: e1000
            name: default
        machine:
          type: q35
        resources:
          requests:
            memory: 4G
      networks:
      - name: default
        pod: {}
      volumes:
      - name: cdromiso
        persistentVolumeClaim:
          claimName: iso-centos
      - name: data
        hostDisk:
          capacity: 50Gi
          path: /home/disk.img
          type: DiskOrCreate

kubectl apply -f vm.yml

5.1.3 启动虚拟机

[root@192 vm]# virtctl start centos
[root@192 vm]# kubectl get vmi
NAME               AGE     PHASE     IP            NODENAME         READY
centos             5h25m   Running   10.20.0.112   192.168.10.160   True

[root@192 vm]# kubectl get po
NAME                                   READY   STATUS    RESTARTS   AGE
virt-launcher-centos-dw2ts             1/1     Running   0          5h26m

5.2 使用qcow2格式镜像创建虚拟机

5.2.1 制作镜像

centos qcow2下载地址：https://cloud.centos.org/centos/7/images

Dockerfile

FROM kubevirt/registry-disk-v1alpha
COPY CentOS-7-x86_64-GenericCloud-1805.qcow2 /disk/centos.img

docker build -t xxx .

5.2.2 创建VM

[root@192 vm]# cat vm-container.yml
apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
  name: centos-container
spec:
  running: true
  template:
    metadata:
      labels:
        kubevirt.io/domain: centos-container
    spec:
      domain:
        cpu:
          cores: 2
        devices:
          disks:
          - disk:
              bus: virtio
            name: cdromiso
          - disk:
              bus: virtio
            name: data
          interfaces:
          - masquerade: {}
            model: e1000
            name: default
        machine:
          type: q35
        resources:
          requests:
            memory: 4G
      networks:
      - name: default
        pod: {}
      volumes:
      - name: cdromiso
        containerDisk:
          image: home.mrlch.cn:8888/kubevirt/centos:1805-1
      - name: data
        hostDisk:
          capacity: 50Gi
          path: /home/disk-container.img
          type: DiskOrCreate

kubectl apply -f vm-container.yml

5.2.3 启动虚拟机

[root@192 vm]# virtctl start centos-container
[root@192 vm]# kubectl get vmi
NAME               AGE     PHASE     IP            NODENAME         READY
centos-container   3h21m   Running   10.20.0.114   192.168.10.153   True

[root@192 vm]# kubectl get po
NAME                                   READY   STATUS    RESTARTS   AGE
virt-launcher-centos-container-wdzd8   2/2     Running   0          3h22m

六、KubeVirt存储

原文链接：https://iswbm.com/385.html

七、KubeVirt网络

VLAN:
https://www.bookstack.cn/read/Kube-OVN-1.2-zh/f1cf5755a5c72519.md
https://www.modb.pro/db/459490

发表评论

您的电子邮箱地址不会被公开。 必填项已用 * 标注

Captcha Code