基于 Kubernetes 1.22 版本

一、基础概念

动态卷供应（Dynamic Volume Provisioning）允许按需创建存储卷。如果没有动态供应，群集管理员必须手动调用其云或存储提供商来创建新的存储卷，然后创建 PV 对象以在 Kubernetes 中表示它们。动态供应功能消除了集群管理员预先供应存储的需要，相反，它会在用户请求时自动配置存储。
动态卷的实现是基于 StorageClass，每个 StorageClass 对象都指定一个卷插件（也称为 provisioner），该插件提供一个卷和一组参数，以便在设置时传递给该 provisioner。

二、NFS 动态卷配置

这里我们使用 Kubernetes NFS Subdir External Provisioner，它是一个自动供应器，它使用您现有的和已经配置的 NFS 服务器来支持通过 PVC 动态供应 PV。
参考文档：https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner
配置流程:

- 配置 NFS 服务器
- 获取 NFS Subdir External Provisioner 文件，地址：https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/tree/master/deploy
- 如果集群启用了 RBAC，设置授权。
- 配置 NFS subdir external provisioner
- 创建 Storage Class
- 创建 PVC 和 Pod 进行测试

1、配置 NFS

[root@master ~]# yum install -y nfs-utils
[root@master ~]# mkdir /nfs
[root@master ~]# echo "/nfs *(rw,no_root_squash)" > /etc/exports
[root@master ~]# systemctl restart nfs rpcbind
[root@master ~]# showmount -e localhost
Export list for localhost:
/nfs *

2、获取文件

[root@cp k8s]# git clone https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner.git
[root@cp k8s]# cd nfs-subdir-external-provisioner/
[root@cp nfs-subdir-external-provisioner]# rm -rf !(deploy)
[root@cp nfs-subdir-external-provisioner]# mv deploy/*.yaml ./ && rm -rf deploy
[root@cp nfs-subdir-external-provisioner]# ll
total 20
-rw-r--r-- 1 root root  255 Oct  4 04:31 class.yaml
-rw-r--r-- 1 root root 1064 Oct  4 04:31 deployment.yaml
-rw-r--r-- 1 root root 1900 Oct  4 04:31 rbac.yaml
-rw-r--r-- 1 root root  199 Oct  4 04:31 test-claim.yaml
-rw-r--r-- 1 root root  401 Oct  4 04:31 test-pod.yaml

3、配置 RBAC

直接使用获取的原始文件，只改变了命名空间：

[root@cp nfs-subdir-external-provisioner]# sed -i'' "s/namespace:.*/namespace: nfs/g" rbac.yaml deployment.yaml
[root@cp nfs-subdir-external-provisioner]# cat rbac.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

4、配置 NFS subdir external provisioner

填写之前配置的 NFS Server 信息：

[root@cp nfs-subdir-external-provisioner]# cat deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: k8s-sigs.io/nfs-subdir-external-provisioner
            - name: NFS_SERVER
              value: 10.60.77.22
            - name: NFS_PATH
              value: /nfs
      volumes:
        - name: nfs-client-root
          nfs:
            server: 10.60.77.22
            path: /nfs

执行：

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f rbac.yaml -f deployment.yaml 
[root@cp nfs-subdir-external-provisioner]# kubectl get pod -n nfs
NAME                                     READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-d89bb6fcd-tkwwb   1/1     Running   0          2s

5、创建 Storage class

[root@cp nfs-subdir-external-provisioner]# cat class.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f class.yaml 
[root@cp nfs-subdir-external-provisioner]# kubectl get sc
NAME                  PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate           false                  9s

6、测试

创建 PVC 测试：

[root@cp nfs-subdir-external-provisioner]# cat test-claim.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
spec:
  storageClassName: managed-nfs-storage
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f test-claim.yaml 
[root@cp nfs-subdir-external-provisioner]# kubectl get pvc
NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
test-claim   Bound    pvc-feafa663-67e9-49fc-bdc3-0b199de77845   1Mi        RWX            managed-nfs-storage   5s

[root@cp nfs-subdir-external-provisioner]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                STORAGECLASS          REASON   AGE
pvc-b5a0ea83-4f73-416f-862b-a0ffda2c3054   1Mi        RWX            Delete           Bound    default/test-claim   managed-nfs-storage            7s

创建 Pod 测试：

[root@cp nfs-subdir-external-provisioner]# cat test-pod.yaml 
kind: Pod
apiVersion: v1
metadata:
  name: test-pod
spec:
  containers:
  - name: test-pod
    image: busybox:stable
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "touch /mnt/SUCCESS && exit 0 || exit 1"
    volumeMounts:
      - name: nfs-pvc
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: nfs-pvc
      persistentVolumeClaim:
        claimName: test-claim

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f test-pod.yaml 
[root@cp nfs-subdir-external-provisioner]# kubectl get pod
NAME                                     READY   STATUS      RESTARTS   AGE
test-pod                                 0/1     Completed   0          28s
[root@cp nfs-subdir-external-provisioner]# tree /nfs
/nfs
└── default-test-claim-pvc-feafa663-67e9-49fc-bdc3-0b199de77845
    └── SUCCESS

默认目录路径为：${namespace}-${pvcName}-${pvName}

7、Storage Class 相关参数

NFS subdir external provisioner 的 Storage Class 相关参数：

参数	描述	备注
onDelete	如果存在且值为delete，则删除该目录；如果存在值为retain，则保存该目录。	将在共享上以名称归档：archived-
archiveOnDelete	默认为true，即删除时会保存目录。如果存在且值为false，则删除该目录。如果onDelete存在，archiveOnDelete将被忽略。	将在共享上以名称归档：archived-
pathPattern	指定用于通过 PVC 元数据（例如标签、注释、名称或命名空间）创建目录路径的模板。要指定元数据，请使用${.PVC.}。示例：如果文件夹应命名为 –，则使用${.PVC.namespace}-${.PVC.name} 作为路径模式。	n/a

示例：

[root@cp nfs-subdir-external-provisioner]# cat class-test.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-test
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:
  pathPattern: "${.PVC.namespace}/${.PVC.annotations.nfs.io/storage-path}"
  onDelete: delete

[root@cp nfs-subdir-external-provisioner]# cat test-pvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
spec:
  storageClassName: nfs-test
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f class-test.yaml -f test-pvc.yaml -f test-pod.yaml
[root@cp nfs-subdir-external-provisioner]# tree /nfs
/nfs
└── default
    └── SUCCESS

参考文章：

https://kubernetes.io/docs/
https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner

一、基础概念

二、NFS 动态卷配置

1、配置 NFS

2、获取文件

3、配置 RBAC

4、配置 NFS subdir external provisioner

5、创建 Storage class

6、测试

7、Storage Class 相关参数

你可能也喜欢

Kubernetes 部署 LNMP 环境

Kubernetes Volume

Kubernetes Local PV

发表评论 取消回复

发表评论取消回复