Kubernetes 动态卷—NFS

  • Post author:
  • Post category:Kubernetes
  • Post comments:0评论
基于 Kubernetes 1.22 版本

一、基础概念

  动态卷供应（Dynamic Volume Provisioning）允许按需创建存储卷。如果没有动态供应，群集管理员必须手动调用其云或存储提供商来创建新的存储卷，然后创建 PV 对象以在 Kubernetes 中表示它们。动态供应功能消除了集群管理员预先供应存储的需要，相反，它会在用户请求时自动配置存储。
  动态卷的实现是基于 StorageClass，每个 StorageClass 对象都指定一个卷插件（也称为 provisioner），该插件提供一个卷和一组参数，以便在设置时传递给该 provisioner。

 

二、NFS 动态卷配置

  这里我们使用 Kubernetes NFS Subdir External Provisioner，它是一个自动供应器，它使用您现有的和已经配置的 NFS 服务器来支持通过 PVC 动态供应 PV。
  参考文档：https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner
  配置流程:

- 配置 NFS 服务器
- 获取 NFS Subdir External Provisioner 文件，地址：https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/tree/master/deploy
- 如果集群启用了 RBAC，设置授权。
- 配置 NFS subdir external provisioner
- 创建 Storage Class
- 创建 PVC 和 Pod 进行测试

 

1、配置 NFS

[root@master ~]# yum install -y nfs-utils
[root@master ~]# mkdir /nfs
[root@master ~]# echo "/nfs *(rw,no_root_squash)" > /etc/exports
[root@master ~]# systemctl restart nfs rpcbind
[root@master ~]# showmount -e localhost
Export list for localhost:
/nfs *

 

2、获取文件

[root@cp k8s]# git clone https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner.git
[root@cp k8s]# cd nfs-subdir-external-provisioner/
[root@cp nfs-subdir-external-provisioner]# rm -rf !(deploy)
[root@cp nfs-subdir-external-provisioner]# mv deploy/*.yaml ./ && rm -rf deploy
[root@cp nfs-subdir-external-provisioner]# ll
total 20
-rw-r--r-- 1 root root  255 Oct  4 04:31 class.yaml
-rw-r--r-- 1 root root 1064 Oct  4 04:31 deployment.yaml
-rw-r--r-- 1 root root 1900 Oct  4 04:31 rbac.yaml
-rw-r--r-- 1 root root  199 Oct  4 04:31 test-claim.yaml
-rw-r--r-- 1 root root  401 Oct  4 04:31 test-pod.yaml

 

3、配置 RBAC

直接使用获取的原始文件，只改变了命名空间：

[root@cp nfs-subdir-external-provisioner]# sed -i'' "s/namespace:.*/namespace: nfs/g" rbac.yaml deployment.yaml
[root@cp nfs-subdir-external-provisioner]# cat rbac.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: nfs
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

 

4、配置 NFS subdir external provisioner

填写之前配置的 NFS Server 信息：

[root@cp nfs-subdir-external-provisioner]# cat deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: nfs
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: k8s-sigs.io/nfs-subdir-external-provisioner
            - name: NFS_SERVER
              value: 10.60.77.22
            - name: NFS_PATH
              value: /nfs
      volumes:
        - name: nfs-client-root
          nfs:
            server: 10.60.77.22
            path: /nfs

执行：

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f rbac.yaml -f deployment.yaml 
[root@cp nfs-subdir-external-provisioner]# kubectl get pod -n nfs
NAME                                     READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-d89bb6fcd-tkwwb   1/1     Running   0          2s

 

5、创建 Storage class

[root@cp nfs-subdir-external-provisioner]# cat class.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f class.yaml 
[root@cp nfs-subdir-external-provisioner]# kubectl get sc
NAME                  PROVISIONER                                   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage   k8s-sigs.io/nfs-subdir-external-provisioner   Delete          Immediate           false                  9s

 

6、测试

创建 PVC 测试：

[root@cp nfs-subdir-external-provisioner]# cat test-claim.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
spec:
  storageClassName: managed-nfs-storage
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f test-claim.yaml 
[root@cp nfs-subdir-external-provisioner]# kubectl get pvc
NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
test-claim   Bound    pvc-feafa663-67e9-49fc-bdc3-0b199de77845   1Mi        RWX            managed-nfs-storage   5s

[root@cp nfs-subdir-external-provisioner]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                STORAGECLASS          REASON   AGE
pvc-b5a0ea83-4f73-416f-862b-a0ffda2c3054   1Mi        RWX            Delete           Bound    default/test-claim   managed-nfs-storage            7s

创建 Pod 测试：

[root@cp nfs-subdir-external-provisioner]# cat test-pod.yaml 
kind: Pod
apiVersion: v1
metadata:
  name: test-pod
spec:
  containers:
  - name: test-pod
    image: busybox:stable
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "touch /mnt/SUCCESS && exit 0 || exit 1"
    volumeMounts:
      - name: nfs-pvc
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: nfs-pvc
      persistentVolumeClaim:
        claimName: test-claim

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f test-pod.yaml 
[root@cp nfs-subdir-external-provisioner]# kubectl get pod
NAME                                     READY   STATUS      RESTARTS   AGE
test-pod                                 0/1     Completed   0          28s
[root@cp nfs-subdir-external-provisioner]# tree /nfs
/nfs
└── default-test-claim-pvc-feafa663-67e9-49fc-bdc3-0b199de77845
    └── SUCCESS

默认目录路径为：${namespace}-${pvcName}-${pvName}

 

7、Storage Class 相关参数

NFS subdir external provisioner 的 Storage Class 相关参数：

参数 描述 备注
onDelete 如果存在且值为delete，则删除该目录；如果存在值为retain，则保存该目录。 将在共享上以名称归档：archived-
archiveOnDelete 默认为true，即删除时会保存目录。如果存在且值为false，则删除该目录。如果onDelete存在，archiveOnDelete将被忽略。 将在共享上以名称归档：archived-
pathPattern 指定用于通过 PVC 元数据（例如标签、注释、名称或命名空间）创建目录路径的模板。要指定元数据，请使用${.PVC.}。示例：如果文件夹应命名为 –，则使用${.PVC.namespace}-${.PVC.name} 作为路径模式。 n/a

示例：

[root@cp nfs-subdir-external-provisioner]# cat class-test.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-test
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:
  pathPattern: "${.PVC.namespace}/${.PVC.annotations.nfs.io/storage-path}"
  onDelete: delete

[root@cp nfs-subdir-external-provisioner]# cat test-pvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
spec:
  storageClassName: nfs-test
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

[root@cp nfs-subdir-external-provisioner]# kubectl apply -f class-test.yaml -f test-pvc.yaml -f test-pod.yaml
[root@cp nfs-subdir-external-provisioner]# tree /nfs
/nfs
└── default
    └── SUCCESS

 

参考文章：

https://kubernetes.io/docs/
https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner

发表回复

验证码： + 27 = 33