k8s nfs storageClass redis集群： nfs无法动态挂载

Question

报错现象：
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/56814578-46dc-4b35-8f03-c69db55cd627/volumes/kubernetes.io~nfs/pvc-1761dc38-2565-4268-82a0-cf60bb6a5865 --scope – mount -t nfs 192.168.5.250:/home/data/redis-cluster-redis-data-redis-0-pvc-1761dc38-2565-4268-82a0-cf60bb6a5865 /var/lib/kubelet/pods/56814578-46dc-4b35-8f03-c69db55cd627/volumes/kubernetes.io~nfs/pvc-1761dc38-2565-4268-82a0-cf60bb6a5865
Output: Running scope as unit run-60393.scope.
mount.nfs: mounting 192.168.5.250:/home/data/redis-cluster-redis-data-redis-0-pvc-1761dc38-2565-4268-82a0-cf60bb6a5865 failed, reason given by server: No such file or directory
Warning FailedMount 21s kubelet MountVolume.SetUp failed for volume “pvc-1761dc38-2565-4268-82a0-cf60bb6a5865” : mount failed: exit status 32
Mounting command: systemd-run

配置文件：
nfs 共享目录
[root@master redis]# showmount -e 192.168.5.250
Export list for 192.168.5.250:
/home/data *

权限和用户配置
apiVersion: v1
kind: ServiceAccount
metadata:
name: redis-nfs-client
namespace: redis-cluster

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-runner
namespace: redis-cluster
rules:

• apiGroups: [""]
  resources: [“persistentvolumes”]
  verbs: [“get”,“list”,“watch”,“create”,“delete”]
• apiGroups: [""]
  resources: [“persistentvolumeclaims”]
  verbs: [“get”,“list”,“watch”,“create”,“delete”]
• apiGroups: [“storage.k8s.io”]
  resources: [“storageclasses”]
  verbs: [“get”,“list”,“watch”]
• apiGroups: [""]
  resources: [“events”]
  verbs: [“get”,“list”,“watch”,“create”,“update”,“patch”]
• apiGroups: [""]
  resources: [“endpoints”]
  verbs: [“create”,“delete”,“get”,“list”,“watch”,“patch”,“update”]

---

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-provisioner
namespace: redis-cluster
subjects:

• kind: ServiceAccount
  name: redis-nfs-client
  namespace: redis-cluster
  roleRef:
  kind: ClusterRole
  name: nfs-client-runner
  apiGroup: rbac.authorization.k8s.io

nfs-client配置:
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis-nfs-client
labels:
app: redis-nfs-client
namespace: redis-cluster
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: redis-nfs-client
template:
metadata:
labels:
app: redis-nfs-client
spec:
serviceAccountName: redis-nfs-client
containers:
- name: redis-nfs-client
image: quay.io/external_storage/nfs-client-provisioner:latest
volumeMounts:
- name: redis-nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME ## 这个名字必须与storegeclass里面的名字一致
value: my-nfs-storage
- name: NFS_SERVER
value: 192.168.5.250 #修改为自己的ip（部署nfs的机器ip）
- name: NFS_PATH
value: /home/data #修改为自己的nfs安装目录
volumes:
- name: redis-nfs-client-root
nfs:
server: 192.168.5.250 #修改为自己的ip（部署nfs的机器ip）
path: /home/data #修改为自己的nfs安装目录

storageClass配置
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: redis-nfs-storage
namespace: redis-cluster
provisioner: my-nfs-storage
reclaimPolicy: Retain
parameters:
archiveOnDelete: false

StatefulSet配置
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis
namespace: redis-cluster
spec:
selector: #标签选择器
matchLabels:
app: redis
serviceName: redis-servie #无头服务
replicas: 2
template: #给 定义标签
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis
command:
- "redis-servie"
args:
- “/etc/redis/redis.conf”
- “–protected-mode”
- "no"
resources:
requests:
cpu: 100m
memory: 300Mi
ports:
- containerPort: 80
name: redis
protocol: TCP
- name: cluster
containerPort: 16379
protocol: TCP
volumeMounts:
- name: redis-conf
mountPath: /etc/redis
- name: redis-data
mountPath: /var/lib/redis
- name: localtime
mountPath: /etc/localtime
volumes:
- name: redis-conf
configMap:
name: redis-conf
items: #更改挂载名称
- key: redis.conf
path: redis.conf
- name: localtime
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
type: File
volumeClaimTemplates:

• metadata:
  name: redis-data
  annotations:
  volume.beta.kubernetes.io/storage-class: redis-nfs-storage
  spec:
  accessModes: [ “ReadWriteOnce” ]
  resources:
  requests:
  storage: 1Gi

Answer 1

清风 2023-10-27 15:38:50

看下pod启动的错误日志

• 提问者 慕瓜1278415 #1

  [root@master home]# kubectl get pod -n redis-cluster
  NAME                               READY   STATUS              RESTARTS   AGE
  redis-0                            0/1     ContainerCreating   0          2d
  redis-nfs-client-8cc644869-mqwtk   1/1     Running             0          2d15h

• 提问者 慕瓜1278415 #2

  [root@master home]# kubectl describe pod -n redis-cluster
  Events:
    Type     Reason       Age                    From     Message
    ----     ------       ----                   ----     -------
    Warning  FailedMount  48m (x140 over 2d)     kubelet  Unable to attach or mount volumes: unmounted volumes=[redis-data], unattached volumes=[default-token-s6vdl redis-conf redis-data localtime]: timed out waiting for the condition
    Warning  FailedMount  14m (x422 over 2d)     kubelet  Unable to attach or mount volumes: unmounted volumes=[redis-data], unattached volumes=[redis-conf redis-data localtime default-token-s6vdl]: timed out waiting for the condition
    Warning  FailedMount  8m43s (x1947 over 2d)  kubelet  (combined from similar events): MountVolume.SetUp failed for volume "pvc-1761dc38-2565-4268-82a0-cf60bb6a5865" : mount failed: exit status 32
  Mounting command: systemd-run

• 提问者 慕瓜1278415 #3

  Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/56814578-46dc-4b35-8f03-c69db55cd627/volumes/kubernetes.io~nfs/pvc-1761dc38-2565-4268-82a0-cf60bb6a5865 --scope -- mount -t nfs 192.168.5.250:/home/data/redis-cluster-redis-data-redis-0-pvc-1761dc38-2565-4268-82a0-cf60bb6a5865 /var/lib/kubelet/pods/56814578-46dc-4b35-8f03-c69db55cd627/volumes/kubernetes.io~nfs/pvc-1761dc38-2565-4268-82a0-cf60bb6a5865
  Output: Running scope as unit run-121341.scope.
  mount.nfs: mounting 192.168.5.250:/home/data/redis-cluster-redis-data-redis-0-pvc-1761dc38-2565-4268-82a0-cf60bb6a5865 failed, reason given by server: No such file or directory
    Warning  FailedMount  2m44s (x123 over 47h)  kubelet  Unable to attach or mount volumes: unmounted volumes=[redis-data], unattached volumes=[redis-data localtime default-token-s6vdl redis-conf]: timed out waiting for the condition

点开查看后面2条评论