General guidelines for troubleshooting
- Contact OpenEBS Community for support.
- Search for similar issues added in this troubleshootiung section.
- Search for any reported issues on StackOverflow under OpenEBS tag
Created a PVC using localpv-device / localpv-hostpath storage class. But the PV is not created and PVC in Pending state. PVC in Pending state
The default localpv storage classes from openebs have
volumeBindingMode: WaitForFirstConsumer. This means that only when the application pod that uses the PVC is scheduled to a node, the provisioner will receive the volume provision request and will create the volume.
Resolution: Deploy an application that uses the PVC and the PV will be created and application will start using the PV
Application pod that uses localpv device is stuck in Application pod using LocalPV not coming into running state
Pendingstate with error
Warning FailedScheduling 7m24s (x2 over 7m24s) default-scheduler persistentvolumeclaim "<pvc-name>" not found
Troubleshooting: Check if there is a blockdevice present on the node (to which the application pod was scheduled,) which mathces the capacity requirements of the PVC.
kubectl bd -n openebs -o wide
If matching blockdevices are not present, then the PVC will never get Bound.
Resolution: Schedule the application pod to a node which has a matching blockdevice available on it.
Stale BDC in pending state after PVC is deleted
kubectl bdc -n openebs
Pending BDCs created by localpv provisioner, even after the corresponding PVC has been deleted.
Resolution: LocalPV provisioner currently does not delete BDCs in Pending state if the corresponding PVCs are deleted. To remove the stale BDC entries,
- Edit the BDC and remove the
kubectl edit bdc <bdc-name> -n openebs
- Delete the BDC
kubectl delete bdc <bdc-name> -n openebs
The BDC created by localpv provisioner (bdc-pvc-xxxx) remains in pending state and PVC does not get Bound BDC created by localPV in pending state
Troubleshooting: Describe the BDC to check the events recorded on the resource
kubectl describe bdc bdc-pvc-xxxx -n openebs
The following are different types of messages shown when the node on which localpv application pod is scheduled, does not have a blockdevice available.
- No blockdevices found
Warning SelectionFailed 14m (x25 over 16m) blockdeviceclaim-operator no blockdevices found
It means that there were no matching blockdevices after listing based on the labels. Check if there is any
block-device-tag on the storage class and corresponding tags are available on the blockdevices also
- No devices with matching criteria
Warning SelectionFailed 6m25s (x18 over 11m) blockdeviceclaim-operator no devices found matching the criteria
It means that the there are no devices for claiming after filtering based on filesystem type and node name. Make sure the blockdevices on the node
have the correct filesystem as mentioned in the storage class (default is
- No devices with matching resource requirements
Warning SelectionFailed 85s (x74 over 11m) blockdeviceclaim-operator could not find a device with matching resource requirements
It means that there are no devices available on the node with a matching capacity requirement.
To schedule the application pod to a node, which has the blockdevices available, a node selector can be used on the application pod. Here the node with hostname
svc1 has blockdevices available, so a node selector is used to schedule the pod to that node.
apiVersion: v1 kind: Pod metadata: name: pod1 spec: volumes: - name: local-storage persistentVolumeClaim: claimName: pvc1 containers: - name: hello-container image: busybox command: - sh - -c - 'while true; do echo "`date` [`hostname`] Hello from OpenEBS Local PV." >> /mnt/store/greet.txt; sleep $(($RANDOM % 5 + 300)); done' volumeMounts: - mountPath: /mnt/store name: local-storage nodeSelector: kubernetes.io/hostname: svc1