General guidelines for troubleshooting
- Contact OpenEBS Community for support.
- Search for similar issues added in this troubleshootiung section.
- Search for any reported issues on StackOverflow under OpenEBS tag
One of the cStorVolumeReplica(CVR) will have its status as
Invalid after corresponding pool pod gets recreated
When User delete a cStor pool pod, there are high chances for that corresponding pool-related CVR's can goes into
Following is a sample output of
kubectl get cvr -n openebs
Sample logs of
cstor-pool-mgmt when issue happens:
From the above highlighted logs, we can confirm
cstor-pool-mgmt in new pod is communicating with
cstor-pool in old pod as first highlighted log says
cstor pool found then next highlighted one says pool is really
When a cstor pool pod is deleted there are high chances that two cstor pool pods of same pool can present i.e old pool pod will be in
Terminating state(which means not all the containers completely terminated) and new pool pod will be in
Running state(might be few containers are in running state but not all). In this scenario
cstor-pool-mgmt container in new pool pod is communicating with
cstor-pool in old pool pod. This can cause CVR resource to set to
Note: This issue has observed in all OpenEBS versions upto 1.2.
Phase of cStorVolumeReplica (cvr) from
Offline. After few seconds CVR will be
Degraded state depends on rebuilding progress.
cStor volume become read only state
Application mount point running on cStor volume went into read only state.
Offline or corresponding target pod is unavailable for more than 120 seconds(iSCSI timeout) then the PV will be mounted as
read-only filesystem. For understanding different states of cStor volume, more details can be found here.
Check the status of corresponding cStor volume using the following command:
kubectl get cstorvolume -n <openebs_installed_namespace> -l openebs.io/persistent-volume=<PV_NAME>
If cStor volume exists in
Degraded state then restarting of the application pod alone will bring back cStor volume to
RW mode. If cStor volume exists in
Offline, reach out to OpenEBS Community for assistance.