In this guide I use my previous Ceph cluster that was shutdown properly to show how to start Ceph cluster again.

Let’s begin.
The following guide is applied only to cephcluster1 node.
1. Unset Ceph OSD flags
ceph osd unset noout
ceph osd unset nobackfill
ceph osd unset norecover
ceph osd unset norebalance
ceph osd unset nodown
ceph osd unset pause
then check Ceph health status and wait until Ceph health status is in OK state.
ceph -s

2. Mount Ceph file system again
mount -t ceph 172.20.20.101,172.20.20.102,172.20.20.103:/ /data -o name=admin,secretfile=cephclusterfs.secret
and verify the mounted Ceph file system.
df -h

The following guide is applied to other nodes (cephcluster2 and cephcluster3).
3. Mount Ceph file system again
mount -t ceph 172.20.20.101,172.20.20.102,172.20.20.103:/ /data -o name=admin,secretfile=cephclusterfs.secret
and verify the mounted Ceph file system.
df -h

4. Ceph cluster is ready to serve again.