In this guide I use my previous Ceph cluster to show how to shutdown Ceph cluster properly.

Let’s begin.
The following guide is applied only to cephcluster1 node.
1. Make sure Ceph health status is in OK state.
ceph -s

2. Un-mount Ceph file system (if mounted).
umount /data
3. Set Ceph OSD flags
ceph osd set noout
ceph osd set nobackfill
ceph osd set norecover
ceph osd set norebalance
ceph osd set nodown
ceph osd set pause
then check Ceph health status (WARN state is expected).
ceph -s

4. Shutdown Ceph services.
systemctl stop ceph-mds.target
systemctl stop ceph-mgr.target
systemctl stop ceph-osd.target
systemctl stop ceph-mon.target
systemctl stop ceph.target

The following guide is applied to other nodes (cephcluster2 and cephcluster3).
5. Un-mount Ceph file system (if mounted).
umount /data
6. Shutdown Ceph services.
systemctl stop ceph-mds.target
systemctl stop ceph-mgr.target
systemctl stop ceph-osd.target
systemctl stop ceph-mon.target
systemctl stop ceph.target

The following guide is applied to all nodes (cephcluster1, cephcluster2, and cephcluster3).
7. Virtual Machines are now safe to power off.
poweroff