Ceph Nautilus was released earlier in the year and it has many new features.
In CentOS, the Ceph Nautilus packages depend on CentOS 7.6 packages. Since we are using local YUM repo mirrors, we needed to download all CentOS 7.6 RPMs and Ceph Nautilus RPMs to our local YUM repo servers, and then update yum configs in all Ceph nodes to CentOS 7.6 and Ceph Nautilus.
Next we just followed the instructions as per the official Ceph documentation. The process is as follows:
- yum update ceph mon packages
- restart ceph mon service one by one
Upgrade Ceph Mgrs:
- yum update ceph mgr packages
- restart ceph mgr service one by one
Upgrade Ceph OSDs:
- yum update ceph osd packages
- restart ceph osd service one by one
Upgrade Ceph MDSs:
- reduce mds ranks max_mds to 1
- stop all standby mds services
- restart the remaining active mds service
- start all other mds services
- restore original value of max_mds
Upgrade Ceph RADOSGW:
- update and restart radosgw
Update CRUSH buckets:
- switch any existing CRUSH buckets to straw2
ceph osd getcrushmap -o backup-crushmapceph osd crush set-all-straw-buckets-to-straw2
- enable v2 network protocol using “ceph mon enable-msgr2”
Configure the Ceph Dashboard:
Ceph’s dashboard has been changed to enable SSL by default, so it will not work without certificates. You will need to either create certificates, or disable SSL. Since our Ceph is running in an internal network, we disabled SSL using the following command:
ceph config set mgr mgr/dashboard/ssl false
For more details about creating certificates, see the dashboard documentation.
We also enabled the Prometheus plugin in Ceph Mgr to collect metrics. In order to enable the plugin, simply run this command:
ceph mgr module enable prometheus
Then configure the scrape targets and some rules in Prometheus to scrape data from Ceph. More details can be found here.
After the scraping is working, you can use Grafana to visualise the metrics. Grafana Dashboards can be installed from RPMs or downloaded from its github repository.
For more information, check out the official Ceph documentation, as it describes how to upgrade from Luminous in great detail. If you’d like to learn more, we have Ceph training available, or ask our Solutionauts for some help.
Straightforward, but why not use ceph ansible to do the upgrade !
Yes agree. However our case is a bit different. Our setup is a hyper converged deployment; ceph MONs are running in OpenStack controller hosts, ceph OSDs are running in OpenStack compute hosts. During the upgrade we wanted to separate MONs from the controllers and put them into dedicated hosts. In fact, manual upgrade turned out to be pretty easy. -Shunde