A quick reference study guide to use for troubleshooting OpenStack. If you’d like to learn more, and practice hands-on exercises to prepare you for the exam, check out our online OpenStack training courses.
* Please note that some commands listed in this guide are for use in CentOS and RedHat only. As such, service names may change in other operating systems such as Ubuntu.
Keystone Identity System
Provides API client authentication, service discovery, and distributed multi-tenant authorization. Administrator level access that bypasses users, tenants, roles, services and endpoints.
Keystone logs: /var/log/keystone and /var/log/httpd/keystone_wsgi_*.log (if Keystone is running as an Apache wsgi application)
Show detailed output: set debug=True in /etc/keystone/keystone.conf
Nova Compute Service
Massively scalable, on demand, self service access to compute resources. Nova has multiple components: api, cert, conductor, consoleauth, scheduler, novncproxy.
- Components log:
- Instance specification: /var/lib/nova/instances/INSTANCE_UUID/libvirt.xml
- Instance dmesg output: /var/lib/nova/instances/INSTANCE_UUID/console.log
- Libvirt instance log: /var/log/libvirt/qemu/instance-INSTANCE_UUID.log
- Show detailed output: set debug=True in /etc/nova/nova.conf
- See state of nova components in the control and compute planes: nova service-list (as user with admin role)
- Verify nova can correctly proxy image API calls to glance: nova image-list
- Verify nova can correctly proxy network API calls to neutron: nova secgroup-list or nova net-list
- Verify nova can correctly proxy storage API calls to cinder: nova volume-list
- Check VM configuration and status with libvirt tools: virsh
Provides network connectivity as a service between interface devices.
- Neutron-server logs: /var/log/neutron/server.log
- Neutron plugins & agents logs: /var/log/neutron/
- Show detailed output: set debug=True in /etc/neutron/neutron.conf
- See state of neutron agents running in the network and compute plane: run neutron agent-list (as user with admin role)
- Show detailed output for Neutron Plugins & Agent logs: set debug=True in the appropriate agent or plugin INI file
- Check network configuration and ip namespace: ip -o addr
- Check ip namespace: ip netns
- Check OpenVSwitch bridges and ports: ovs-vsctl show
- Check OpenVSwitch datapath and packets: ovs-dpctl
- Packet analysis: tcpdump
Glance Image Service
Glance provides discovery, registration and delivery services for disk and server images. Glance has three components: api, glare and registry.
- Glance-api logs: /var/log/glance/api.log
- Glance-glare logs: /var/log/glance/glare.log
- Glance-registry logs: /var/log/glance/registry.log
- Show detailed output: set debug=True in:
‣ Glance-api logs: /var/log/glance/api.log
‣ Glance-glare logs: /var/log/glance/glare.log
‣ Glance-registry logs: /var/log/glance/registry.log
Cinder Block Storage
Cinder is designed to present storage resources to end users that can be consumed by the OpenStack Compute Project (Nova). Cinder has four components: api, backup, scheduler and volume.
- Cinder-api logs: /var/log/cinder/cinder-api.log
- Cinder-backup logs: /var/log/cinder/cinder-backup.log
- Cinder-scheduler logs: /var/log/cinder/cinder-scheduler.log
- Cinder-volume logs: /var/log/cinder/cinder-volume.log
- Show detailed output: set debug=True in /etc/cinder/cinder.conf
- View state of cinder components in control and storage planes: cinder service-list (as user with admin role)
- LVM backend, use LVM commands to check volumes: lvs
Telemetry Data Collection
Telemetry is a project designed to reliably collect data on the utilization of the physical and virtual resources comprising deployed clouds, persist these data for subsequent retrieval and analysis, and trigger actions when defined criteria are met. It consists of four projects: Aodh, Ceilometer, Gnocchi and Panko.
- Aodh log: /var/log/aodh/*.log
- Ceilometer log: /var/log/ceilometer/*.log
- Gnocchi log: /var/log/gnocchi/*.log
- Panko log: /var/log/panko/*.log
- Show detailed output
‣ set debug=True in /etc/aodh/aodh.conf
‣ set debug=True in /etc/aodh/ceilometer.conf
‣ set debug=True in /etc/aodh/gnocchi.conf
‣ set debug=True in /etc/aodh/panko.conf
- Check if processes are running:
‣ systemctl status openstack-aodh-*
‣ systemctl status openstack-ceilometer-*
‣ systemctl status openstack-gnocchi-*
‣ systemctl status openstack-panko-*
Note that their api services can run as Apache wsgi applications. In that case you need to check if apache is running.
- List metrics: openstack metric metric list
Swift Object Store
Cloud storage software to store and retrieve lots of data with a simple API. Ideal for storing unstructured data that can grow without bound.
• Swift log: /var/log/swift/*.log
- List objects: swift list
- Create or download objects: swift upload/download
- Show detailed output: change log_level in /etc/swift/*-server.conf and restart the corresponding service
- Get swift object info: swift-object-info
An orchestration engine to launch multiple composite cloud applications based on templates in the form of text files that can be treated like code. Heat has two main components heat-api and heat-engine.
- Heat-api log: /var/log/heat/heat-api.log
- Heat-engine log: /var/log/heat/heat-engine.log
- List heat stacks: openstack stack list
- Show details of a stack: openstack stack show STACK_UUID
- List heat services running: openstack orchestration service list
- Check if heat services are running: systemctl status openstack-heat-*