Skip to content

Monitoring rally scenarios

Jump to bottom
Peter Razumovsky edited this page Sep 17, 2018 · 5 revisions

NOTE that this documentation made for rally-openstack version < 1.0. Rally OpenStack repo 1.0 contains upstream scenarios for Elasticsearch and Grafana.

Monitoring scenarios consists of 3 scenarios: two for Grafana scenarios and one for Elasticsearch.

You need to create openstack deployment to use it. Official rally-openstack documentation contains detailed guide how to create this deployment.

Grafana scenarios

There are two scenarios for Grafana service:

  • Test monitoring service by pushing metrics from nova instance:

    Scenario tests monitoring system, which uses Pushgateway as metric exporter and Grafana as metrics monitoring.

    The goal of the test is to check that monitoring system works correctly with nova instance. Test case is the following: we deploy some env with nodes on Openstack nova instances, add metric exporter (using Pushgateway in this test) inside nodes (i.e. nova instances) for some interested metrics (e.g. CPU, memory etc.). We want to check that metrics successfully sends to metrics storage (e.g. Prometheus) by requesting Grafana. Create nova instance, add Pushgateway push random metric to userdata and after instance would be available, check Grafana datasource that pushed metric in data.

  • Test monitoring system availability with local pushing random metric.

Check that you have these scenarios in rally plugin list by commands:

rally plugin show Grafana.push_metric_from_instance

rally plugin show Grafana.push_metric_locally

Grafana.push_metric_from_instance arguments:

  • image: image for server with userdata script
  • flavor: flavor for server with userdata script
  • monitor_vip: monitoring system IP to push metric
  • pushgateway_port: Pushgateway port to use for pushing metric
  • grafana: Grafana dict with creds and port to use for checking metric. Format: {user: admin, password: pass, port: 9902}
  • datasource_id: metrics storage datasource ID in Grafana
  • job_name: job name to push metric in it
  • sleep_time: sleep time between checking metrics in seconds
  • retries_total: total number of retries to check metric in Grafana

Use next command to run this task:

rally task start --task samples/scenarios/grafana/push-metric-from-instance.yaml

Grafana.push_metric_locally arguments:

  • monitor_vip: monitoring system IP to push metric
  • pushgateway_port: Pushgateway port to use for pushing metric
  • grafana: Grafana dict with creds and port to use for checking metric. Format: {user: admin, password: pass, port: 9902}
  • datasource_id: metrics storage datasource ID in Grafana
  • job_name: job name to push metric in it
  • sleep_time: sleep time between checking metrics in seconds
  • retries_total: total number of retries to check metric in Grafana

Use next command to run this task:

rally task start --task samples/scenarios/grafana/push-metric-locally.yaml

Elasticsearch scenarios

Elasticsearch service has one scenario, which allows to verify logs pushed from nova instance to elasticsearch indices.

Let OpenStack platform already has logging agent (for example, Filebeat), which sends nova logs to Elasticsearch through data processing pipeline (e.g. Logstash). The test verifies Openstack nova logs stored in logging system. It creates nova instance with random name and after instance becomes available, checks it's name in Elasticsearch indices by querying.

Check that you have this scenario in rally plugin list by command:

rally plugin show ElasticsearchLogging.log_instance

ElasticsearchLogging.log_instance arguments:

  • image: image for nova instance aka server
  • flavor: flavor for nova instance
  • logging_vip: logging system IP to check server name in elasticsearch index
  • elasticsearch_port: elasticsearch port to use for check server
  • sleep_time: sleep time in seconds between elasticsearch request
  • retries_total: total number of retries to check server name in elasticsearch

Use next command to run this task:

rally task start --task samples/scenarios/elasticsearch/log-instance.yaml

Clone this wiki locally