Cloud Hub Overview

Content

This page references the GroundWork Cloud Hub for VMware, OpenStack and Red Hat.

1.0 Overview and Theory of Operation

In order to create advanced functionality for simultaneous monitoring of multiple and heterogeneous virtual server environments subject to fairly rapid changes in configuration, GroundWork created GroundWork Cloud Hub.

1.1 GroundWork "Hubs"

A GroundWork hub is a self-contained monitoring subsystem that performs auto discovery, monitoring configuration by an operator, synchronization between monitoring configuration and target system configurations, and monitoring of multiple and different target environments using only the GroundWork API as its data interface to the main GroundWork Monitor system. Hubs rely upon GroundWork central monitoring services for role based access control, event management, notifications, performance graphing and analysis, reporting, dashboards, status maps, and run time user interfaces.

The first hub GroundWork developed was the Cloud Hub, the second is Log Hub and we have plans for other types for software controlled networking and storage management.

1.2 Monitoring Architecture and the GroundWork Cloud Hub

The Cloud Hub is a complete Java application that is pre-installed on the GroundWork server, and can also be installed on free standing VMs and bare metal machines provided that they have a Java environment installed. Cloud Hub is deployed in either a centralized, distributed, or mixed architecture.

1.2.1 Centralized Mode

In the centralized mode, the single pre-installed Cloud Hub can be interfaced to multiple server virtualization managers as in the case where a customer has more than one Virtualization Management system and also in the case of virtualization managers of different vendors such as VCenter® Server from VMWare®, RHEV-M® from Red Hat®, or the OpenStack© framework.

In addition to API based monitoring of vendor and open source virtualization managers, Cloud Hub can also monitor hypervisors directly if virtualization managers aren’t available. Centralized deployment for Cloud Hub is illustrated in the following figure.

Figure: Centralized Deployment

1.2.2 Decentralized Mode

In decentralized Cloud Hub deployment multiple Cloud Hubs are used to monitor a single virtualization API data source. Mixed deployments that are a combination of centralized and decentralized architectures can also be created. Decentralized deployment of Cloud Hubs is illustrated in the following figure.

Figure: Decentralized Deployment

1.3 Functional Description of GroundWork Cloud Hub

GroundWork Cloud Hub performs the following functions:

• Based on operator input, Cloud Hub connects to the port containing the Virtualization Manager’s API and auto discovers the inventory and configuration relationships of hypervisors, virtual machines (VMs), network domains, storage domains, and resource pools etc.
• Cloud Hub automatically adds the discovered hosts, and Configuration Items to the GroundWork online database using the following GroundWork virtualization data model:
  • Each hypervisor shall be created as both a host and a host group
  • The Hypervisor profile will be applied to the Host representing the monitoring metrics for the hypervisor
  • All hypervisors shall be automatically added to a host group containing all hypervisors
  • All VMs currently running on each hypervisor will be added to the host group for that hypervisor
  • All VMs sharing a network domain, resource pool, or storage domain shall be collected into host groups associated with each of these entities
  • The VM profile will be applied to all virtual hosts discovered by the API

1.3.1 GroundWork Virtualization Model

The GroundWork Virtualization Data Model is made visible and actionable in the Tree View portlet that is used in both the Status and Event Console pages of the GroundWork Monitor user interface. A simple example is shown in the figure below.

• Automatically commences monitoring of all of the discovered VMs and hypervisors that have been discovered. Monitoring includes collection of each monitoring check result and its defined threshold and for those so configured storing each value for presentation in graphical format.
• For each metric a performance graph can be created depending on the setting in the profiles
• Automatically updates the configuration relationships when the virtualization manager moves VMs between hypervisors or changes resource assignments without requiring operator action.
• Cloud Hub provides a web based user interface that permits the system administrator to customize the monitoring profile that is used. Changes that are supported in this way include:
  • Adding or subtracting items to be monitored from the full list of several thousand parameters that are exposed by the APIs.
  • Assigning short indicative alias names to replace long form strings
  • Setting threshold values
  • Selecting synthetic measurements so that thresholds can be set in percentage of 100% to make threshold setting
  • Choosing whether the parameter is to be graphed or not

Figure: GroundWork Virtualization Data Model

1.4 Technical Description of the GroundWork Cloud Hub

The internal modules and interfaces between the GroundWork Cloud Hub and the virtualization managers and hypervisors that it monitors are illustrated in the diagram below.

The functional description of each internal module and interface of the GroundWork Cloud Hub is as follows:

• Virtual Environment Monitoring API will receive and normalize data collected from different connectors before it is sent to the GroundWork Connector.
• Connectors are used to connect the Cloud Hub to the following virtual systems:
  • Red Hat KVM or Red Hat RHEM-M data sources using the oVirt API
  • VMWare Connector is used to connect the Cloud Hub to vSphere API interfaces on VMWare hypervisors virtualization managers.
  • OpenStack Connector interacts with the Open Stack API’s for compute (Nova), Network (Neutron) or storage (Cinder).
• Configuration U/I is a web application accessed from the GroundWork server that performs the configuration functions described previously.
• Synchronization and Monitoring Module “listens,” based on the contents of the Monitoring Profile and Hub preferences file, to the data stream coming from the Virtual Environment Monitoring API (VEMA) and sends both inventory updates (i.e. synchronization messages) and monitoring metrics to the GroundWork connector to be transmitted to the GroundWork central server.
• GroundWork Connector normalizes the inventory and monitoring data before sending it to the GroundWork server using the secure REST API. This module generates alerts when it detects that the value of a metric exceeds its threshold and generates performance data calls. Performance Data and Alerts messages are sent over the secure REST API to the GroundWork server.

Agent configuration properties files contain the following:

• Configuration for GroundWork API endpoint
• Configuration for virtualization API's
• Check interval
• Enabling Monitoring of Resource Pools, Network, and Storage devices

Monitoring profiles contain the following:

• Metrics and threshold to be monitored for hypervisors and VM's
• Enabling /disabling automatic performance graph creation

The results of auto discovery, manual configuration, synchronization, and monitoring with Cloud Hub flows into the GroundWork Monitoring system where it is written directly to the central database through the REST API. The data is used as follows:

• Status information (Up/Down, OK, Warning, Critical) and the hierarchical relationships between all of the levels of the virtual server stack are displayed in the tree view portlet on each page of the Status Viewer and Event Console.
• Collection of all of the services for all of the layers of virtual server stacks depends upon the use of single consistent host names throughout the environment.
• Events coming from Cloud Hub are assigned an application type of VEMA for VMware, CHRHEV for RHEV-M and OS for Open Stack that allows them to be selected or excluded from public, system, or personal filters of the GroundWork Event Console.
• Alerts are created within the Cloud Hub feeder and made available to the Notification and Escalation subsystem by the RESTful API that also writes the associated events to the GroundWork database.
• The notification and escalation subsystem will communicate alerts to designated contacts in accordance with its configuration.
• Performance data is written both to fixed length files, (RRDs) and to the production data base where it is summarized for statistical analysis.

Figure: Cloud Hub Internal Data Flow

2.0 Description of Operation

The following provides detailed instructions for operating Cloud Hub in installation, configuration, and run time modes.

2.1 Install Mode

Cloud Hub is pre-installed in GroundWork Monitor Enterprise that is, in turn, installed by the GroundWork Installer. The cloud Hub is available to Administrators under the GroundWork Administrator menu in the Portal.

Distributed Cloud Hubs are available for installation as stand alone WAR files and can be added to any standalone Application Servers (tomcat, JBoss Server) running Java 7 SDK.

2.2 Configuration Mode

The initial configuration of the Cloud Hub Agent is described in detail in the document [DOC70:Cloud Hub Connectors]. The documentation describes the configuration of the different connector and will be extended when new connector become available.

2.3 Operation Mode

During normal operation, the Cloud Hub keeps itself in synchronization with the virtualized server inventory and delivers monitoring data for hypervisors, vm containers, and network, storage, and other resource pools. Alerts are generated within the Cloud Hub when it is detected that the value of a metric exceeds its threshold. Such alerts are passed to the GroundWork Event Console for further analysis and processing by NOC operators and are also passed to the Notification and Escalation subsystem that applies its rules to notify those contacts that are scheduled to receive the alert at a particular point in time.