E-mail
Overview
This page provides an overview of GroundWork's Hub architecture, and functional and technical operations. GroundWork Cloud Hub currently supports the virtualization environments; Amazon AWS, Cloudera, Docker, Icinga 2, NetApp, OpenStack, Red Hat Enterprise Virtualization (RHEV), and VMware vSphere. For Net Hub there is currently the OpenDaylight ODL connector. Cloud Hub and Net Hub are available to administrators under the GroundWork Administrator option in the portal.
CONTENTS | RELATED RESOURCES |
WAS THIS PAGE HELPFUL? |
1.0 About Hybrid Cloud Monitoring
In order to create advanced functionality for simultaneous monitoring of multiple and heterogeneous virtual server environments subject to fairly rapid changes in configuration, GroundWork created GroundWork Hubs. Hybrid Cloud Monitoring focuses on API to API connectors and GroundWork Hubs talk directly to the API of the management systems to get configuration and metrics into the GroundWork Monitor system.
GroundWork Cloud Hub is a light weight Java based application that can run anywhere that supports Java and doesn't consume a lot of processing time on the host where it's running. It's agentless, so no agents are installed. Whether it's virtualization hubs for compute nodes, network, or storage you define a connection to the API, select the profile metrics, and the information maps into GroundWork Monitor and starts monitoring.
Hub connectors include an automatic setup of availability, performance graphs, events, reports, notifications using the integrated NoMa application, and the use of aggregation functions such as Custom Groups and Business Service Monitoring.
GroundWork Net Hub offers discovery and monitoring for dynamic Software Defined Networks (SDN). GroundWork Net Hub is built off of GroundWork’s popular Cloud Hub code base. And similar to Cloud Hub, GroundWork Net Hub only needs to be configured once. From that point, Net Hub remains in communication with the SDN controller to understand the configuration of the network, when elements have been added or removed and to know their performance characteristics. The benefits of GroundWork Net Hub:
- Monitor OpenDaylight/OpenFlow based SDN controllers
- Native API-based monitoring
- Continuous network discovery of new elements
- Ability to gather topology, port statistics, flow statistics, packet and byte counts
The first version of GroundWork Net Hub will include feeders for Open SDN (OpenDaylight/OpenFlow), with feeders for Cisco ACI and VMware NSX to follow.
Figure: GroundWork Hubs - Agentless Monitoring
2.0 GroundWork Cloud Hub Monitoring Architecture
A GroundWork hub is a self-contained monitoring subsystem that performs continuous configuration discovery, defining of monitoring profiles (by an operator), synchronization between monitoring and target system configurations, and monitoring of multiple and different target environments using only the GroundWork API as its data interface to the main GroundWork Monitor system. Hubs rely upon the GroundWork central monitoring services for role based access control, event management, notifications, performance graphing and analysis, reporting, dashboards, status maps, and run time user interface.
Cloud Hub is a complete Java application that is pre-installed on the GroundWork server (installed by the GroundWork Installer), and can also be installed on free standing virtual machines (VMs) and bare metal machines provided that they have a Java environment installed. Cloud Hub is deployed in either a centralized, distributed, or mixed architecture. Distributed Cloud Hubs are available for installation as stand alone WAR files and can be added to any standalone Application Server (e.g. Tomcat, JBoss) running Java 7 SDK.
During normal operation, Cloud Hub keeps itself in synchronization with the virtualized server inventory and delivers monitoring data for hypervisors, VM containers, network, storage, and other resource pools. Alerts are generated within Cloud Hub when it is detected that the value of a metric exceeds its threshold. Such alerts are passed to the Event Console for further analysis and processing by NOC Operators and are also passed to the notification and escalation subsystem that applies its rules to notify those contacts that are scheduled to receive the alert at a particular point in time.
- In the Centralized Mode, the single pre-installed Cloud Hub can be interfaced to multiple server virtualization managers as in the case where a customer has more than one virtualization management system and also in the case of virtualization managers of different vendors such as VCenter Server from VMware, RHEV-M from Red Hat, or the OpenStack framework. In addition to API based monitoring of vendor and open source virtualization managers, Cloud Hub can also monitor hypervisors directly if virtualization managers are not available.
- In Decentralized Mode, multiple Cloud Hubs are used to monitor a single virtualization API data source. Mixed deployments that are a combination of centralized and decentralized architectures can also be created.
Figure: Cloud Hub Centralized and Decentralized Deployment
3.0 GroundWork Cloud Hub Functional Description
3.1 Cloud Hub Configuration Work Flow
Based on operator input, Cloud Hub connects to the port containing the virtualization manager API and auto discovers the inventory and configuration relationships of hypervisors, virtual machines (VMs), network domains, storage domains, and resource pools, etc.
The GroundWork Cloud Hub provides a web based user interface permitting a system administrator to enter the agent configuration properties file information including the configuration for GroundWork API endpoint, configuration for virtualization API's, a check interval, and enabling monitoring of resource pools, network, and storage devices.
The administrator is also able to customize the selected monitoring profile. Changes supported in this way include adding or subtracting items to be monitored from parameters exposed by the APIs, setting threshold values, selecting synthetic measurements so that thresholds can be set in percentage of 100% to make threshold settings, and choosing whether the parameter is to be graphed or not. See the GroundWork Cloud Hub Monitoring Profiles section below for details on monitoring profiles and file locations.
Figure: Cloud Hub Configuration Work Flow
3.2 Cloud Hub Virtualization Data Model
The GroundWork Virtualization Data Model is made visible and actionable in the Tree View portlet that is used in both the Status and Event Console pages of the GroundWork Monitor user interface.
Cloud Hub automatically adds the discovered hosts, and configuration items to the GroundWork online database using the data model as follows:
- Each hypervisor is created as both a host and a host group
- The hypervisor profile is applied to the host representing the monitoring metrics for the hypervisor
- All hypervisors are automatically added to a host group containing all hypervisors
- All VMs currently running on each hypervisor will be added to the host group for that hypervisor
- All VMs sharing a network domain, resource pool, or storage domain are collected into host groups associated with each of these entities
- The VM profile will be applied to all virtual hosts discovered by the API
Figure: GroundWork Virtualization Data Model - Status
3.3 Unified Monitoring
So how does all this get represented in the unified monitoring context? The data for the monitored services selected are passed to the GroundWork REST API and are directly inserted into the Status and Event Console tables in the GroundWork Foundation database which makes them show up in the UI almost immediately.
In a couple minutes, after adding and starting a hub connection, the Status viewer application will display the automatically created host groups corresponding to the views chosen in setup. You may assign the discovered host groups to Custom Groups to organize the status display.
You will see names of instances or containers prefixed with a specific string (e.g., EBS, EC2) for identification. In the first image below, we show the EC2.CPUCreditBalance service Status Information as CRITICAL, as this is reflective of the current threshold set in the profile. In this view you can also see the graphs coming in under Service Availability and Performance Measurement, and the events being logged at the bottom of the screen.
Figure: Unified Monitoring: Status, Event Console, Dashboards, NoMa
This next image image shows the Event Console, we have selected the system applications filter AWS, which lists events for the AWS application type. From here you can select specific events and apply various actions. The third image displays the Enterprise View in Dashboards and indicates the host Micro1 status as Host Recently Recovered. And the last image shows the NoMa log for notifications in which you can see alerts for the service EC2.CPUCreditBalance.
The monitoring can be adjusted by returning to the Cloud Hub configuration screen and modifying metrics collected (check/un-check) or modifying threshold values.
Figure: Event Console
The third and last images display the Virtualization dashboard and NoMa generated notifications log.
Figure: Dashboards
Figure: NoMa log
4.0 GroundWork Cloud Hub Technical Description
The internal modules and interfaces between GroundWork Cloud Hub and virtualization managers and hypervisors that it monitors are illustrated in the diagram below.
- The Virtual Environment Monitoring API (VEMA) will receive and normalize data collected from different Connectors before it is sent to the GroundWork Connector
- Connectors are used to connect Cloud Hub to the virtual systems, for example:
- VMware Connector is used to connect Cloud Hub to vSphere API interfaces on VMware hypervisors virtualization managers
- OpenStack Connector interacts with the Open Stack API’s for compute (Nova), network (Neutron) or storage (Cinder)
- Red Hat KVM or Red Hat RHEM-M data sources using the oVirt API
- The Configuration user interface is a web application accessed from the GroundWork server that performs the configuration functions described previously
- The Synchronization and Monitoring module listens based on the contents of the Monitoring Profile and Hub Preferences file, to the data stream coming from the VEMA and sends both inventory updates (i.e. synchronization messages) and monitoring metrics to the GroundWork Connector to be transmitted to the GroundWork central server
- The GroundWork Connector normalizes the inventory and monitoring data before sending it to the GroundWork server using the secure REST API. This module generates alerts when it detects that the value of a metric exceeds its threshold and generates performance data calls.
- The results of auto discovery, manual configuration, synchronization, and monitoring with Cloud Hub flows into the GroundWork monitoring system where it is written directly to the central database through the REST API. The data is used as follows and mentioned above under the virtualization data model:
- Status information (Up/Down, OK, Warning, Critical) and the hierarchical relationships between all of the levels of the virtual server stack are displayed in the tree view portlet on each page of the Status viewer and Event Console
- Collection of all of the services for all of the layers of virtual server stacks depends upon the use of single consistent host names throughout the environment
- Events coming from Cloud Hub are assigned an application type of VEMA for VMware, CHRHEV for RHEV-M and OS for Open Stack that allows them to be selected or excluded from public, system, or personal filters in the Event Console
- Alerts are created within the Cloud Hub feeder and made available to the Notification and Escalation subsystem by the RESTful API that also writes the associated events to the GroundWork database.
- The notification and escalation subsystem will communicate alerts to designated contacts in accordance with the configuration
- Performance data is written both to fixed length files (RRDs) and to the production data base where it is summarized for statistical analysis
Figure: Cloud Hub Internal Data Flow
5.0 GroundWork Cloud Hub Monitoring Profiles
The master monitoring profiles for virtual environments contain metrics and thresholds to be monitored for hypervisors and VMs and the enabling/disabling of automatic performance graph creation and are stored on the GroundWork server. Each time an administrator goes into the Cloud Hub configuration screens the monitoring profile(s) from the GroundWork server are loaded into the Cloud Hub. This means you manage and maintain the monitoring profiles for Cloud Hub in a central location. The monitoring profile templates, working copies, and the actual virtual manager connection to be monitored are outlined in this section.
5.1 Monitoring Profile Templates
The Virtual Environment Monitoring API (VEMA) monitoring profiles (templates for instantiating) are stored on the GroundWork server in the /usr/local/groundwork/core/vema/profiles directory:
 | If you wish, you may carefully edit a profile XML file to include additional numeric metrics. If you edit, PLEASE test immediately. Any metric test that is slightly misspelled or otherwise rejected short-circuits ALL the metrics from reporting silently and without raising flags. In general, we can't recommend adding additional numeric metrics, at the time of this writing all useful metrics have been included as part of the released XML file contents. |
[root@demo /]# cd /usr/local/groundwork/core/vema/profiles [root@demo profiles]# ls amazon_monitoring_profile.xml opendaylight_monitoring_profile.xml cloudera_monitoring_profile.xml openstack_monitoring_profile.xml docker_monitoring_profile.xml rhev_monitoring_profile.xml netapp_monitoring_profile.xml vmware_monitoring_profile.xml
Here we show an example of the Amazon monitoring profile template, amazon_monitoring_profile.xml:
[root@demo profiles ]# cat amazon_montoring_profile.xml
<?xml version= "1.0" encoding= "UTF-8" ?>
<vema-monitoring>
<profileType>amazon</profileType>
<hypervisor>
<metric name="EBS.VolumeWriteBytes" description="Total Bytes Written on an Elastic Block Storage device" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="EBS.VolumeReadOps" description="Total Read Operations on an Elastic Block Storage device" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="EBS.VolumeWriteOps" description="Total Write Operations on an Elastic Block Storage device" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="EBS.VolumeTotalWriteTime" description="Total seconds spent by Write operations completed on EBS device" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="EBS.VolumeIdleTime" description="Total seconds spent by in Idle time on EBS device" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="EBS.VolumeQueueLength" description="Total Read and Write Operations on an Elastic Block Storage device" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="EBS.VolumeReadBytes" description="Total Bytes Read on an Elastic Block Storage device" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="EBS.VolumeTotalReadTime" description="Total seconds spent by Read operations completed on EBS device" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="EC2.CPUCreditUsage" description="Number of CPU credits consumed on an EC2 instance" monitored="true" graphed="true" warningThreshold="20" criticalThreshold="50" />
<metric name="EC2.CPUCreditBalance" description="Number of CPU credits accumulated on an EC2 instance" monitored="true" graphed="true" warningThreshold="5" criticalThreshold="15" />
<metric name="EC2.CPUUtilization" description="Percentage of allocated EC2 compute units currently in use on EC2 instance" monitored="true" graphed="true" warningThreshold="75" criticalThreshold="90" />
<metric name="EC2.DiskReadOps" description="Completed read operations from all ephemeral disks available on EC2 instance" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" />
<metric name="EC2.DiskWriteOps" description="Completed write operations to all ephemeral disks available on EC2 instance" monitored="true" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="EC2.DiskReadBytes" description="Bytes read from all ephemeral disks available on EC2 instance" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" />
<metric name="EC2.DiskWriteBytes" description="Bytes written to all ephemeral disks available on EC2 instance" monitored="true" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="EC2.NetworkIn" description="Bytes received on all network interfaces on EC2 instance" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" />
<metric name="EC2.NetworkOut" description="Bytes written to all network interfaces on EC2 instance" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" />
<metric name="RDS.BinLogDiskUsage" description="Disk space occupied by binary logs on the master RDS node" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.DatabaseConnections" description="Number of database connections currently in use on RDS (Relational Database service)" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.DiskQueueDepth" description="Number of outstanding IOs (read/write requests) waiting to access RDS disks" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.FreeableMemory" description="Amount (bytes) of available RAM on RDS service" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.FreeStorageSpace" description="Amount (bytes) of available Storage space on RDS service" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.ReplicaLag" description="Amount of time a Read Replica DB Instance lags behind the source DB Instance" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.SwapUsage" description="Amount (bytes) of swap space used on the DB instance" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.ReadIOPS" description="Average number of disk I/O read operations per second (RDS)" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.WriteIOPS" description="Average number of disk I/O write operations per second (RDS)" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.ReadLatency" description="Average amount of time in seconds taken per disk Read I/O operation (RDS)" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.WriteLatency" description="Average amount of time in seconds taken per disk Write I/O operation (RDS)" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.ReadThroughput" description="Average number of bytes read from disk per second (RDS)" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.WriteThroughput" description="Average number of bytes written to disk per second (RDS)" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.NetworkReceiveThroughput" description="Incoming (Receive) network traffic on the DB instance in bytes/second, includes both database and RDS traffic" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
<metric name="RDS.NetworkTransmitThroughput" description="Outgoing (Transmit) network traffic on the DB instance in bytes/second, includes both database and RDS traffic" monitored="true" graphed="true" warningThreshold="-1" criticalThreshold="-1" sourceType="storage" />
</hypervisor>
<vm>
<metric name="info.ebs.createtime" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ebs.isencrypted" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ebs.iops" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ebs.volumesize" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ebs.volumetype" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.architecture" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.isebsoptimized" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.hypervisor" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.imageid" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.instanceid" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.instancetype" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.detailedmonitoring" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.tenancy" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.platform" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.privatedns" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.privateip" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.publicdns" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.publicip" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.rootdevice" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.srcdestchecking" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.enhancednetworking" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.subnetid" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.ec2.vpcid" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.allocatedstorage" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.autominorupgrade" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.backupretention" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.charset" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.dbinstanceclass" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.dbname" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.endpoint" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.engine" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.engineversion" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.createtime" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.iops" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.latestrestoretime" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.licensemodel" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.masteruser" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.ismultiaz" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.preferredbackup" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.preferredmaintenance" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.ispublic" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.primaryaz" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.replicationmaster" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.storageencrypted" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
<metric name="info.rds.storagetype" description="" monitored="false" graphed="false" warningThreshold="-1" criticalThreshold="-1" />
</vm>
<excludes>
<exclude>EBS.VolumeConsumedReadWriteOps</exclude>
<exclude>EBS.VolumeThroughputPercentage</exclude>
</excludes>
</vema-monitoring>
5.2 Monitoring Profile Working Copies
Once a connection has been created, tested, and saved within the GroundWork Monitor UI, a profile working copy is written to the /usr/local/groundwork/config/cloudhub directory with a unique name:
[root@demo /]# cd /usr/local/groundwork/config/cloudhub [root@demo cloudhub]# ls cloudhub-amazon-1.xml cloudhub-cloudera-3.xml
Here we show an example of the Amazon monitoring profile working copy, cloudhub-amazon-1.xml. As you can see, this configuration file includes the authentication for both the GroundWork server and the virtualization manager.
[root@demo cloudhub]# cat cloudhub-amazon-1.xml<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<vema>
<common>
<virtualSystem>AMAZON</virtualSystem>
<displayName>AWS</displayName>
<agentId>a6425774-e405-4859-bf13-37f4abbb61e4</agentId>
<checkInterval>5</checkInterval>
<syncInterval>2</syncInterval>
<comaInterval>15</comaInterval>
<connectionRetries>-1</connectionRetries>
<hypervisorView>true</hypervisorView>
<canAccessMultipleVersions>false</canAccessMultipleVersions>
<storageView>true</storageView>
<networkView>true</networkView>
<resourcePoolView>false</resourcePoolView>
<customView>true</customView>
<createProfileDisabled>false</createProfileDisabled>
<testConnectionDisabled>false</testConnectionDisabled>
<serverSuspended>false</serverSuspended>
<configurationFile>cloudhub-amazon-1.xml</configurationFile>
<pathToConfigurationFile>/usr/local/groundwork/config/cloudhub/</pathToConfigurationFile>
<uiCheckIntervalMinutes>5</uiCheckIntervalMinutes>
<uiSyncIntervalMinutes>2</uiSyncIntervalMinutes>
<uiComaIntervalMinutes>15</uiComaIntervalMinutes>
<uiConnectionRetries>-1</uiConnectionRetries>
<enableGroupTag>true</enableGroupTag>
<groupTag>GWTraining</groupTag>
<prefixServiceNames>false</prefixServiceNames>
</common>
<gwos>
<gwosVersion>7.1</gwosVersion>
<gwosPort>4913</gwosPort>
<gwosServer>localhost</gwosServer>
<gwosSSLEnabled>false</gwosSSLEnabled>
<wsEndPoint>/foundation-webapp/services</wsEndPoint>
<wsHostName>wshost</wsHostName>
<wsUsername>RESTAPIACCESS</wsUsername>
<wsPassword>7UZZVvnLbuRNk12Yk5H33zeYdWQpnA7j9shir7QfJgwh</wsPassword>
<wsPortNumber>80</wsPortNumber>
<wsHostGroupName>wshostgroup</wsHostGroupName>
<rsEndPoint>/api</rsEndPoint>
<mergeHosts>true</mergeHosts>
</gwos>
<amazon>
<server>us-west-1.amazonaws.com</server>
<username>AKIAJ73VJIYOIIMOCIVA</username>
<password>ip4rAGDtB3aSVFauHZCwLCOFsy+SqRBv88tfoAoxKmgN31rlcmwRN/e9h7SUEnUl</password>
<sslEnabled>true</sslEnabled>
</amazon>
</vema>
5.3 Virtual Manager Connections
Once a virtual manager is connected, the directory /usr/local/groundwork/config/cloudhub/profiles gets a file describing the connection to be monitored:
[root@demo /]# cd /usr/local/groundwork/config/cloudhub/profiles [root@demo profiles]# ls amazon-a6425774-e405-4859-bf13-37f4abbb61e4.xml cloudera-b6ed914a-27a3-43ff-8799-4a55cbe9e186.xml
Here we show an example of the Amazon profile connection to be monitored, amazon-a6425774-e405-4859-bf13-37f4abbb61e4.xml. As you can see, this configuration file includes the choices for elements , thresholds, and optionally Service Names, as selected in the GroundWork Monitor UI. The Service Name option adds the ability to report the polled metrics under a unique name that is set by the administrator.
[root@demo profiles]# cat amazon-a6425774-e405-4859-bf13-37f4abbb61e4.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<vema-monitoring>
<agent>a6425774-e405-4859-bf13-37f4abbb61e4</agent>
<profileType>amazon</profileType>
<hypervisor>
<metric name="EBS.VolumeWriteBytes" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Total Bytes Written on an Elastic Block Storage device"/>
<metric name="EBS.VolumeReadOps" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Total Read Operations on an Elastic Block Storage device"/>
<metric name="EBS.VolumeWriteOps" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Total Write Operations on an Elastic Block Storage device"/>
<metric name="EBS.VolumeTotalWriteTime" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Total seconds spent by Write operations completed on EBS device"/>
<metric name="EBS.VolumeIdleTime" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Total seconds spent by in Idle time on EBS device"/>
<metric name="EBS.VolumeQueueLength" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Total Read and Write Operations on an Elastic Block Storage device"/>
<metric name="EBS.VolumeReadBytes" customName="" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Total Bytes Read on an Elastic Block Storage device"/>
<metric name="EBS.VolumeTotalReadTime" customName="" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Total seconds spent by Read operations completed on EBS device"/>
<metric name="EC2.CPUCreditUsage" customName="" monitored="true" graphed="true" warningThreshold="20.0" criticalThreshold="50.0" description="Number of CPU credits consumed on an EC2 instance"/>
<metric name="EC2.CPUCreditBalance" customName="" monitored="true" graphed="true" warningThreshold="5.0" criticalThreshold="15.0" description="Number of CPU credits accumulated on an EC2 instance"/>
<metric name="EC2.CPUUtilization" customName="" monitored="true" graphed="true" warningThreshold="75.0" criticalThreshold="90.0" description="Percentage of allocated EC2 compute units currently in use on EC2 instance"/>
<metric name="EC2.DiskReadOps" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" description="Completed read operations from all ephemeral disks available on EC2 instance"/>
<metric name="EC2.DiskWriteOps" customName="" monitored="true" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description="Completed write operations to all ephemeral disks available on EC2 instance"/>
<metric name="EC2.DiskReadBytes" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" description="Bytes read from all ephemeral disks available on EC2 instance"/>
<metric name="EC2.DiskWriteBytes" customName="" monitored="true" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description="Bytes written to all ephemeral disks available on EC2 instance"/>
<metric name="EC2.NetworkIn" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" description="Bytes received on all network interfaces on EC2 instance"/>
<metric name="EC2.NetworkOut" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" description="Bytes written to all network interfaces on EC2 instance"/>
<metric name="RDS.BinLogDiskUsage" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Disk space occupied by binary logs on the master RDS node"/>
<metric name="RDS.DatabaseConnections" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Number of database connections currently in use on RDS (Relational Database service)"/>
<metric name="RDS.DiskQueueDepth" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Number of outstanding IOs (read/write requests) waiting to access RDS disks"/>
<metric name="RDS.FreeableMemory" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Amount (bytes) of available RAM on RDS service"/>
<metric name="RDS.FreeStorageSpace" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Amount (bytes) of available Storage space on RDS service"/>
<metric name="RDS.ReplicaLag" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Amount of time a Read Replica DB Instance lags behind the source DB Instance"/>
<metric name="RDS.SwapUsage" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Amount (bytes) of swap space used on the DB instance"/>
<metric name="RDS.ReadIOPS" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Average number of disk I/O read operations per second (RDS)"/>
<metric name="RDS.WriteIOPS" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Average number of disk I/O write operations per second (RDS)"/>
<metric name="RDS.ReadLatency" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Average amount of time in seconds taken per disk Read I/O operation (RDS)"/>
<metric name="RDS.WriteLatency" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Average amount of time in seconds taken per disk Write I/O operation (RDS)"/>
<metric name="RDS.ReadThroughput" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Average number of bytes read from disk per second (RDS)"/>
<metric name="RDS.WriteThroughput" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Average number of bytes written to disk per second (RDS)"/>
<metric name="RDS.NetworkReceiveThroughput" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Incoming (Receive) network traffic on the DB instance in bytes/second, includes both database and RDS traffic"/>
<metric name="RDS.NetworkTransmitThroughput" customName="" monitored="true" graphed="true" warningThreshold="-1.0" criticalThreshold="-1.0" sourceType="storage" description="Outgoing (Transmit) network traffic on the DB instance in bytes/second, includes both database and RDS traffic"/>
</hypervisor>
<vm>
<metric name="info.ebs.createtime" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ebs.isencrypted" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ebs.iops" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ebs.volumesize" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ebs.volumetype" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.architecture" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.isebsoptimized" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.hypervisor" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.imageid" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.instanceid" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.instancetype" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.detailedmonitoring" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.tenancy" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.platform" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.privatedns" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.privateip" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.publicdns" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.publicip" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.rootdevice" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.srcdestchecking" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.enhancednetworking" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.subnetid" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.ec2.vpcid" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.allocatedstorage" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.autominorupgrade" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.backupretention" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.charset" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.dbinstanceclass" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.dbname" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.endpoint" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.engine" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.engineversion" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.createtime" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.iops" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.latestrestoretime" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.licensemodel" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.masteruser" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.ismultiaz" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.preferredbackup" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.preferredmaintenance" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.ispublic" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.primaryaz" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.replicationmaster" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.storageencrypted" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="info.rds.storagetype" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
</vm>
<custom>
<metric name="System/Linux.DiskSpaceUtilization" customName="" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="System/Linux.MemoryUtilization" customName="" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
<metric name="System/Linux.SwapUtilization" customName="" monitored="false" graphed="false" warningThreshold="-1.0" criticalThreshold="-1.0" description=""/>
</custom>
<excludes>
<exclude>EBS.VolumeConsumedReadWriteOps</exclude>
<exclude>EBS.VolumeThroughputPercentage</exclude>
</excludes>
</vema-monitoring>
5.4 Performance Data Processing Configuration Properties
VEMA connection metrics can be optionally graphed. The data files produced by Cloud Hub are found and processed by the performance data daemon at the same time that Nagios performance data is graphed. Even though that daemon is started and managed by Nagios directives, the metrics coming from the Cloud Hub are not in any way dependent on Nagios. The format of the data conforms to the plugin output standard. And if the perfdata.properties file designates that performance data is also to be sent to Foundation then Cloud Hub metrics are saved there as well.
[root@demo \]# cd /usr/local/groundwork/config
Here we show a partial view of the the VEMA performance data properties file perfdata.properties. See Performance Configuration for more information.
[roo@demo \]# cat perfdata.properties
# These files are output by a Foundation REST API consumer, feeding
# off the internal Foundation topic service for performance data.
<service_perfdata_files>
# Virtual Environments Monitoring Agent performance data.
<perfdata_source VEMA>
perfdata_file = "/usr/local/groundwork/core/vema/var/vema-perfdata.dat.being_processed"
seek_file = "/usr/local/groundwork/core/vema/var/vema-perfdata.dat.seek"
</perfdata_source>
# Cloud Hub for Red Hat Virtualization performance data.
<perfdata_source CHRHEV>
perfdata_file = "/usr/local/groundwork/core/vema/var/chrhev-perfdata.dat.being_processed"
seek_file = "/usr/local/groundwork/core/vema/var/chrhev-perfdata.dat.seek"
</perfdata_source>
# Cloud Hub for Open Stack performance data.
<perfdata_source OS>
perfdata_file = "/usr/local/groundwork/core/vema/var/os-perfdata.dat.being_processed"
seek_file = "/usr/local/groundwork/core/vema/var/os-perfdata.dat.seek"
</perfdata_source>
# Cloud Hub for Docker performance data.
<perfdata_source DOCK>
perfdata_file = "/usr/local/groundwork/core/vema/var/dock-perfdata.dat.being_processed"
seek_file = "/usr/local/groundwork/core/vema/var/dock-perfdata.dat.seek"
</perfdata_source>
# Cloud Hub for Open DayLight performance data.
<perfdata_source ODL>
perfdata_file = "/usr/local/groundwork/core/vema/var/odl-perfdata.dat.being_processed"
seek_file = "/usr/local/groundwork/core/vema/var/odl-perfdata.dat.seek"
</perfdata_source>
# Cloud Hub for Amazon EC2 performance data.
<perfdata_source AWS>
perfdata_file = "/usr/local/groundwork/core/vema/var/aws-perfdata.dat.being_processed"
seek_file = "/usr/local/groundwork/core/vema/var/aws-perfdata.dat.seek"
</perfdata_source>
# Cloud Hub for NetApp performance data.
<perfdata_source NETAPP>
perfdata_file = "/usr/local/groundwork/core/vema/var/netapp-perfdata.dat.being_processed"
seek_file = "/usr/local/groundwork/core/vema/var/netapp-perfdata.dat.seek"
</perfdata_source>
# For Cacti feeder metrics
<perfdata_source CACTI>
perfdata_file = "/usr/local/groundwork/core/vema/var/cactifeeder-perfdata.dat.being_processed"
seek_file = "/usr/local/groundwork/core/vema/var/cactifeeder-perfdata.dat.seek"
</perfdata_source>
# My-Application-Type performance data.
# <perfdata_source my_application_type>
# perfdata_file = "/usr/local/groundwork/my_app/var/{my_application_type}-perfdata.dat.being_processed"
# seek_file = "/usr/local/groundwork/my_app/var/{my_application_type}-perfdata.dat.seek"
# </perfdata_source>
</service_perfdata_files>