Black List

WAS THIS PAGE HELPFUL? Leave Feedback

CONTENTS

Overview

The Black List feature allows you to stop Cloud Hub discovered hosts from being included in the system monitoring. Hosts that are blacklisted will not be displayed, monitored, distribute notifications, be counted against the system license limit, and overall be reported on within the system. This can be useful for hosts that are not currently being used but may be in the future. Blacklisted hosts are not seen by the system: once blacklisted the entry for that host is DELETED from devices and host table, and the CloudHub connector logic will NOT try to add any host that matches a blacklist regular expression.

Cloud Hub consults this blacklist for references to hosts that will not be discovered and added to the Foundation database and used against the license limit. This is useful when Cloud Hub discovers devices like templates and test servers in the same location where production devices are found. To start monitoring blacklisted hosts again, you would remove the host from the list.

Blacklisting at present is only effective for hosts discovered by Cloud Hub. Hosts monitored by Smart Cacti Feeder, Nagios, Syslog, SNMPTrap, SCOM Feeder, etc may not be blacklisted (entries made in the Blacklist table will be ignored).

Adding hosts

VIDEO

To add a host to the blacklist, log in as an administrator, select GroundWork Administration > Device Management > Black List. Select the (1) + icon. In the (2) Host Name box enter the exact host name or a regular expression (see below), (3) click Add which places the record in the (4) list of blacklisted hosts. The added host(s) will be removed from the Status viewer.

Figure: New black list record dialog

Adding regular expressions

Regular expressions are patterns used to match character combinations in strings. In JavaScript, regular expressions are also objects. You can select several hosts by using a regex in the Host Name field for the Black List feature. Below we show an example of how to use the Black List feature with a regex. It is recommended to use java/javascript regex patterns instead of using file path wildcard syntax. For additional information regarding regular expressions you may want to visit the following link which is specific to the regex use case in Java 7 we are deploying, we've also listed a link to test regular expressions:

• Java Class Pattern reference
• Test site

Other reference sites exist but the subject is complicated. Perl regular expressions will not work here.

Example:

In this first image we show the Status view for a VMware Cloud Hub connection. You can see the host group NET:VM Network which lists hosts that include the string "ELK" and "elk".

Figure: Status view of host group

Next we use the regular expression (elk-)\w+ where (captures the group character codes), \w matches any word character (alphanumeric and underscore), and + matches one or more of the preceding token.

Figure: RegEx (elk-)\w+

This captures all of the hosts in our example except ELK-GW7-1 and elk_mgmt_ubuntu_14. You can see the remaining hosts by searching for "elk".

Figure: Searching for hosts

Additional examples:

Negative matching

You may find that you have a production infrastructure with wonderfully regular names that are to be monitored. Yet there are many templates, unused instances, test instances, ad hoc clones and other detritus with names made up at random that are not following a pattern. The challenge is how to blacklist the unwanted hosts without slavishly typing in each and every unwanted name, a fool's errand. Here is the answer.

You need a regular expression that identifies the hosts to be blacklisted as a negative, that is blacklist those that are "NOT this pattern". You will identify in ONE regular expression the patterns for ALL the hosts you wish to KEEP; any that do not match at least one of the patterns will be blacklisted. The important point to remember is that this is an inclusive expression that says, if the host Cloud Hub has discovered matches ANY part of the expression it is safe, but if it matches NONE of the patterns it will be blacklisted.

There is a further twist to this approach, that is the hosts seen by a connector are associated with a grouping factor we refer to as the "hypervisor". In VMware the hypervisor is the ESX server where the host instance is located. In Amazon the hypervisor is the Region, for example "us-west-2a". So in this negative blacklist regex you must first identify the regions and the hypervisors to be kept, followed by the regular expressions identifying the host name patterns to be kept. The expression is interpreted from left to right. Leaving out a region results in ALL the hosts in that region being blacklisted.

All the patterns, not just this negative regex but all the others, are applied to every discovered host, by every connector, in every cycle.

Example:

Let's imagine all your production servers are in one region, "us-west-1a", and they are all named with a starting string in the name "PROD-denver" or "PROD-dallas" or "PROD-washington" (with numbers or letters following the prefix). And let's imagine that all the other uninteresting hosts, also in the region "us-west-1a" are not named with any of those prefixes. Let's say they are all prefixed as "TEST" or "QA" or "Template" along with a few outliers "abc123", "Joe-Test", "20180704-large-system", "JD-ubuntu-1604".

For clarity and copying the exact string here it is unformatted. You can have one or more of the strings like "allowedpattern1":

^(?!allowedpattern1|allowedpattern2|allowedpattern3).*$

The example is for matching of first part of each name. A more complex pattern could be used. See reference link above.

It should be clear that you could use this negative match approach to isolate monitoring discovery to a selected set of regions or hypervisors.

It should be clear that the process of identifying a host as one to be blacklisted is a sequential search against all the blacklist rules. The host must not be identified in any of the rules, to survive into the set that is added to and updated in Foundation.

Removing blacklisted records

To start monitoring the blacklisted hosts again, you would first remove the host (or pattern) from the list. As an administrator, select GroundWork Administration > Device Management > Black List. (1) Check all or individual host names to remove from the blacklist. You may also use the Filter Hosts box to search for specific hosts to manage. Click the Trashcan icon and click (2) Delete.

Figure: Check host to delete black list record

Timing of the changes and debugging hints

As a rule it can take 0-5 min to refresh the regex cache in Cloud Hub where the Black Lists are kept in real time (whatever set of specifications you added to the Black List page will be read in by Cloud Hub and cached for faster access). Then, it can take 0-X minutes to refresh the connector on top of that where X is the refresh interval of the individual connector. Total maximum wait time is X+5 minutes.

Should you run in to trouble getting the result you expected you might like to take these steps to obtain logging feedback. Edit the file /usr/local/groundwork/jpp/standalone/configuration/standalone.xml and change the entry:

            <logger category="org.groundwork.cloudhub">
                <level name="ERROR"/>
            </logger>

to read this instead:

            <logger category="org.groundwork.cloudhub">
                <level name="INFO"/>
            </logger>

Then restart GroundWork services with the following command:

/etc/init.d/groundwork restart gwservices

After this, tail the file framework log looking for cloudhub events, for example:

tail -f /usr/local/groundwork/jpp/standalone/log/framework.log | grep org.groundwork.cloudhub

You should see messages like this:

2018-07-21 17:12:28,079 INFO  [org.groundwork.cloudhub.monitor.CloudhubMonitorAgentClient] (Thread-144) Cloudhub Start the Monitor Process for agent cloudhub-amazon-1.xml
2018-07-21 17:12:44,781 INFO  [org.groundwork.cloudhub.monitor.CloudhubMonitorAgentClient] (Thread-144) Host black listed: us-west-2b
2018-07-21 17:12:44,781 INFO  [org.groundwork.cloudhub.monitor.CloudhubMonitorAgentClient] (Thread-144) Host black listed: us-west-2a
2018-07-21 17:12:44,880 INFO  [org.groundwork.cloudhub.monitor.CloudhubMonitorAgentClient] (Thread-144) Synchronize DataCenter execution time: 99 ms
2018-07-21 17:12:44,880 INFO  [org.groundwork.cloudhub.monitor.CloudhubMonitorAgentClient] (Thread-144) Gathering Hosts+VMs for agent cloudhub-amazon-1.xml
2018-07-21 17:12:44,963 INFO  [org.groundwork.cloudhub.monitor.CloudhubMonitorAgentClient] (Thread-144) Time to execute sync operation [63] ms  (hosts & VMs: 0) for agent cloudhub-amazon-1.xml
2018-07-21 17:12:44,984 INFO  [org.groundwork.cloudhub.gwos.GwosServiceBizImpl] (Thread-144) ### cloudhub-amazon-1.xml: mod hyp: storing 0 hosts with 0 services
2018-07-21 17:12:44,984 INFO  [org.groundwork.cloudhub.gwos.GwosServiceBizImpl] (Thread-144) ### cloudhub-amazon-1.xml: done mod hypervisors
2018-07-21 17:12:44,984 INFO  [org.groundwork.cloudhub.gwos.GwosServiceBizImpl] (Thread-144) ### cloudhub-amazon-1.xml: mod vm: storing 0 hosts with 0 services
2018-07-21 17:12:44,984 INFO  [org.groundwork.cloudhub.gwos.GwosServiceBizImpl] (Thread-144) ### cloudhub-amazon-1.xml: done mod vm
2018-07-21 17:12:44,985 INFO  [org.groundwork.cloudhub.monitor.ConnectorMonitor] (Thread-144) Updating Groundwork Connector (host-state: , service-state: )...
2018-07-21 17:12:44,985 INFO  [org.groundwork.cloudhub.gwos.GwosServiceBizImpl] (Thread-144) ### cloudhub-amazon-1.xml: mod hyp: storing 1 hosts with 1 services
2018-07-21 17:12:45,031 INFO  [org.groundwork.cloudhub.gwos.GwosServiceBizImpl] (Thread-144) ### cloudhub-amazon-1.xml: done mod hypervisors
2018-07-21 17:12:45,120 INFO  [org.groundwork.cloudhub.monitor.ConnectorMonitor] (Thread-144) ...Groundwork Connector updated
2018-07-21 17:12:45,121 INFO  [org.groundwork.cloudhub.monitor.CloudhubMonitorAgentClient] (Thread-144) Time to execute monitor operation [17042] ms for agent cloudhub-amazon-1.xml
2018-07-21 17:17:45,133 INFO  [org.groundwork.cloudhub.monitor.CloudhubMonitorAgentClient] (Thread-144) Cloudhub Start the Monitor Process for agent cloudhub-amazon-1.xml
2018-07-21 17:18:01,269 INFO  [org.groundwork.cloudhub.monitor.CloudhubMonitorAgentClient] (Thread-144) VM black listed: terra.gwos

When you are through debugging make sure to set the logging level back to "ERROR" and again restart gwservices.