Installing or Upgrading GroundWork Monitor

Overview

Welcome to the installation instructions for GroundWork Monitor Enterprise Edition, version 7.2.0. These instructions are maintained in the GroundWork knowledge base (kb.groundworkopensource.com). If you are reading these in a PDF or other offline form, you should check the online form for the most recent version.

This document covers the installation of GroundWork Monitor 7.2.0 and supported upgrades from 7.1.1. After reviewing Read Me First documents noted below, for new installs see SECTION I and for Upgrades see SECTION II.

Read Me First
Before performing an installation, please review 7.2.0 Release Notes for EE which contain details of issues fixed, new features, and items of interest to those users who have customized or significantly enhanced GroundWork Monitor Enterprise, System Requirements for 7.2.0, and any Technical Bulletins for 7.2.0 so you can prepare for any additional steps required either before or after the process.

CONTENTS

RELATED RESOURCES

WAS THIS PAGE HELPFUL?

SECTION I: New Install of GroundWork Monitor Enterprise 7.2.0

If you need a new installation of GroundWork Monitor versus an upgrade, first take a look at the Installation Options document, then follow the steps below to install. The major installation decisions and steps include:

Installation steps

Post-install configuration

Starting and Stopping GroundWork Monitor
GroundWork Monitor includes all prerequisites and components within a single installation package. The software components of GroundWork Monitor are installed under /usr/local/groundwork, with the exception of the log rotation configuration and the start/stop script named /etc/init.d/groundwork. The start/stop script is usually used indirectly, as follows: 
service groundwork status
service groundwork start
service groundwork stop

This script can also be used to stop, start, or restart individual services. For example:

service groundwork restart nagios
The service command generally resides in the /sbin/ directory (/usr/sbin/, on Ubuntu). That directory will normally be part of the root user's command-search path. If not, or you experience issues with the mapping of the service command to systemd sysctl calls in your distro, you can run the /etc/init.d/groundwork script directly: 
/etc/init.d/groundwork start

The service command is preferred because it is easier to remember and type, however as of version 7.2.0 GroundWork has not supplied systemd compatible init scripts, so if you are running on a systemd-enabled distribution, you may have to use the init script directly.

SECTION II: Upgrade Installation

Upgrades to 7.2.0 are supported from version 7.1.1 ONLY. If you wish to do an upgrade, review this section. For a fresh installation, see SECTION I: New Install of GroundWork Monitor Enterprise 7.2.0 above.

If you are unsure if this has already been done or whether this applies to your installation, please contact GroundWork Support for assistance.

Prerequisites

Before you begin it is critically important that you test for each of these prerequisites and obtain them. For instance, getting partway into the process only to realize that you don't have the postgres user password and that no one can obtain it quickly means wasted time and disappointment, as well as the chance that the system may be down and require rolling back. Don't let this happen to you.

Special Considerations
  • These instructions assume you are doing an in-place upgrade. If you wish to do a migration to a new server, please contact GroundWork Support for advice on your options and the trade-offs involved.
  • Migration to a different machine may be required, if your current operating system version is no longer supported by GroundWork Monitor. See the OS Platform Requirements section in System Requirements for 7.2.0.
  • Some customers have additional add-on integrations installed (e.g., Ganglia Integration, AlertSite Integration, Webmetrics Integration, ServiceNow Integration, JIRA Integration, other helpdesk integrations). Special considerations apply in those cases. If these or other integrations or extensions were provided by GroundWork Professional Services, you must contact GroundWork Professional Services for advice. Otherwise, contact GroundWork Support for details.
  • LDAP integration will NOT be reset by this upgrade unless you are using multiple endpoints, in which case you will need to follow the instructions described in post-upgrade tasks. Otherwise no action is needed post install to support LDAP configuration.
  • HTTPS Support will be reset, however this is now scripted. See post-upgrade tasks below.

Backups

If you have a standard uncustomized installation, simply make a backup of the GroundWork Monitor installation using the provided backup tool. Download the latest version of the backup utility from the knowledge base, and make a full backup of your installed system as described in the Backup utility description.

If any customizations have been applied, including additional scripts, plugins, etc., we recommend that you instead create a complete on-disk backup of /usr/local/groundwork directory, so that you can easily restore the files you added or changed. The installer will flag the files that are updated and that should be merged, but if you have the disk space available this step makes it easier to restore your customizations that the installer can't detect or anticipate. You should stop the GroundWork server and cron jobs, ensure you have adequate disk space, and make the copy like so (as root):

  1. Check for space available: 
    df -h
  2. Check for space used by GroundWork: 
    du -sh /usr/local/groundwork
  3. Stop all GroundWork Processes and cron (called crond on some systems): 
    service groundwork stop
service cron stop
  4. Copy the directory (in this example to /tmp/gwbackup): 
    mkdir /tmp/gwbackup
cp -a /usr/local/groundwork/* /tmp/gwbackup/
  5. Start the services again (Note: You don't need all the services to be running for an upgrade to work, only postgresql): 
    service cron start
service groundwork start

    (or)

    service groundwork start postgresql

Permission and ownership check

From the command line run the following:

find /usr/local/groundwork -type f -group root -exec ls -la '{}' \; |grep -E "root" |grep -Ev "\->" |grep -Ev "supervise"|grep -Ev "control" |grep -Ev "lock" |grep -Ev "status" |grep -Ev "mib_" |grep -Ev ".ctl" |grep -Ev "backup/" |grep -Ev "Catalina" |grep -Ev "/scripts/" |grep -Ev "/ntop/" |grep -Ev "apache2" |grep -Ev ".pid" |grep -Ev ".index" |grep -Ev ".log" |grep -Ev ".lck"

This will list files owned by root, and should not include any java libraries (.war, .jar, .ear) and configurations (.properties, .cfg). If any such files are found, change the user/owner to nagios.

Postgres user configuration on SLES

If you are upgrading a Groundwork Installation that is installed on SLES you will need to run the following command, prior to running the installer, to verify that the postgres user is set up properly.

if mkdir ~postgres; then chown $(id -u postgres):$(id -g postgres) ~postgres; fi

Installations with GDMA connecting to GroundWork server via HTTPS

If you are using GDMA and the GroundWork server version 7.1.1 you are upgrading from is using HTTPS, you will need to re-set most of the settings for HTTPS after upgrade. You will then be able to use the later version of GDMA (2.5.X) supplied with 7.2.0, and to transition the potentially large numbers of older agents without service interruptions. However, as of 7.1.1, the default SSL configuration uses only TLS 1.2, and will not accept connections from older GDMA agents without adjustment.

Please see the Post upgrade tasks section below for details. Be sure to plan for this activity after the upgrade is complete. You will eventually want to transition your GDMA agents one-by-one to the later versions (GDMA 2.5.0 or later) and eventually enable only the more secure settings of GroundWork Monitor Enterprise 7.2.0.

Upgrade options

Distributed database configurations are supported. Therefore, a GroundWork Monitor single-server installation can be upgraded to a configuration where the GroundWork Monitor software and the database are installed on different servers. The new database instance needs to be installed before the upgrade of GroundWork Monitor can be started.

Moving the PostgreSQL Database
These instructions only describe an upgrade that keeps the database where it already is, either separated from the GroundWork Monitor server or on the same machine. Additional steps not documented here will be needed for an upgrade that involves moving the PostgreSQL database to a separate machine as part of the upgrade. If you desire to run a separate database server, refer to the GroundWork Knowledge Base for further information, or contact GroundWork Support.

Upgrade procedure

The high-level upgrade steps for this transition are:

Step 1 - Complete the backup described above
Step 2 - Upgrade a Remote PostgreSQL Database, (if you were previously running a Remote PostgreSQL database, that component must be upgraded first)
Step 3 - Upgrade GroundWork Monitor Enterprise, and select your graphing option
Step 4 - Install new license key, if required
Step 5 - Re-merge any specialized configuration changes, Reset HTTPS settings, etc.

Step 1 - Backup (above)
Step 2 - Upgrade a Remote PostgreSQL Database
Skip this step if you are running the PostgreSQL database directly on the GroundWork Monitor server.

In order to upgrade a PostgreSQL database running on a remote machine:

  1. Run a backup as you would on a normal GroundWork server, see Backup utility description.
  2. Download the installer (e.g., groundworkenterprise-7.2.0-br475-gw3635-linux-64-installer.run) to the Remote PostgreSQL database server.
  3. Make the installer executable: 
    chmod +x groundworkenterprise-7.2.0-br475-gw3635-linux-64-installer.run
  4. Back on the GroundWork Monitor server, stop the software for the duration of this part of the upgrade: 
    service groundwork stop
  5. Run the installer on the Remote PostgreSQL database machine, and answer the prompts in the obvious ways. Note that you will be prompted for the root login and password. 
    ./groundworkenterprise-7.2.0-br415-gwMMMM-linux-64-installer.run --mode text
  6. As usual, you will find the upgrade report in the /usr/local/groundwork/upgrade-report.txt file. For the upgrade from 7.1.1 to 7.2.0, we made changes to the postgresql.conf file. If you made changes to the default as well, you'll have to merge any differences between the installed copy and the backed-up copy on a case-by-case basis.
Step 3 - Upgrade of GroundWork Monitor Enterprise

Once you have a pre-upgrade backup, and (if necessary) the Remote PostgreSQL database upgraded, you may then proceed with the upgrade.

The upgrade has a dependency on the curl package being installed on the system. The curl package can be installed using the system package manager.
Example RHEL/CentOS: yum install curl, apt-get install curl for Ubuntu or yast -i curl for SLES
  1. Download the installer (e.g., groundworkenterprise-7.2.0-br475-gw3635-linux-64-installer.run) to the GroundWork system that needs to be upgraded.
  2. Make the installer executable: 
    chmod +x groundworkenterprise-7.2.0-br475-gw3635-linux-64-installer.run
  3. Execute the installer. We recommend text mode. 
    ./groundworkenterprise-7.2.0-br475-gw3635-linux-64-installer.run --mode text


    Optional: Unattended installation options relating to GrafBridge

    Use these options on the command line if you want to run an unattended upgrade. Be advised that these options will skip the selection of graphing technologies, and should be used only if you know exactly what you want to do, or are advised to do by GroundWork Support staff.
    Select the options to use accordingly:

    • InfluxDB only: 
       --mode unattended --postgres_password $PGPASSWORD --groundwork_portal_root_user root --groundwork_portal_root_password root
    • InfluxDB and RRD: 
      --mode unattended --postgres_password PGPASSWORD --groundwork_portal_root_user root --groundwork_portal_root_password root --tsdb_server_rrd 1
    • RRD only: 
      --mode unattended --postgres_password PGPASSWORD --groundwork_portal_root_user root --groundwork_portal_root_password root --rrd_only_tsdb 1


  4. The installer detects an upgrade. Choose yes to continue. 
    Export the display to user's IP address to see the installation wizard.
You may exit the installation at this point or continue with the
installation in text mode.
Do you wish to Continue? [y/N]: y
GroundWork is already installed. Do you want to upgrade? [Y/n]: y


  5. Several questions here should be self-explanatory; answer them in the obvious ways (backup, database user verification, root login id and password).
  6. When prompted as follows, you should enter "N" for no if you wish to use GrafBridge: 
    ----------------------------------------------------------------------------
Do you want to use RRD as your only performance data time-series database? [y/N]:


  7. If you enter "y" for yes here, you will have only RRD for graphing, as you had in 7.1.1. If you enter "N" for no, you will get the choice of technology: 
    ----------------------------------------------------------------------------
Please select which time-series databases you wish to configure.
RRD [y/N]:
InfluxDB [Y/n]:
  8. The defaults will install GrafBridge (InfluxDB and Grafana) and are recommended.
  9. Allow the installer to complete on its own, don't interrupt it. In our testing on medium-grade equipment, this part ran for 15 minutes.
  10. Right before the end of the upgrade, a list of files you need to deal with manually will have been displayed: 
    Warning: During the upgrade procedure, the following files were detected to have
modifications. They were backed up to this directory,
/usr/local/groundwork/backup-2017-11-01. You will need to login in to this
server and manually merge these files. Some files may appear in this list
because you probably made local changes to certain options, and those changes
should now be brought forward.  Some files may appear in this list simply
because the content of the file has changed between releases. If that is the
case for a given file, and you had not changed any option values in the previous
release, you should not need to do any work to merge the old and new copies at
this time.
Press [Enter] to continue :
List of modified files:
-----------------------
/usr/local/groundwork/apache2/conf/httpd.conf
/usr/local/groundwork/apache2/conf/groundwork/apache2-noma.conf
/usr/local/groundwork/common/etc/snmp/snmptt.ini
/usr/local/groundwork/common/etc/syslog-ng.conf
/usr/local/groundwork/config/cloudhub.properties
/usr/local/groundwork/config/foundation.properties
/usr/local/groundwork/config/perfdata.properties
/usr/local/groundwork/config/status-feeder.properties
/usr/local/groundwork/config/status-viewer.properties
/usr/local/groundwork/config/ws_client.properties
/usr/local/groundwork/config/event-feeder.conf
/usr/local/groundwork/config/fping_process.conf
/usr/local/groundwork/config/log-archive-receive.conf
/usr/local/groundwork/config/log-archive-send.conf
/usr/local/groundwork/nedi/nedi.conf
/usr/local/groundwork/postgresql/data/postgresql.conf
/etc/logrotate.d/groundwork
(etc.)
Press [Enter] to continue :

    For easy reference after the fact, this information is available in the upgrade report in the /usr/local/groundwork/upgrade-report.txt file.

Step 4 - Install new license (if needed)

Normally, old licenses that are still valid will continue to work after upgrade, but if you have been issued a new license file, this is a good time to install it, (GroundWork Administration > GroundWork License).

Step 5 - Re-merge file changes you made

Once the installer has completely finished, you must compare the new copies of these files with the backup copies, and merge any local customizations in the old files into the new files. In certain cases, the backup copy may live under a different name. For the example above, the new and old files would be found in the following locations. For simplicity of presentation, all the non-absolute pathnames in this table are specified relative to the /usr/local/groundwork/ directory. For example:

New file Backup copy of old file
apache2/conf/httpd.conf backup-2017-11-01/apache2/conf/httpd.conf
common/etc/syslog-ng.conf backup-2017-11-01/common/etc/syslog-ng.conf
config/bronx.cfg backup-2017-11-01/config/bronx.cfg
config/cacti.properties backup-2017-11-01/config/cacti.properties
config/console.properties backup-2017-11-01/config/console.properties
config/db.properties backup-2017-11-01/config/db.properties
config/foundation.properties backup-2017-11-01/config/foundation.properties
config/perfdata.properties backup-2017-11-01/config/perfdata.properties
config/status-feeder.properties backup-2017-11-01/config/status-feeder.properties
config/status-viewer.properties backup-2017-11-01/config/status-viewer.properties
config/ws_client.properties backup-2017-11-01/config/ws_client.properties
postgresql/data/postgresql.conf backup-2017-11-01/postgresql/data/postgresql.conf
/var/spool/cron/crontabs/nagios backup-2017-11-01/crontab-nagios-2016-11-01

Often, the differences you find between the new copy and the old copy will be due to small differences in timestamps, commenting, or spacing, and can be safely ignored. Sometimes, new information will be present in the new copy that should be left alone. Only change those areas of the new files that you know you are responsible for.

In case you had SSL enabled, the upgrade will absolutely break your settings in files like httpd.conf, because we revert to a default non SSL condition. Your best course may be to use the new tool as described below and here to restore the SSL capability. Following this step you can then review the differences between the backup and what is running, for additional changes you may have set up in 7.1.1
Understanding Installation Problems
The installer leaves a log file in a filename like /tmp/bitrock_installer_10602.log which contains a record of the install processing. In case of trouble during the installation phase, that's a primary place to look for clues.

Post upgrade tasks

After a successful upgrade to GroundWork Monitor Enterprise 7.2.0, some additional steps are necessary to finish the upgrade process. Please review the following notes and make sure that you apply the changes to your installation.