How to data retention

WAS THIS PAGE HELPFUL? Leave Feedback

Overview

The data retention procedures are used for maintaining database events (log messages) and event reporting in case of low disk space or system performance issues. Additionally, you will find information on managing the different GroundWork Monitor generated logs files. This page covers how to maintain the Foundation and Insight Reports database. Specifically, how to delete event messages and report entries, and clean log files.

Important Notes Notifying Users - Before starting with any system maintenance you will need to notify GroundWork Monitor system users that the system will be unavailable for a short time and that they will need to log out otherwise they will receive error messages and will loose their current session. You can schedule a maintenance window to notify all system users. Command not found - If you received a message similar to command not found or no such file or directory you may need to set the environment variables for the shell by running the script source /usr/local/groundwork/scripts/setenv.sh. This works in each shell session and only in shells similar to the Bourne shell, like bash. It will not work in either csh or tcsh. Required password - You will need to have the PostgreSQL postgres user password available to perform various commands. This is the same database password that was used for the GroundWork Monitor installation or upgrade.

Steps

Setting up the Archive process archive_gwcollagedb

• See How to back up and restore: Archive Database.

Maintaining the Insight Reports Database dashboard

To selectively delete report entries from the dashboard database that are older then a certain date, use the following procedure.

1. Open a terminal session and login as root user.
2. Make sure no one is logged into the system, then stop GroundWork services and the Apache httpd daemon with the following commands:

   /etc/init.d/groundwork stop gwservices

   /etc/init.d/groundwork stop apache

3. Enter the following command:

   psql -d dashboard

4. Delete specific report entries by entering the following commands with specified dates. The example below will delete daily alert entries before January 1, 2017.

   delete from measurements where TIMESTAMP < '2017-01-01';

   This will delete daily host availability entries before January 1, 2017.

   delete from host_availability where DATESTAMP < '2017-01-01'

   This will delete daily service availability before January 1, 2017.

   delete from service_availability where DATESTAMP < '2017-01-01'

   This will delete host group availability entries before January 1, 2017.

   delete from hostgroup_host_availability where DATESTAMP < '2017-01-01'

   This will delete daily service availability entries before January 1, 2017.

   delete from hostgroup_service_availability where DATESTAMP < '2017-01-01'

5. Exit the psql command-line utility by entering the following command:

   \q

6. Restart GroundWork services and the Apache httpd daemon with the following commands:

   /etc/init.d/groundwork start gwservices

   /etc/init.d/groundwork start apache

Log Rotation

The packages that comprise GroundWork Monitor will generate log files in several places on the disk. This is by design, and is useful in many cases to find out what is happening with the data that the packages gather and store. Nagios, for example will use its log files for the default reports it generates. The down side is that log file take up space, and so there are several settings needed for the Linux utility program known as logrotate to clean up these files periodically. GroundWork has adjusted these settings according to parameters that will work in most cases. Sometimes, however, it will be necessary to add or remove logs from rotation. For example, if a package is added, and it generates log files, these should be rotated to avoid filling the disk.

• Log rotate settings:
  All the settings specific to GroundWork can be found in the /etc/logrotate.d/groundwork file. This file is created when GroundWork Monitor is installed, and is removed upon and uninstall. This file may be edited to adjust the settings as necessary, however, when adjusting logrotate settings, be sure to test them with the following command using root user:

  logrotate -d /etc/logrotate.d/

• Log files rotated by default:
  The following log files are rotated by default at the interval of once a day, with old files saved for 8 days:
  • /usr/local/groundwork/apache2/logs/access_log
  • /usr/local/groundwork/apache2/logs/error_log
  • /usr/local/groundwork/core/services/foundation/log/main
  • /usr/local/groundwork/foundation/container/logs/nagios2collage_hostgroupcheck.log

• Other files:
  The following files may or may not exist, depending on user settings. They will be rotated once a day, with old files saved for 8 days, if they do exist:
  • /usr/local/groundwork/nagios/eventhandlers/process_service_perf.log
  • /usr/local/groundwork/common/var/log/snmp/snmptt.debug
  • /usr/local/groundwork/common/var/log/snmp/snmptrapd.log
  • /usr/local/groundwork/common/var/log/syslog-ng/ (all files)
• The following files may be accidentally created, due to default settings in the Nagios tool. These settings are off by default in GroundWork's implementation of Nagios, but they would grow without bound if turned on, and that can fill the disk. Users should be aware that these files will be rotated by default every day, and that only one day's worth of old files will be kept.
  • /usr/local/groundwork/nagios/var/host-perfdata.dat
  • /usr/local/groundwork/nagios/var/service-perfdata.dat

• Nagios log files:
  Finally, the Nagios log files are archived by Nagios, but are never deleted. These files are useful for reporting purposes within the Nagios system (Nagios Reports). They can be large, however, especially if the default logging is changed in the Nagios tool to log more information, such as state stalking. Thus GroundWork settings will remove old log files after a month. The old Nagios log files are located in the /usr/local/groundwork/nagios/var/archives directory.