Tech Tip 4 - Solid backup of monitoring server

Version 2 by Bren Eckles
on Apr 15, 2019 09:56.

compared with
Current by Bren Eckles
on Apr 15, 2019 10:49.

Key
This line was removed.
This word was removed. This word was added.
This line was added.

Changes (12)

View Page History
Your GroundWork Monitor (like any other application) may be subject to an unexpected failure. This stuff happens, and it always happens at the worst time. Like 5:45pm on a Friday night when you are leaving for a three-day weekend. Yes, we speak from sad experience. You might have a hardware failure, or if you are running virtual instances, just an outage at the hypervisor level. Often the cause is human error: an unintended action that deletes important files. No matter what the cause, when you find that GroundWork Monitor is no longer responsive, it may be necessary to put a new server or instance in place with the most recent image you can recover. If the outage is not a complete loss, a less extensive recovery may be sufficient, and while GroundWork support is here to help, you can give a a lot or a little to work with, and help us to get you great results.

h5. So what's the best practice?

As usual, it depends\! You might just be running GroundWork as a virtual machine or instance, in which case you should snapshot it regularly, and automate that process. We will assume you aren't doing this, or that it is otherwise impractical to do so. Even if you use snapshots, though, don't forget to build a test machine from them in a maintenance window, just to be sure you can. See "Restore" below.
So, this TechTip provides a typical, best practice example. Please put this into place. It's a matter of time before something breaks, and if you are prepared, you will look like a genius. So, by extension, this is a tech tip on how to look like a genius.

Requirements:
h5. Requirements

A good place to put the backups (like a mounted drive on the GroundWork Server)
Access as the root user to all of the GroundWork Servers. If you have multiple GroundWork servers in a distributed architecture, you are going to set this up for each one.

h5. How to make the backups?

We are providing examples of two kinds of backup, here. The first is a dump of the databases and the RRD files, and it can be run without interrupting monitoring. The second demands a maintenance window, because it creates a complete recovery snapshot of GroundWork in a quiescent state.
{noformat}

Here is the how to How To for the backup utility. If you do not have the latest version of the backup utility, use the link to download and install it on your system. Make sure the script above references it, if the name changes.

* [GW Backup Utility|https://kb.groundworkopensource.com/display/SUPPORT/Backup+utility]

Restore:
h5. Restore

Restoring a system may involve using one or the other of the elements generated above. If you were so unlucky as to lose the whole system, you can follow the directions here, using the most recent full backup you created:
* [GW Backup Utility|https://kb.groundworkopensource.com/display/SUPPORT/Backup+utility]

Then you may ADDITIONALLY follow directions for restoring one or more of the databases, for which you might have a more recent backup. Look here for the detail on how to do that:
* [Restoring GroundWork|https://kb.groundworkopensource.com/display/DOC70/How+to+restore+a+system+configuration]

Restoring the RRD files is just an unpacking of the saved archive. Find the most recent copy in your backup directory and run this command, substituting the name of the tar file you find:
Make sure you can reboot the system and have GroundWork come up and start monitoring without intervention. Power failures can happen too...

Final Notes:
h5. Final Notes

In our example we show backups running once a week or once a month. You might like to do it more frequently.