Overcome the MongoDB Backup is down alert

04 June,2020 by Jack Vamvas

I've set up a new MongoDB single node replicaset using MongoDB 4.2.7.    Although I'm using the same server and agents configuration settings as other replicasets in the same deployment - there is a regular MongoDB alert being triggered. 

Backup is down
Backup Agent

Alert Details

CREATED 2020/05/29 13:12 GMT
DATA-BEARING HOSTS (INSTALL AGENT ON AT LEAST ONE) rsdevelop (at least one of the following mongod(s) needs a co-located agent: [muserver.xxxxx.net:27017])
OPEN

The scheduled backups are being completed , and there aren't any errors. Why is the Backup is down  alert being triggered?

I checked through the MongoDB documentation and the related text for this alert is:

Backup is down

The Backup for a project with at least one active replica set or cluster is down for more than 1 hour.

To resolve this alert:

  1. To see which host serves the Backup, click Deployment, then the Servers tab.
  2. Check the Backup log file on that host.

 

In the MongoDB documentation there is a specification about how MongoDB alert notifications works:

"When a condition triggers an alert, you receive the alert at regular intervals until the alert resolves or Ops Manager cancels it. You can acknowledge an alert for a period of time, but if the alert condition persists, you will again receive notifications once the acknowledgment period ends"

I followed the path to the Backup log file on the host and reviewed all messages  - with no error messages - apart from the initial one.

So I started reviewing the Backup Agent status of the specific host - just doublecking the status and any other  observations. I noticed the host Backup Agent was in standby , and the other host in the deployment was Active. But remember reading in the MongoDB documentation, suggesting it's not a problem:

"Only one host can backup a deployment at a time. On the Server tab, the host that is backing up the deployment displays Backup - active. Any other host with Backup activated displays Backup - standby"

 

Next step was to restart the automation agent for the MongoDB server. 

--Stop the Automation Agent 

sudo service mongodb-mms-automation-agent stop

--Start the Automation Agent 

sudo service mongodb-mms-automation-agent start

 

The restart of the Automation agent - did not resolve the issue. I went to the next step , and restarted the MongoDB server - this stopped the alert reappearing. The Backup Agent is working fine , all snapshots and oplog slices are successfully and up to date. 

 


Author: Jack Vamvas (http://www.dba-ninja.com)


Share:

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment on Overcome the MongoDB Backup is down alert


dba-ninja.com