Alert Definitions - x360Recover

Written By Tami Sutcliffe (Super Administrator)

Updated at October 24th, 2023

Within Axcient x360Recover, there are thirteen (13) different classes of alerts that are automatically monitored and generated by the device. 

All of the alert events are automatically created on each x360Recover appliance or vault and are visible from the Alerts page of the main menu. 

  • Whenever an alert is raised on a device, if email alerting is enabled, and the corresponding alert class is selected for alerting, an email will be sent to the configured email address, detailing the event. This email can be delivered to your PSA/RMM email ticket portal to automatically generate a ticket in your ticketing system.
  • Whenever an alert is raised on a device, if ConnectWise is enabled and ticketing is configured for the corresponding alert class, a ticket will be created on the configured Service Board within your ConnectWise system. If a ticket already exists, the existing ticket will be updated.  If the alert is cleared, the existing ticket within ConnectWise will be set to the configured ‘Closed’ ticket status.

We recommended enabling ticket creation for all classes of alerts for ConnectWise, and all classes except for ‘Successful Backup’ with email alerting. 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The following tables enumerate each class of x360Recover alert, describe all possible error conditions and list the affected device types:

     Agent Status Appliance Only
     Agent Disconnected This alert will be created when communications between the agent and the appliance times out.  Occasionally this can occur during a running backup if the protected system is under heavy load or network bandwidth is being saturated by the backup.
     Agent Connection Restored This alert will be automatically cleared when communications between the agent and the appliance is restored.

 Note:  Agent Status alerts can be configured under Settings to apply to  'Servers and Workstations’, ‘Servers Only’, or ‘None’;

  • If you select ‘None’, Agent Status alerts will not be raised.  
  • If you select ‘Servers Only’, Agent Status alerts will not be raised for workstations, only for servers.
     Backup Metadata Appliance Only
     Missing {Guid}.rmt The system did not find the file {Guid}.rmt located within the agent installation folder on the protected system.  Restart the agent and check the agent log files for errors regarding the creation of the RMT file
     Missing GPTEFIDump.rbi The system did not find the file GPTEFIDump.rbi located within the agent installation folder on the protected system.  Restart the agent and check the agent log files for errors regarding the creation of the GPTEFIDump file
     Missing GPTBKPDump.rbi The system did not find the file GPTBKPDump.rbi located within the agent installation folder on the protected system.  Restart the agent and check the agent log files for errors regarding the creation of the GPTBKPDumpfile
     Missing {Guid}.meta File The system did not find the file {Guid}.meta located within the protected system data folder on the appliance.  Reboot the appliance to recreate the missing file, then manually clear the alert by clicking the red triangle in the Actions pane 

 Note: On x360Recover version 6.0.0 and newer, missing metadata files should automatically be repaired, and this alert should automatically close immediately. 

     Boot VM Check Appliance and Vault
     Insufficient Resources Nightly Boot VM Check is enabled but was unable to run due to insufficient resources.  Ensure there is at least 3GB of RAM available and storage pool utilization is <80%
     Count of Screenshots does not match count of Boot VM Checks One or more protected systems configured for Nightly Boot VM Checks failed to run and capture a screenshot.  Open a ticket with support to investigate the failure
     Check Boot VM Screenshots All protected systems configured for Nightly Boot VM Checks successfully booted and captured a screenshot.  Visually verify that the screenshots were good by viewing the images attached to the ticket, in each protected system details page, or using the Boot VM Report tab on the Management Portal

 

     Disk Usage Appliance and Vault
     xx% of storage space is used When > 80% of storage pool space is in use the appliance or vault will raise a storage space alert
     xx% of allocated space is used When > 80% of an allocated quota has been consumed by a customer, the appliance or vault will raise an allocated space alert
     xx% of OS Disk space is used When > 70% of the Operating System disk is in use, the appliance or vault will raise an OS Disk space alert

 

     Failed Backup Appliance Only
     Backup Failed The agent failed to complete a successful backup.  Check the Backup Jobs tab and the agent log files on the protected system for more details
     No backed-up volumes found The agent completed a backup but there are no backed up volumes present on the appliance afterwards.  Open a ticket with support to troubleshoot
     Volume is empty  The agent completed a backup but the backed-up volume on the appliance is empty.  Open a ticket with support to troubleshoot

 

     Firmware  
     Various Error messages from third party firmware update packages will be returned.

 

     Legacy Replication Not Supported Appliance and Vault
     Legacy replication is not supported on vault  x360Recover vaults that have been converted to Scale-Out Cloud mode do not support legacy replication and are unable to replicate with any appliance running versions older than 8.3.0

 

     License Unavailable Appliance Only
     License unavailable An agent attempted to register a new server or workstation with the appliance, but there were no available licenses of the correct type available.  Assign additional licenses to the appliance and perform a license sync.  Restart the agent service on the protected system to retry the registration process

 

     Missed Backup Appliance Only
     Missed Backup A scheduled backup operation was missed.  The Agent or Protected System could be offline, or a previous backup may still be running

 

     Pool Status Appliance and Vault
     Storage Pool degraded One or more disks in a RAID storage pool are missing or corrupted, but RAID integrity has not yet been breached.  Storage pool is operating in recovery mode and performance will be impacted.  Replace the problem disk as soon as possible and contact support for assistance beginning the RAID recovery process
     Storage Pool faulted Two or more disks in a RAID storage pool are missing or corrupted.  RAID integrity has failed and all data is lost.  Replace the failed disks and recreate the storage pool.
     Storage Pool unavailable None of the disks in the storage pool are accessible.  Investigate the status of physical disks in the device

 

     Protected System Appliance Only
     AutoVerify chkdsk failed (no action required) This alert will be raised if AutoVerify chkdsk scan fails on the protected system for any volume.  Raising this alert triggers the next scheduled backup to become a full scan in an attempt to self-heal the potential data corruption.  This alert will not be raised for a given protected system more than once every 30 days
     Configured volume missing A Windows volume that has specifically been configured for backup in aristos.cfg (e.g. BACKUP_VOLUMES=) is missing
     Partition layout change detected:      automatically scheduling a full volume scan to detect any new data (no action required) If the size or location of a protected Windows partition changes, the backup image volume no longer matches the source protected system and the data becomes corrupt.  When this is detected, the appliance will trigger the next scheduled backup to be a full scan, to synchronize the backup image with the source system

 

     Successful Backup Appliance Only
     Backup completed successfully The agent completed a successful backup of the protected system.  Any outstanding missed or failed backup alerts for this protected system will be automatically closed.

 

     Replication Appliance Only
     Vault ingestion is delayed by (x) days The most recent snapshot ingested at the vault for a protected system is more than (x) days older than the most recent snapshot on the appliance.  (x) days is configured on the appliance under Vault Settings.

 

Best Practices

Monitoring your managed services infrastructure is a key component to the success of your business model.  

To ensure the most reliable monitoring of your x360Recover devices and protected systems, x360Recover has two facilities to track and report issues with your systems:

Alerting and ticketing

The first, primary method is the alerting and ticketing features described above.  This method provides active, real-time ticketing when something is determined to be wrong.  However, ticketing fails to be useful when a device is offline, since a dead device cannot report itself as being down.

Management Portal report engine

While ticketing provides a real-time active event-driven monitoring solution, the Management Portal report engine provides for a proactive ‘current state’ view of your devices and protected systems. 

As part of your daily process, the Management Portal Health Report and/or Trouble Report pages should be checked briefly, to verify the status of all devices and protected systems.  Issues detected with appliance, vaults, and protected systems will be highlighted in red or yellow.  Hovering the mouse over the highlighted cells will display a popup tool tip with more detailed information about the reported issue.

report.jpg

Using the Management Portal report engine, you may proactively verify that all your devices and protected systems are online and up to date.  Issues with offline devices, storage pool issues, and backups or replications not up to date are highlighted boldly to facilitate investigation and troubleshooting.

Using ticketing to monitor your deployed devices provides near real-time alerting to issues with your environment.  This will facilitate your ability to rapidly respond to such issues as they arise.


 

SUPPORT | 720-204-4500 | 800-352-0248 

  • To learn more about any of our Axcient products,  sign up for free one-on-one training.
  • Please contact your Partner Success Manager or Support if you have specific technical questions.
  • Subscribe to the Axcient Status page for a list of status updates and scheduled maintenance.