Overview

A ShadowProtect continuous incremental chain has many pieces and parts, not all of which are needed at any given time. Understanding the chain, and how the pieces and parts are used, is an integral part of managing your ShadowProtect backups.

ShadowProtect is a chain-based backup. This means that each backup relies on the backup before it to continue taking backups and to be able to do any type of restore. If a piece of the chain is missing, a restore is not possible from that point forward in the chain. To keep from needing every single incremental image taken forever, ShadowProtect uses its program ImageManager to collapse the chain – allowing retention to remove images per the settings you specify, while still keeping the chain intact.

To achieve this goal, ImageManager looks at the files from the period of time that will be collapsed, and creates a new file that incorporates as much of the changed data as possible into one single file. The backup images that have been incorporated into this collapsed image are now not needed for the chain to be intact. ImageManager keeps these files for whatever retention you configured, and then deletes them. This keeps your space usage down, and ensures the most efficient restore process possible for your chain at any given time.

Reading Your ShadowProtect Backup Chain 

[Click for a larger view]

.SPF = ShadowProtect Full image
.SPI = ShadowProtect Incremental image
.SPK = ShadowProtect Key file for encryption
.MD5 = MD5 hash file
.bitmap = bitmap for image file

The first and most important image in any chain is the .spf. This is the ‘full’ image. This is what the entire rest of the chain relies on, and must be present for the chain to be intact. This will usually be the largest image in the chain, though in some unique cases it is not.

The other files that are associated with this specific image file are the .md5, the .spk, and the .bitmap.

• The MD5 must be present for the image to useable.
• The bitmap does not have to be present.
• The SPK is also not required for the file to be useable. This file, if present, is used by ImageManager when a file is encrypted. If encryption is not set in the ShadowProtect backup job, this file will not be present. If encryption is set through the ShadowProtect backup job, but the SPK is not present, the encryption key must be programmed into ImageManager before the files can be collapsed. This file can be re-created if it is deleted or removed for any reason.

Collapse types 

As ImageManager collapses the chain, letters are appended to the name of the image to designate the different collapse types:

-cr = collapsed rolling (not always present)
-cm = collapsed monthly
-cw = collapsed weekly
-cd = collapsed daily

For each collapse type, there will only be one file for that given time period – for example: one collapsed daily per day. 

Each consolidated file name will be the same number as the last incremental or smaller consolidated file present in the chain where the rollup occurs.

It is important to note that the time/date stamp on the collapsed files will be the same as that on the last file image rolled into that collapse. The time that the collapse actually occurred will be the time/date stamp on the MD5 file.  

This behavior of naming and timestamp association continues when daily files are consolidated into weekly (C_VOL-b###-i###-cd-cw.spi), weekly into monthly (C_VOL-b###-i###-cd-cw-cm.spi), and monthly into rollup (C_VOL-b###-i###-cd-cw-cm-cr.sp).  

Note: The monthly file name (and, correspondingly, the rollup) might not always have a "cw" as part of the name.  The "cw" in the monthly name would only happen if the end of the week (default Saturday) also fell at the end of the month so both the weekly and monthly were created at the same time.  

The collapsed rolling file is created by the ‘Cleanup consolidated monthly image files (-cm)’ in your ImageManager retention options. This is optional, and does not have to be utilized. If you opt to use this, you will have to specify the number of months after which you want it to start. If you chose 3, for example, your chain would have 3 months of -cm files immediately after the base image. On the 4^th month after the base image, the first -cm would be rolled into the -cr file and deleted. The -cr file then connects the chain to the base image. Each month after the -cr is created, the oldest -cm is ‘rolled into’ the -cr. Keep in mind that the -cr file itself doesn’t change.  ImageManager creates a new -cr file each month as it rolls the changes into the last -cr file. After it successfully creates the new -cr, the old -cr is then deleted by ImageManager. As each new month is rolled into the -cr file, the new -cr file created will reflect the name of the month that was just rolled into it.

Understanding Which Files Are Required In The Backup Chain

It is best to allow ImageManager to apply retention for you, as ImageManager will never delete a file that is still required in the chain.

It is important to understand how the chain dependencies work, though.

Because the chain only relies on the ‘largest’ image available to it within the chain, once it jumps ‘up’ to a larger collapse, it never drops back down. From ‘largest’ to ‘smallest’ the images go: .spf, -cr.spi, -cm.spi, -cw.spi, -cd.spi, -.spi

If you were to need to restore from 9/10/20, as the data existed before someone deleted a file at 1:05 PM, you'd need to use the image C_VOL-b002-i633.spi

Only the following images would be required:

C_VOL-b002-i633
C_VOL-b002-i632
C_VOL-b002-i631
C_VOL-b002-i630-cd
C_VOL-b002-i624-cd
C_VOL-b002-i620-cd-cw
C_VOL-b002-i593-cd-cm
C_VOL-b002-i407-cd-cm
C_VOL-b002-i222-cd-cm
C_VOL-b002-i042-cd-cm
C_VOL-b002

• Even though the other images are not “needed” for the chain integrity, you should keep these images for granularity.  Each time a roll up occurs, the backup image you’re left with loses granularity. Not every change of every file can be documented in the roll up. Only one change per file can be kept. So, even though files may not be needed in regards to an intact backup chain capable of a restore, keep the files in case you need a more granular restore from the time frame of a given backup.
• You must also keep any file necessary for any given backup image to still be part of a complete chain as well, for as long as you want to keep a given image. This why it is always best to allow ImageManager to apply retention to your backup chain and not attempt to do it yourself manually. 

ShadowProtect does offer a way to confirm what files are required in a given chain via command prompt with the command ‘image qp’.

There are different switches available, and if you ever need to use this command, you will likely want to use the switches “f=fsr d=\$n” as it gives you a clean, easy to read output.

For this same example we’ve been using - C_VOL-b002-i633.spi – the image qp output would look like this: