What is an incremental backup?

Incremental backups allow for continuous storage of changing data sets. This minimizes the time, bandwidth, and storage requirements compared to repeated full backups. Incremental backups begin with a full backup, followed by a chain of smaller incremental backups, but results in higher complexity of backup and restore processes. Special software is used to cope with this.

An incremental backup is a backup method that is often used as part of a wider backup strategy. Unlike a full backup, it does not create a complete copy of the dataset to be backed up. Instead, only changes made to files as of the last backup are saved.

Unlike a differential backup, the reconciliation of changes does not necessarily refer to the last full backup. Rather, the changes since the last full or incremental backup are saved.

In general, there are two broad categories of approaches to incremental data protection:

1. The granularity of changes
2. The use of the strategy to be employed

Usually, hybrid approaches are used. For example, the popular tool Rsync creates `block-level synthetic full backups´ in normal use. Let’s look at the different types of incremental backup in detail.

Digitally stored data consists of almost infinitely long chains of zeros and ones. These bits are grouped into logical units, of which files are the best known. Below file level, however, there are `blocks´ and `bytes´ as logical groupings of individual bits.

Comparing changes between the last backup and the current state of the dataset can refer to different levels. The closer the matching of changes gets to the individual bits, the more efficient the incremental backup becomes. While files vary in size, blocks and bytes have a defined, fixed size. For example, a byte comprises eight consecutive bits; blocks are usually between 512 and 4,096 bytes long.

A file-level incremental backup only distinguishes whether a file has been changed. The extent of the change is not taken into account. If even a single bit has been changed, the entire file is backed up again in the course of the incremental backup. To illustrate this, imagine a large book manuscript that is stored as a single file on a data storage device. If even a single letter is changed, the entire manuscript is backed up again as part of a file-level incremental backup.

A file-level incremental backup is the easiest to implement because file systems log the modification date of files. To reconcile the changes, it is sufficient to compare the timestamps of the last modification of a file on the source and target systems. If the timestamp on the source system is newer, the file has been modified and must be backed up again. Thus, file-level incremental backups are inefficient for small changes to large files.

The term `block´ originates from data storage. Hard disks and other mass storage devices organise sections of consecutive bytes as a contiguous logical area. When creating the block-level incremental backup, only the changed blocks are backed up. The advantage is the smaller size of the data to be transferred and stored. Thus, only the changed sections of individual files are backed up.

The main disadvantage is that a mechanism is needed to log which blocks have changed. Some storage technologies are able to manage this information and thus can implement block-level incremental backups. Otherwise, the full backup at block level would need to be compared with the current state to detect changes.

A byte-level incremental backup is an even more detailed version than a block-level incremental backup. Only the changed bytes of a block are backed up. In terms of the time required and the storage space needed, the byte-level incremental backup may be significantly more efficient. However, the process is more demanding. This is because a journal of the byte-level changes is required to identify the changed bytes. The additional effort for byte-level incremental backups is useful for selective changes to very large files.

Incremental backups are part of a chain of individual backups that contain changes to the previous state. At the beginning of the chain there is always a full backup. There are various methods of constructing the chain and creating new full backups. This has advantages in terms of complexity and duration of backup and restore processes. Here are some strategies for creating incremental backups.

A traditional incremental backup strategy requires periodic full backups. Creating a full backup is time-consuming and slow. With synthetic full backups, a new full backup is created. The original full backup and the subsequent incremental backups are used. This saves the copying of the entire dataset from the source system that would otherwise be required when creating a full backup. The process is also known as `forward incremental backup´.

The incremental forever backup is optimised for backing up hard disks and similar media. Initially, a full backup is created; afterwards only the changed blocks are stored in a `forever´ continuous chain. Unlike a synthetic full or a reverse incremental backup, no further full backups are created after the initial one. This saves storage space and minimises the data transferred during the copy operations.

The reverse incremental backup works similar to the synthetic full backup. After each incremental backup, changes are merged with the last full backup. This means a full backup exists by the end of the backup chain. It is true that creating a reverse incremental backup takes longer than a forward incremental backup. However, if necessary, the source system can be restored without delay, since the last current status is already available as a complete copy.

An enhanced incremental backup can be of any type. A special feature is that it also detects changes to files caused by moving or renaming them. Provided large files are involved, this boosts efficiency.

The biggest advantage of incremental backups is that individual backups are small. Since only the differences as of the last backup are saved, the backup process requires relatively little time, bandwidth, and storage space. The prerequisite is that the period between the individual backups is kept short. Otherwise, large amounts of changed data accumulate and have to be transferred.

The inherent advantage of incremental backups is well illustrated by the term `backup window´. This is the time period within which a backup can be created without disrupting operations. If the amount of data is so large that the copying process would take longer than the backup window allows, it becomes impossible to create a full backup during operation. In that case it is advisable to create small incremental backups at short intervals.

The immediate disadvantage of incremental backups is that specialised software is usually necessary. Planning the backup strategy also requires more effort than creating full backups. Restoring the original state from the backups is similar. Because data is spread across several backups, the process is more complex. There is also a higher risk of data loss. If one incremental backup in a chain is damaged, the integrity of all subsequent backups is affected.

Incremental backups are primarily aimed at minimising the storage requirements and time needed to create a backup. They are therefore always used when the repeated creation of complete backups would be logistically nonsensical. Let’s look at a few scenarios in detail.

To create a backup on Mac, it is best to use the macOS built-in backup tool “Time Machine”. The tool backs up changes to the internal data storage incrementally to an external hard drive. As usual with incremental backups, an initial full backup is created during the first run. Subsequent changes to the file system are logged and are available for further backups.

Time Machine allows you to restore individual files to earlier points in time. Furthermore, the complete system can be reconstructed from the backup. This is practical in case of faulty hardware or if you want to migrate your own system to new hardware. Its simple handling of the backup software is particularly impressive. The user only has to connect an external hard disk and launch the backup process; the rest is automated.

Incremental backups are also available on Windows. On the one hand, a backup can be created in Windows 10 with the help of Windows Backup. On the other hand, you can use the Robocopy backup tool in the command line to incrementally backup the contents of a directory. Let’s look at an example:

The /MIR option here stands for `mirror´. The command mirrors the source directory to the destination path. If a directory with the same name already exists, an incremental backup is performed. Robocopy then transfers only the changes as of the last backup operation.

Robocopy exists only under Windows. To create a server backup with Rsync under Linux, an incremental backup is used. First, a full backup is created. When the command is next called up, Rsync transfers the blockwise changes only as of the last backup. The transferred data is merged with the existing data set. Thus, the result of the backup operation is a synthetic full backup. Here is an example of a corresponding Rsync call: