Series Compression Methods

Data series compression is used to reduce the amount of information required to represent a data object. The compression is particularly useful when lots of data is needed to be transmitted or stored.

Delta Compression

Delta compression involves encoding the difference between successive values instead of transmitting or storing the actual values themselves. This method takes advantage of the fact that in many real-world data sets, consecutive values are often similar or change gradually over time.

Here's how delta compression works for a series of numerical values. First, let's assume we have a data series like this:

`X_1,X_2,X_3,X_4,X_5,...,X_n`

Instead of transmitting the original series, delta compression involves transmitting the first value and then transmitting the differences (deltas) between consecutive values, like this:

`X_1,X_2-X_1,X_3-X_2,X_4-X_3,X_5-X_4,...,X_n-X_{n-1}`

In this compressed representation, each value after the first one is represented as the difference between the current value and the preceding one.

This method is particularly effective when the differences between successive values are smaller and can be represented using fewer bits than the original values.

Delta-of-Delta Compression

Delta-of-Delta compression (also known as "Delta-Delta" or "Delta2"), is an extension of Delta Compression where instead of storing the differences between consecutive values, differences between these differences are stored. In other words, it captures the rate of change in the data.

In the Delta-of-Delta compressed series, the first value is again the same as the first value in the Delta compressed encoded series. The subsequent values represent the differences between consecutive differences.

Delta-of-Delta compression can be particularly effective when dealing with data that has periods of consistent change and periods of constant values, as it efficiently captures these patterns while minimizing the amount of data that needs to be stored or transmitted.

Example

Original series (Unix epoch timestamps):

`1634496000,1634496100,1634496200,1634496299,1634496399`

Series compressed using Delta compression:

`1634496000,+100,+100,+99,+100`

The same series using Delta-of-Delta compression:

`1634496000,+100,0,-1,+1`