Introduction

A Sequencer block for reading data from CDP Logger or LogServer and exposing it as a message-based API.

Usage

To set up a ValueLogReader, add it to the configuration and connect it to a CDP Logger or LogServer by setting the ServerAddress and ServerPort properties.

When extracting data, it is recommended to first call ReqDefinitions to get a list of all logged nodes with their tags and then call ReqData to read the actual data.

The following is a block diagram of the typical usage of the ValueLogReader block. See below for instructions on how to replicate the example.

Tips

• Combine the ValueLogReader with RangeScanner and LogWriter to extract a part of the log to a new Datastore.
• The logs can be quite large, so it is recommended to use the Limit property to read data in batches.
• Disable the IncludeInitialValues property to prevent repeating values at batch borders when extracting data in batches (meaning the RangeEnd emitted with the last ReqDataDone is the RangeStart of the next ReqData). Otherwise, the first row of each batch always contains the initial value of every node.

Configuration

See below for a list of configuration options specific to this block.

Message Events

Reading Tags

When reading data, the associated tags can be accessed in two ways:

• Parse the semicolon-separated list in the Tags argument.
• Add each tag as a separate output argument with the tag name as the argument name. In that case, the tag value will be set to the argument value. Note each user-added argument must be listed in the Data property of the message event.

This affects the Definition and Data messages.

Arguments

Properties

Example

Here is a minimal example using the ValueLogReader. The example ValueLogReader connects to a CDP Logger logging a CPULoad signal and extracts all logged values to a CSV file in small batches and keeps extracting as new values are logged.

To run the example, start the application and trigger manually the ValueLogReader's ReqDefinitions message. The extracted values will be written to a "log.csv" file in the application's working directory.

Here is how to replicate the example:

• Create a system.
• From the Resource tree, add
  • CDPLogger.CDPLogger - the built-in logger component which by default logs values to CDPCompactDatastore.
  • Sequencer.RangeScanner - scans the time range by small batches.
  • CDPLogger.ValueLogReader - reads values from the CDPLogger component. In this minimal example, we have not set any filters, so it will read all values in full resolution and forward them to the LogWriter.
  • CDPLogger.LogWriter - writes the received values to a new Datastore (CSVDatastore in this example).
• Optionally, add two Sequencer.DebugPrinter blocks to print any Error messages.
• Add a CDPLogger.CSVDatastore into the LogWriter block to store extracted values in a CSV file.
• Right-click on the CPULoad signal in the application and use the Add to Logger... menu to add it to the CDPLogger component.
• Connect the blocks as shown in the image above.

Possible Improvements

Some ideas not implemented in this minimal example:

• Instead of directly extracting to LogWriter on localhost, use Retagger and LogDataAggregator to re-tag and serialize the data for uploading to a cloud service. In the cloud side, deserialize using LogDataDisaggregator before passing the data to LogWriter. This way multiple edge devices can upload data to a central cloud database.
• Replace the CSVDatastore in the LogWriter block with a time-series database such as InfluxDatastore for better performance, scalability and data analysis capabilities.
• Set the Resolution (Hz) property to a non-zero value to extract low-resolution data for long-term storage.