Introduction

The database extractor tool will take a CDP Logger (configured with CDPCompactDatastore back-end) database and copy some of the data from there. Unlike a simple file copy, this tool can be run while the logger is still logging, meaning it is also useful for making backups of whole databases.

Motivation

Consider an incident is reported on a remote site. To figure out what happened, it is necessary to analyze the logs. For that, the user might want to remotely log into the system and download the CDP Logger database to later analyze it in the main office. Often the logger database is huge and downloading it will take a long time.

This tool allows extracting some of the data into a separate file. The user can select the time range, list of logged values and even reduce the data resolution. Then the user can take this smaller database file and using third-party tools, compress it and download it. Or in case of poor internet connection, keep the backup of the log and later physically access it on site.

The downloaded database can then be opened by CDP Studio Analyze mode which can also later convert it into CSV format if necessary, see the Historic Data manual for more information. The second option is to open the file using the Database Graph Widget.

Features

• Configure the start and end time of the extraction
• Extract only some of the logged values
• Reduce the data resolution

In addition to extracting data, the tool can print some basic information about the input database:

• List of all logged values
• The time range logged (start and end time)

Usage

Interactive Mode

For interactive mode run as:

./dbextractor InputDatabase.db

Next, the tool will guide you through the supported options. This is most useful when manually logging into the target controller to extract a small part of the logged data.

Non-interactive Mode

For non-interactive mode run as:

./dbextractor InputDatabase.db [options] --destination Output.db

This is most useful when running pre-saved commands or executing the extraction periodically or conditionally through other tools.

Use the following command to list all allowed options of the tool:

./dbextractor --help

Example

The following command will extract data from InputDatabase.db. There is a custom start and end time for the extraction range and data will be written with a lower resolution (0.5 Hz). The list of LoggedValue nodes to extract will be read from nodes.txt which is a file containing the Name of each LoggedValue node separated by newlines. The output data will be written to Output.db.

./dbextractor InputDatabase.db --start-time 2022-03-02T12:00:00.000 --end-time 2022-03-02T12:30:00.000 \
--resolution 0.5 --nodes-to-extract-file nodes.txt  --destination Output.db

Tip! A good way to generate the initial nodes.txt file is to run

./dbextractor InputDatabase.db --list-logged-nodes > nodes.txt

Then manually open the file and delete nodes which you are not interested in.

Alternatives

The DBExtractor command-line tool is most useful for creating periodically or conditionally automatic backups of the CDP Logger data or when the network connection is too poor for downloading large amounts of data.

There is however a quite similar data extraction feature in CDP Studio which allows connecting to a running remote system, plotting relevant data in the Analyze mode and then downloading any data currently visible on the plot directly to your PC. This graphical tool might often be easier to use when investigating an incident, at least when a good network connection is available. See the Historic Data manual for more information.