EventLogReader
Introduction
A Sequencer block for reading CDP event data from CDP Logger or LogServer and exposing it as a message-based API.
Glossary
This documentation refers to two kinds of events:
- CDP events - CDPEventManager events. For example, CDPAlarm Set, Clear and Acknowledge events. These are the events that are logged by CDP Logger, LogWriter and read by EventLogReader.
- Message events - the event inputs and outputs of sequencer blocks as defined by the IEC 61499 standard. In this document, also referred as event messages or just messages.
Usage
To set up an EventLogReader, add it to the configuration and connect it to a CDP Logger or LogServer by setting the ServerAddress and ServerPort properties.
Once connected, the EventLogReader will start continuously emitting Definition messages as new CDP event senders are discovered. The Definition message contains the Path of the CDP event sender and a list of Tags associated with it. To read logged CDP event data, send a ReqEvents message to the EventLogReader and listen for the Event messages.
The following is a block diagram of the typical usage of the EventLogReader block. See below for instructions on how to replicate the example.
Tips
- Combine the EventLogReader with RangeScanner and LogWriter to extract a part of the log to a new Datastore.
- The logs can be quite large, so it is recommended to use the Limit property to read data in batches.
- Disable the IncludeInitialValues property to prevent repeating values at batch borders when extracting data in batches (meaning the RangeEnd emitted with the last ReqDataDone is the RangeStart of the next ReqData). Otherwise, the first row of each batch always contains the initial value of every node.
Configuration
See below for a list of configuration options specific to this block.
Message Events
Name | Type | Description |
---|---|---|
Init | Input | Clears any internal buffers, connects or reconnects to the log server and and starts listening for CDP event definitions. It is mandatory to call this when the WaitForInit property is true . Will emit Inited when done. |
Inited | Output | Will be emitted after Init is done meaning the logger connection is up, known CDP event definitions are sent and the block is ready to serve requests. Note when the WaitForInit property is false , the block will automatically connect at startup and emit Inited. |
Definition | Output | Definition (tags) of a single CDP event sender. Emitted every time a new CDP event sender is found before the matching Event. If Init is called, all definitions are emitted again. For each CDP event sender, emits its Path, Tags and TagSources (if available). |
CountEvents | Input | Counts the number of CDP events that would be returned by ReqEvents with given arguments. When done, will emit either CountEventsDone or Error. Note: unlike ReqEvents, the CountEvent query ignores the Limit and Offset arguments. |
CountEventsDone | Output | Emitted once the CountEvents request is done. The result is set to the Count argument. |
ReqEvents | Input | Requests CDP events between RangeStart and RangeEnd with the specified Limit and Offset for a batch and includes nodes allowed by the PathFilter. When done, will emit either ReqEventsDone or Error. Note: for large databases, extracting small batches by moving RangeStart has better performance than using the Offset argument. |
Event | Output | A single CDP event. Emitted for each CDP event in ReqEvents response. The result is set to the Timestamp, Path, Id, Code, Status, Logstamp, Text, Level, Desc, and EventData arguments. If definitions are available to the CDP event sender then Tags and TagSources arguments are also emitted. |
ReqEventsDone | Output | Emitted once all Event messages are sent. The RangeStart and RangeEnd will include the timestamp of the first and last CDP event in the batch. The Count argument will contain the total number of CDP events emitted. |
Error | Output | Failure when requesting data described by ErrorMsg. |
Reading Tags
When reading data, the associated tags can be accessed in two ways:
- Parse the semicolon-separated list in the Tags argument.
- Add each tag as a separate output argument with the tag name as the argument name. In that case, the tag value will be set to the argument value. Note each user-added argument must be listed in the Data property of the message event.
This affects the Definition and Event messages.
Arguments
Name | Type | Description |
---|---|---|
ServerAddress | Input | IP address (e.g. 127.0.0.1) or NetworkInterface Name (defined under application node, e.g. ETH0) of the CDP Logger's built-in server or LogServer. |
ServerPort | Input | Port of the CDP Logger's built-in server or LogServer. |
RangeStart | Input | Start timestamp for log reading (inclusive when IncludeInitialValues is enabled, otherwise exclusive). When used as input to ReqEvents or CountEvents, leave it empty to read values from the beginning of the log. The unit is specified by the TimeUnit property. |
RangeEnd | Input | End timestamp for log reading (inclusive). When used as input to ReqEvents or CountEvents, leave it empty to read values until the end of the log. The unit is specified by the TimeUnit property. |
Limit | Input | Maximum batch size (the number of CDP events). Reading in large batches will improve performance but also allocate more memory. |
Offset | Input | Offset for batch reading (similar to SQL OFFSET). Leave empty to start from the first CDP event. |
PathFilter | Input | Semicolon-separated list of node paths to read. See the FilterMode property for wildcard support. Leave empty to read all CDP events. |
NewestFirst | Input | If true, the CDP events are returned in the newest-first order. Otherwise, the oldest-first order is used. |
SkipRepriseEvents | Input | If true, the CDP events that are periodically retransmitted by the sender are skipped. Otherwise, all CDP events are returned. |
Count | Output | The number of CDP events returned by the query. |
Timestamp | Output | The Unix timestamp when the CDP event occurred. The unit is specified by the TimeUnit property. |
Path | Output | The path (full Routing) to the node that generated the CDP event. |
Id | Output | The unique identifier for the CDP event. Note the Id is only unique until the application is restarted, so check both the Timestamp and the Id when comparing historical events. |
Code | Output | CDP Event code flags. See below for details. |
Status | Output | The new status of the object that caused the CDP event after the CDP event occurred. For alarms see the Alarm Status Defines in CDPAlarm documentation. |
Logstamp | Output | The time the CDP event was received by the logger component. Note: there is a separate Timestamp argument for the time when the CDP event occurred. Useful when extracting CDP events by small batches as in multi-application system events might arrive out of order. Also useful, when continuously extracting CDP event reprises because for reprises the initial Timestamp is repeated. The unit is specified by the TimeUnit property. |
Text | Output | Optional text sent with the CDP event. |
Level | Output | Optional level data (for example sent with CDPAlarm events). |
Desc | Output | Optional description sent with the CDP event. Describes the node that emitted the CDP event. |
EventData | Output | Semicolon-separated name=value pairs containing other optional data sent with the CDP event. |
Tags | Output | Semicolon-separated list of all tags associated with the node that generated the CDP event. |
TagSources | Output | A semicolon-separated tagname=source list that describes the path where the Tags information was fetched from. |
ErrorMsg | Output | The error message emitted when a request fails. |
CDP Event Code Flags
Common event codes used in the CDP framework (note that multiple flags can be set at the same time):
Name | Code | Description |
---|---|---|
AlarmSet | 0x1 | The alarm's Set flag/state was set. The alarm changed state to "Unack-Set" (The Unack flag was set if not already set). |
AlarmClr | 0x2 | The alarm's Set flag was cleared. The Unack state is unchanged. |
AlarmAck | 0x4 | The alarm changed state from "Unacknowledged" to "Acknowledged". The Set state is unchanged. |
AlarmReprise | 0x40 | A repetition/update of an event that has been reported before. Courtesy of late subscribers. |
SourceObjectUnavailable | 0x100 | The provider of the event has become unavailable (disconnected or similar). |
NodeBoot | 0x40000000 | The provider reports that the CDPEventNode has just booted. |
Properties
Name | Description |
---|---|
Debug | The debug level. Increase it to see more printouts. |
FilterMode | Either ExactMatch or Wildcard. Affects the PathFilter argument behavior. Note: the ExactMatch means the full path must match, use Wildcard for partial match support. |
IncludeInitialValues | Disable to prevent repeating CDP events at batch borders when extracting data in batches. More specifically, this controls whether the RangeStart is inclusive or exclusive when used as input to ReqEvents or CountEvents. |
RangeCriterion | Selects whether RangeStart and RangeEnd refer to Timestamp or Logstamp in queries. The latter is useful when extracting data in small batches as in a multi-application system, the CDP events might arrive out of order to the logger component. Also useful, when continuously extracting CDP event reprises (the SkipRepriseEvents is false ) because for reprises the initial Timestamp is repeated. |
TimeUnit | Unit of the Timestamp argument. Can be set to Second, Millisecond, Microsecond or Nanosecond since epoch (Unix timestamp). Affects Timestamp, RangeStart, and RangeEnd. |
Example
Here is a minimal example using the EventLogReader. The example connects to a CDP Logger running on localhost and extracts all CDP events to a CSV file in small batches and keeps extracting as new CDP events are emitted.
To run the example, start the application and trigger manually the EventLogReader's Init message. The extracted values will be written to a "log-events.csv"
file in the application's working directory.
Here is how to replicate the example:
- Create a system.
- From the Resource tree, add
- CDPLogger.CDPLogger - the built-in logger component which by default logs all CDP events that occur in a system to CDPCompactDatastore.
- Sequencer.RangeScanner - scans the time range by small batches.
- CDPLogger.EventLogReader - reads CDP events from the CDPLogger component. In this minimal example, we have not set any filters, so it will read all CDP events and forward them to the LogWriter.
- CDPLogger.LogWriter - writes the received CDP events to a new Datastore (CSVDatastore in this example).
- Optionally, add two Sequencer.DebugPrinter blocks to print any Error messages.
- Add a CDPLogger.CSVDatastore into the LogWriter block to store extracted CDP events in a CSV file.
- Because we are continuously extracting CDP events, set the RangeCriterion property of EventLogReader to
Logstamp
. Since an alarm reprise event contains an old timestamp (the event is old), we must use the Logstamp that is updated when the event is received to get the RangeScanner to move forward. - Connect the blocks as shown in the image above.
Possible Improvements
Some ideas not implemented in this minimal example:
- Instead of directly extracting to LogWriter on localhost, use Retagger and LogDataAggregator to re-tag and serialize the data for uploading to a cloud service. In the cloud side, deserialize using LogDataDisaggregator before passing the data to LogWriter. This way multiple edge devices can upload data to a central cloud database.
- Replace the CSVDatastore in the LogWriter block with a time-series database such as InfluxDatastore for better performance, scalability and data analysis capabilities.
Get started with CDP Studio today
Let us help you take your great ideas and turn them into the products your customer will love.