InfluxDatastore
Introduction
InfluxDB (https://www.influxdata.com/) is an open-source time series database. It is designed to handle high write and query loads and provides a SQL-like query language called InfluxQL for interacting with data.
The CDP Logger InfluxDB integration stores data in an InfluxDB server. The server can be some InfluxDB instance running on some cloud provider or a private InfluxDB server.
Note: The CDP Logger integration was made for version 1.x. Newer InfluxDB 2 will work through the v1 compatibility API as described in InfluxDB 2 Support.
Configuring the Datastore
To use this integration, add InfluxDatastore.InfluxDB from the Resource tree to your CDP Logger component. The following are the most important properties to fill:
| Property | Description |
|---|---|
| URL | Address and port of the InfluxDB server (e.g. http://127.0.0.1:8086). |
| Database | Name of the database to use. Created if one does not already exist and configured user is allowed to create databases. |
| Measurement | InfluxDB measurement used for reading/writing data. An InfluxDB measurement is similar to a SQL database table. |
| UserPwd<string> | Login details for password-protected databases. String in format [user name]:[password]. |
| APIToken<string> | Alternative to UserPwd. The API token is an access key that can be generated in an InfluxDB 2 server. |
In addition, the following properties are available:
| Property | Description |
|---|---|
| LastRequestTime | Time stamp of the last request sent |
| LastStatusCode | Protocol-specific status code the request received by the last response. For example, for HTTPRequest status code 200 means successful request. |
| Interface<string> | Specific interface name (like ETH0, from Application configuration) to be used for request. Useful in multi-interface setup to determine the network device to use for request. Can be left unspecified - then the system default network interface will be used. |
| Proxy<string> | Proxy server to be used for request e.g. http://proxyserver:8080/. Useful in some network setups that require a proxy to be used in order to reach the Internet or special network segment the URL is located in. |
| ProxyUserPwd<string> | Login details for password-protected proxy servers. String in format [user name]:[password]. |
| ProxyAuthMethod | Force use of specific proxy authentication method. The Authentication method is negotiated with the proxy server and used to send the user name and password (specified via ProxyUserPwd property) to the proxy server. Can be one of the following:
|
| FollowLocation<bool> | If set to true follow (re-request) to any new URL that is pointed in the response with HTTP 3xx status code and Location header. The default is not to follow. |
| Encryption | Request connection TLS encryption. Choose between options:
|
| TLSCACertFile<string> | Server certificate issuer (CA) file name, that should contain one or multiple certificates to verify the URL host with. When left empty, then the common public CA bundle from the operating system (OS) will be used for verification. Note, that OS common public CA bundle should be kept up-to-date by updating OS regularly. |
| TLSAllowVerificationErrors | Disable some remote host TLS certificate verifications. Choose between options:
|
| TypeConversion<int> | Controls how CDP data types (int, short, string, etc.) are mapped to InfluxDB data types. Choose between options:
|
| TagForMeasurementName<string> | A tag that sets the Influx Measurement name for a logged value. By default, it will be the logged value parent node's Model. When empty, the name is taken from the Measurement property |
| TagForFieldName<string> | A tag that sets the Influx field name for a logged value. By default, it will be the logged value short name, e.g. "temperature" or "humidity". When empty, uses the LoggedValue's Name. |
Database Format
The CDP Logger creates one measurement (named by the Measurement property) for storing value changes. Each point in the measurement has a field with the changed value, along with the appropriate tags and the timestamp.
The field names are used in the logger and will be visible in external tools to view data. It is recommended to use friendly and meaningful field names. Fields are defined in the LoggedValues table of the CDP Logger. To rename a field, just change the Name property of the LoggedValue.
Each field is tagged with metadata fetched by the CDP Logger TagLookup. In addition, there is a "path" tag which contains the full path of the logged node (defined by the Routing property in the LoggedValues table).
Note: By default, the field name is the name of the logged signal, parameter or property and if there are several fields with the same name, a number is appended to make the field unique. E.g. if there are 3 signals all called Temp, the logger field names will be Temp, Temp1 and Temp2. It is recommended to rename these to meaningful names e.g. outsideTemp, insideTemp…
A second measurement is created for storing metadata of logged values for the CDP Studio Analyze mode integration. It is named by appending "_nodemap" to the Measurement property and it contains the name, routing and all the tags of the logged values.
InfluxDB 2 Support
The CDP Logger integration was made for version 1.x. Newer InfluxDB 2 will work through the v1 compatibility API with the following changes:
- Use the APIToken instead of the UserPwd property.
- The v1 API database (configured by the Database property) will automatically be mapped to a v2 API bucket.
If you have an existing v2 API bucket that doesn't follow the database/retention-policy naming convention, you must manually create a database and retention policy mapping to write data to that bucket with the v1 compatibility API. Note that the CDP Logger uses the default "autogen" retention policy.
For creating a custom mapping, use the InfluxDB CLI and execute:
influx v1 dbrp create \ --db <your-database> \ --rp autogen \ --bucket-id <your-bucket-id> \ --org <your-organization> \ --default
Installing InfluxDB
As mentioned, the CDP Logger connects to a running InfluxDB instance specified by the URL property. This means InfluxDB must be set up first. One option is to use a cloud provider (for example https://aiven.io/). Other option is to install InfluxDB on your own PC. Download it from https://www.influxdata.com/.
After installing and starting InfluxDB, you may now connect to it by filling in the datastore URL (default is 127.0.0.1:8086), Database and Measurement properties in CDP Studio. By default authentication is disabled but if enabled from InfluxDB settings, you must also fill the UserPwd property of the datastore (format is [user name]:[password]). Make sure the user has a right to create databases, otherwise you must configure the datastore to use an already existing database.
Note: The CDP Logger integration was made for version 1.x. Newer InfluxDB 2 will work through the v1 compatibility API as described in InfluxDB 2 Support.
Viewing Logged Data
There is basic integration to access logged data through CDP Logger built-in server (see Viewing Data) but performance may suffer when the database gets large. For huge data sets it is recommended to set up continuous queries aggregating data and plot data using third-party tools (like Grafana).
Note: There is limited support for querying string values from CDP Logger built-in server. Only full-resolution queries are supported. The low-resolution queries that return the min, max and last values over a time span do not work with strings and the entire query will fail even when there is only one string field in the query.
Troubleshooting
Here are some tips to figure out why logging or plotting data is not working:
- Try both InfluxDB Data Explorer tool and CDP Studio Analyze mode for viewing logged data to determine if any data was logged at all.
- When connected to the system with CDP Studio, check the InfluxDB Datastore LastStatusCode property. This HTTP status code gives basic information about logging and reading status (the latter in case a DatabaseGraphWidget or CDP Studio Analyze mode is connected). Read more about status codes in InfluxDB documentation.
- Set the CDP Logger Debug property to
3. This will make the InfluxDB integration print out raw curl requests with their status. Any errors there can be compared with InfluxDB documentation. - Switch the TypeConversion property to Prefer double when logging into an older version of InfluxDB that doesn't support unsigned integers. Note that data types can't be changed without deleting the existing log (either drop the entire Measurement or just delete data for all affected fields in the Measurement).
Get started with CDP Studio today
Let us help you take your great ideas and turn them into the products your customer will love.