Data storage

Data is stored in several places in IoT Open depending on the category of data in question.

All configuration, user information, installations, devices and their metadata are stored in an SQL cluster and used by the Access and Asset Manager (AAM). User logs are saved in SQL database and on file.

Incoming data enters a framework for processing data streams (see Collecting data) where data can be transformed in real time. Batch processing from TSIs and other storage is also possible as an integration module.

Measurement data and control data are stored in a time series database cluster (TSD) where data is stored quickly accessible for fast processing.

Data is stored in a time series database (TSD) and can, depending on system requirements, undergo thinning in several steps before being saved for the long term.

Depending on installation requirements, a policy for how long data should be stored is quickly available in the TSI. Several steps can be performed before being removed from the system for any long-term storage.

Data can be thinned in one or more steps before it is archived and the way it is done is adapted to current requirements. Examples of thinning can be that repeated identical values ​​are removed or that the resolution is reduced by, for example, consolidation at the mean value.

Data archived on file can be read back to the TSI for analysis on old data, but this is currently done manually. Depending on the selected hardware, the TSI can store very large amounts of data and can be scaled with more hardware or more cluster nodes if necessary.