go directly to the => Online Monitor (old version) or to the new version
We are creating a monitoring system to integrate data from different solar system manufacturers (like Growatt and Sun/Deye) or battery vendors (like Victron/Pylontech). The concept is open for other domains like heating or water consumption.
Technically we will use a temporal database (InfluxDb) and a report generator (Grafana). The concept is as follows:
- The different solar system installations are called “clients”. They are mapped to correspondingly named Buckets in Influx.
- normal client names, i.e. Buckets start with a country code, followed by a short identifier. We only use [a-zA-Z0-9_], e.g.
ua_rv_01for a client installation somewhere near Rivne in Ukraine. - Maybe in future we will have a metadata bucket named clients describing important properties of each client.
- Measurements are fundamental logic groupings of measured values:
- cur: a momentary (current) value which changes a lot. Can be negative.
- cum: cumulated values, grow over time, always positive
- config: long term configuration data, changes almost never
- The basic data element is called a Field. It has a name and a value which is in most cases a number. FieldNames are written in camelCase syntax, e.g. “hotWater”. Normally FieldValues are float numbers, integers require an “i” at the end (413i is an integer, whereas 413 is a float number)
- Tags are used to characterize the measured values. There is a Tag named type which describes the domain, origin) of measured values and there is another tag named unit to describe the physical unit; we do not use special chars in tag values.
- Note that a certain combination of Bucket, Measurement and Tags can have several Fields for a given Point of Time. The FieldNames are used to specify the character of the field, e.g. “grid” and “pv” for electric power values in Watts for a given timestamp.
- Fields which have identical tags and the same time stamp shall be inserted together in one transaction.
- To improve efficiency when inserting cur and cum measurements which belong together (but need technically two transactions because they the unit tag differs) we use the same time stamp for both transactions. This means that the timestamp must be explicitly specified.
- The insert rate depends on the data sources, it should not be longer than 15 minutes. Precision of timestamps is seconds.
- We have separate Users for each bucket. Users and their access tokens are generated in the ssh using the Influx CLI.
- Regular Grafana queries show the data of a single client installation: Therefore they use the specific Influx User for the bucket
- The admin user can access all buckets and he can define queries which cover more than one installation.
BUCKET clients
name
country
timezone
location
group
----------------------------------------------------
BUCKET de_abc_1
measurement=cum,type=water,unit=m3
values: usage
measurement=cur,type=water,unit=d
values: saltRange
measurement=config,type=eStore,unit=Wh
values: capacity
measurement=cur,type=eStore,unit=Percent
values: batSoc
measurement=cur,type=heating,unit=C
values: furnace, radSet, radTo,
floorSet, floorTo,
hotWater
measurement=cur,type=heating,unit=OnOff
values: radPump, floorPump, tankPump,
HotWaterPump
measurement=cur,type=eSum,unit=W
values: grid,bat,pv,load
measurement=cur,type=eLoad,unit=W
values: heatPumpHotWater, diskWasher,...
measurement=config,type=eProd,unit=Wp
values: panelsEast, panelsWest, panelSouth
measurement=cur,type=eProd,unit=W
values: pv1, pv2,
measurement=cur,type=eStore,unit=W
values: bat1, bat2, ..
measurement=cur,type=eGrid,unit=V
values: ph1, ph2, ph3
----------------------------------------------------
TAGS W, kWh, Percent, d, m3, ..EXAMPLE: de_bn_25 / water / m3 / use_m3 / 17.55 / timestamp
This would be the cold water total value in cubic meters for de_bn_25