This is a simple application that reads information from Dutch smart power meters via the so called P1 port and sends them to an MQTT gateway. See the website of Netbeheer Nederland for information on the protocol and message format.
Connection is possible either via direct serial connection to the meter, or via an intermediate device that presents the P1 telegrams via a TCP connection.
The P1 protocol supports a bus of meters, where measurements from multiple meters can be deliviered through a single telegram. A common setup is to report on both electricity and gas consumption.
In case multiple meters deliver data in a single telegram, multiple separate measurements will be sent to MQTT, one for each meter.
This project uses pyserial to communicate with the serial port, and paho-mqtt for talking to MQTT.
Installation via pip into a venv is possible with pip install . from
the git checkout root, or via pip install git+https://github.com/Lalufu/p1-mqtt.
This will also create the executable scripts in the bin dir of the venv.
In case you want to do things manually, the main entry point into
the program is p1_mqtt/cli.py:p1_mqtt().
This project uses Poetry for dependency
management, and it's probably easiest to use this, Executing poetry install followed by poetry run p1-mqtt from the git checkout root should
set up a venv, install the required dependencies into a venv and run
the main program.
--config
: Specify a configuration file to load. See the section Configuration file
for details on the syntax. Command line options given in addition to the
config file override settings in the config file.
--host
: Host name or IP of device connected to the P1 meter
--port
: TCP port of device connected to the P1 meter
--device
: Device file of the serial port connected to the P1 meter
Config file: Section general, device
--mqtt-host
: The MQTT host name to connect to. This is a required parameter.
Config file: Section general, mqtt-host
--mqtt-port
: The MQTT port number to connect to. Defaults to 1883.
Config file: Section general, mqtt-port
--buffer-size
: The size of the buffer (in number of measurements) that can be locally
saved when the MQTT server is unavailable. The buffer is not persistent,
and will be lost when the program exits. Defaults to 100000.
Config file: Section general, buffer-size
--mqtt-topic
: The MQTT topic to publish the information to. This is a string that is put
through python formatting, and can contain references to the variables device_id
and channel. device_id will contain the serial number of the meter, which
is part of the P1 telegram.
channel will contain the channel assigned to a meter. The main electricity
meter will use channel 0, while other meters on the bus use channels 1
and up.
The default is p1-mqtt/tele/%(channel)s/%(device_id)s/SENSOR.
Config file: Section general, mqtt-topic
--mqtt-client-id
: The client identifier used when connecting to the MQTT gateway. This needs
to be unique for all clients connecting to the same gateway, only one
client can be connected with the same name at a time. The default is
p1-mqtt-gateway.
Config file: Section general, mqtt-client-id
--mqtt-rate
: The time between mqtt messages sent to the broker in seconds.
The default is 0 (which means as soon as a telegram is ready).
Config file: Section general, mqtt-rate
--dsmr-22
: Use DSMR 2.2 compatible parameters when setting up the serial port. This
might be needed for older meters, try this when the default settings do
not produce any data. The default is to use DSMR 4.0 and later parameters.
--serial-dump
: Takes a file name, and will record all data received from the serial
port into the given file. This is mainly useful to capture data for
debugging.
Config file: Section general, serial-dump
--prefer-local-timestamp
: Use the time from the machine running p1-mqtt as the authoritative
time stamp on the data sent to MQTT, instead of the time stamp contained
in the telegrams themselves.
--time-ms
: Send p1mqtt_* time stamp values to MQTT in milliseconds instead of
seconds.
The program supports a configuration file to define behaviour. The
configuration file is in .ini file syntax, and can contain multiple sections.
The [general] section contains settings that define overall program
behaviour.
[general]
device = /dev/ttyUSB0
mqtt-client-id = p1-gateway-01
mqtt-host = mqtt.example.com
serial-dump = /tmp/dumpfile
The script pushes the data received from the P1 meter to MQTT as a JSON
string. Information received from the meter(s) are presented as values
under keys starting with p1_.
In addition the following fields are added:
-
a
p1mqtt_telegram_timestampfield is added, containing the time the measurement was taken. This information is contained in the P1 telegram, and relies on the clock in the meter. Note that not all meters update values at the same frequency that telegrams are sent, multiple subsequent telegrams might contain the same timestamp and readings. -
a
p1mqtt_collector_timestampfield is added, containing the time the measurement was taken. This information is taken from the clock of the machine running p1-mqtt, and might be different from the time inp1mqtt_telegram_timestamp. -
a
p1mqtt_timestampfield is added, containing either of thep1mqtt_telegram_timestampandp1mqtt_collector_timestampvalues. The value chosed depends on the--prefer-local-timestampcommand line argument. For easier management this is the value that should be used as the authoritative time stamp in further processing of the data. -
a
p1mqtt_device_idfield is added, containing the serial number of the meter that this set of measurements belongs to. -
a
p1mqtt_channelfield with added, containing the channel number of the meter that this set of measurements belong to.
To minimize delay in reading and processing data, the script tries to detect the size of a single telegram sent by the meter, and to read exactly one telegram at once. If it succeeds in doing so, it considers itself to be 'in sync'. Changes in telegram size, which should be rare, result in messages to the console. These are usually harmless, and mainly for informational purposes. Since the script does not know the telegram size at startup there will be at least some messages about loss of sync, and resync, during startup. This is normal.
Using the default parameters, the telegram
/Ene5\XS210 ESMR 5.0
1-3:0.2.8(50)
0-0:1.0.0(171105201324W)
0-0:96.1.1(4530303437303030303037363330383137)
1-0:1.8.1(000051.775*kWh)
1-0:1.8.2(000000.000*kWh)
1-0:2.8.1(000024.413*kWh)
1-0:2.8.2(000000.000*kWh)
0-0:96.14.0(0001)
1-0:1.7.0(00.335*kW)
1-0:2.7.0(00.000*kW)
0-0:96.7.21(00003)
0-0:96.7.9(00001)
1-0:99.97.0(0)(0-0:96.7.19)
1-0:32.32.0(00002)
1-0:32.36.0(00000)
0-0:96.13.0()
1-0:32.7.0(229.0*V)
1-0:31.7.0(001*A)
1-0:21.7.0(00.335*kW)
1-0:22.7.0(00.000*kW)
0-1:24.1.0(003)
0-1:96.1.0(4730303538353330303031313633323137)
0-1:24.2.1(171105201000W)(00016.713*m3)
!8F46
will result in two messages being sent to MQTT.
The first message will be sent to the topic
p1-mqtt/tele/0/E0047000007630817/SENSOR and consist of the below
data:
{
"p1mqtt_channel": 0,
"p1mqtt_device_id": "E0047000007630817",
"p1mqtt_timestamp": 1509909204,
"p1_actual_power_consuming": 0.335,
"p1_actual_power_consuming_l1": 0.335,
"p1_actual_power_producing": 0.0,
"p1_actual_power_producing_l1": 0.0,
"p1_current_l1": 1.0,
"p1_energy_consumed_tariff1": 51.775,
"p1_energy_consumed_tariff2": 0.0,
"p1_energy_produced_tariff1": 24.413,
"p1_energy_produced_tariff2": 0.0,
"p1_long_power_failure_count": 1.0,
"p1_power_failure_count": 3.0,
"p1_timestamp": 1509909204,
"p1_voltage_l1": 229.0,
"p1_voltage_sag_l1_count": 2.0,
"p1_voltage_swell_l1_count": 0.0
}
The second message will be sent to the topic
p1-mqtt/tele/1/G0058530001163217/SENSOR and consist of the below
data:
{
"p1mqtt_channel": 1,
"p1mqtt_device_id": "G0058530001163217",
"p1mqtt_timestamp": 1509909000,
"p1_device_type": 3.0,
"p1_gas_consumed_timestamp": 1509909000,
"p1_gas_consumed_volume": 16.713
}