Support for Apator Metra E-RM 30 water meter
carpalex opened this issue · comments
Hello,
I'm trying to decode the telegrams coming from some Apator Metra water meters, equipped with the E-RM 30 radio module.
They transmit on 868.9M and I managed to capture some samples at 1M sample rate. The modulation seems to be FSK, but I wasn't able to figure out the data encoding. Here is an archive with samples captured from 6 different meters: samples.zip. The folders in the archive have the following format for their names: SERIALNUMBER_METERVALUE (e.g. 80121968_00002875 contains samples from a water meter with 80121968 serial number and displaying 00002.875 m3 value.
I attempted to use the following flex decoder: -X "n=test,m=FSK_PCM,s=25,l=25,r=5000", which outputs plausible bitstrings.
For one of the meters, I captured samples at consecutive meter values, and there are changes in the bitstrings only at a certain offset, which may mean that the data is not encrypted. For example (meter values are in square brackets):
{239}555555555554d335d89b1d36cadb9c49d4f5a472e12eae14f43bcf7161f4 [00002873]
{239}555555555554d335d89b1d36cad79c49d4f5a472e12eae14f43bcf716ddc [00002874]
{239}555555555554d335d89b1d36ca7b9c49d4f5a472e12eae14f43bcf71c234 [00002875]
{239}555555555554d335d89b1d36ca779c49d4f5a472e12eae14f43bcf71ce1c [00002876]
{239}555555555554d335d89b1d36ca5b9c49d4f5a472e12eae14f43bceabe628 [00002877]
{239}555555555554d335d89b1d36ca579c49d4f5a473732eae14fc3bcea46f18 [00002878]
{239}555555555554d335d89b1d36cbbb9c49d4f5a473732eae14fc3bcea58770 [00002879]
{239}555555555554d335d89b1d36cbb79c49d4f5a473732eae14fc3bcea58b58 [00002880]
{239}555555555554d335d89b1d36cb9b9c49d4f5a473732eae14fc3bcf7fa36c [00002881]
{239}555555555554d335d89b1d36cb979c49d4f5a473732eae14fc3bcf7faf44 [00002882]
I can provide additional samples if needed.
Thanks in advance!
Aligned to 34cd the checksum works as CRC-16 poly 0x8005 init 0x1aaa.
Strange however that only two fields in that alignment change, and not in a linear way.
See this BitBench.
Applying the same alignment to readings from a meter in which the value is constant, it seems that the second field that changes can be some sort of timestamp / sequence number.
(bitbench here - serial: 79909924, meter value: 00000055)
Most likely the first variable field participates in the encoding of the meter value, but in a non-trivial way.
The BitBench link is just the old data. Your findings of a sequence/timestamp at the end are plausible.
There might be a simple key like xor to preserve privacy. A plain text search for the id and counter could work.
Updated the bitbench link, in my previous comment.
Hi, this water meter was already part of this isssue/discussion
Sounds like a data coding is present and need to be decoded to get the figures ...
Hi, this water meter was already part of this isssue/discussion
Sounds like a data coding is present and need to be decoded to get the figures ...
Yes, but in the discussion you mentioned there were no samples provided, nor attempts to analyze the bit strings.
I think it's worth taking another look to try to analyze the message format and identify if/what fields are encrypted or not.
@zuckschwerdt already identified the CRC, I'm trying to figure out where the length can be, but with no success so far.
@merbanan Any ideas on your side?
The length indication should be 24 bytes or less, depending on a sync word and if a header /checksum is included. I don't see a plain number of hex 1x though. It could be encrypted too.
But we already see that the encryption is weak(-ish). There is no counter or IV involved, it feels like a fixed XOR, definitly no S-Box (mixing of bits) or such.
A plain text search does not immediatly support just an XOR mask, but that's still possible.
Logging data for consecutive values over a longer period could provide insight, esp. when bits or bytes overflow to a next column.
What I mean is: if we look at the first variable field and watch the carry then we can adjust the bits so only a set bit does carry.
E.g. try 8h8h 8h8h 8h8h8h ?2b.2b.2b.2b 8h8h 8h8h 8h8h ?8h8h 8h8h ?8h 8h8h ?8d CRC:16h 8x
then use 2a as XOR and the values are ascending.
But the steps greater than 1 in that value, is your reading in gallons or m3? Are there values in between, i.e. different messages for the same reading?
Or there could be a mixing of bit-positions. Looking at that value's bits we see:
both last columns could the 1-bit (toggle with every row), the first and 6th column are MSBs (fixed value), the 4th column is the 2-bit (toggle every 2nd row), the 3rd column is the 4-bit (as in 0x40) and the 2nd column maybe the 8-bit.
Thanks for elaborating. I'll capture more samples for consecutive values and analyze them as you suggested. I'll also post the bit strings.
Regarding your questions, the readings are in m3, with 3 decimal values, i.e possible values from 00000.000 to 99999.999. So I suppose that the encoded value is first multiplied by 1000, i.e from 0 to 99999999, which should require 32 bits.
There are no values in between. If I keep a meter with a constant value, it always transmits the same telegram (apart from the CRC and the field towards the end, which we suspect to be a sort of sequence, and also repeats every few messages).
Related to the samples: can we be sure that the flex decoder parameters are correct (m=FSK_PCM,s=25,l=25,r=5000)? Should we try with different ones?
The samples are FSK and PCM. The bit width looks close enough to 25 µs. rtl_433 will auto-adjust on the preamble (0x5555). And the CRC checks out. All good there. Alignment is fixed by the CRC. The bits could be inverted though.
Also the poly 0x8005 is well known. The init 0x1aaa seems random but perhaps we need to start somewhere else or include some other value to get a nice number, random init's are used though.
Ok, so I managed to collect samples for 65 consecutive meter values, and I think I identified a pattern. Here is the bitbench.
This is what I noticed for the first variable field:
- 1st and 6th column are always zero
- 8th column seems to be a parity bit
- actual information is encoded only in the remaining 5 bits (columns 2, 3, 4, 5 and 7).
Which means that the values for this field should wrap around after 32 iterations, which does happen (it can be seen that they repeat in the same order). However, I couldn't identify a simple transformation between the last 5 LSBs of the counter value and the 5 bits in the field. Maybe it's just a hardcoded lookup table? Any ideas here?
Now, after 32 increments, the carry seems to be in the next 2 bytes, so now I'm collecting samples for increments of 32 of the meter value, maybe I can notice another pattern there.
Do my assumptions seem correct for now?
Yes, perfect. Seeing a (or multiple) full range of the 5 LSBs we can try a table or some clever shuffling.
The overflow clearly marks the first and last value, strange that if we adjust the first value to 0's the last is not all 1's -- not a simple xor then. At least the positions of the bits seem in a fixed order, which I would not expect with a table.
edit: not true, except for the last two which toggle every row the others toggle randomly.
It will be interesting to see if this "5-of-8" coding for the lower byte is then the same in the upper byte.
I think it is a 6in8 coding with some xor mask.
[0]0011[0]01 [0]1111[0]11
The 1st and 6th columns are 0 but the 7th and 8th bits is just something related to the lsb (both bits).
If we take byte 8 and 9 we get 12 bits which is somewhat large enough to encode the meter value.
Plausible. There is however a slight chance that a 16-bit value is encoded and the 1st and 6th column are MSBs?
If there is a linear relation we can solve this by matrix inversion. But going from 4994 to 4995 changes 2 bytes (byte 9 and byte 10 thus we could even have 3 bytes). 4975 to 4976 changes 2 bits, this implies some kind of extra operation ie xor because of bit carry.
Noticed that too, value steps sometimes affect a variable number of coded bits. It's very much not linear. But also bit columns seem fixed, not random like an S-Box.
@carpalex I think we need more values.
If there is a linear relation we can solve this by matrix inversion. But going from 4994 to 4995 changes 2 bytes (byte 9 and byte 10 thus we could even have 3 bytes). 4975 to 4976 changes 2 bits, this implies some kind of extra operation ie xor because of bit carry.
At a value step, I noticed that at least 2 coded bits are changing. Could it be that the 8th column is just a parity bit?
@carpalex I think we need more values.
I captured more samples here, with increments of 32 of the meter value, so that the 5 LSBs of the meter are value are constant, and we can see how the higher order bits are changing. So far, no clear pattern is seen , but I'll continue capturing samples (I only have 17 now, hoping to see a repetition after 32 or 64).
I gathered enough samples (with increments of 32) to observe a pattern. There is a 4-bit group that changes at each line, and they start repeating after 16 such iterations (captured the repetition twice for confirmation).
So most probably the 4-bit group encodes the next 4 LSBs of the meter value. But the transformation is again strange, maybe another lookup table?
Now for observing what happens further, we would need samples with increments of 512 of the meter value. I will try to capture them, but it would take a longer time. For now, we see that they overflow at the left of the 4-bit group.
With the info we have so far, (the LSB in clear text, but at a strange offset, then the next 2 groups of 4 LSBs each encoded with different lookup tables, and again at strange offsets), does this start to look like something? Or is it just arbitrary?
We have not seen any similar scheme where bits are somewhat randomly encrypted but at fixed positions. It's oddly weak. Something like AES with a fixed key or plain XOR would look very different. Seems like rather easy to map out and break, unless the position or key is per device.