LocalLookupService.lookup timeout but works with CloudLookupService.lookup

Question

LocalLookupService.lookup timeout but works with CloudLookupService.lookup

mrlux opened this issue 4 years ago · comments

Hey,

First of all, I want to thank you for creating this library.
I received my Eufy battery doorbell last week and I'm thrilled to see that I can already start playing with it.
I started with the examples repo and everything worked perfectly (HTTP, push).
When trying to connect locally over the p2p connection I hit a timeout issue with the LocalLookupService.
I switched then to the run.ts in this repo and tested the cloud version witch worked like a charm:

const mainP2pCloud = async () => {
  const lookupService = new CloudLookupService();
  try {
    const addresses = await lookupService.lookup(P2P_DID, DSK_KEY);
    console.log('Found addresses', addresses);
  } catch (err) {
    console.error('Not found any address...', err);
  }
};

Found addresses [
  { host: 'xx.xx.xx.xx', port: xxxx },
  { host: '192.168.x.x', port: xxxx }
]

If I use the LocalLookupService:

const mainP2pLocal = async () => {  
  const lookupService = new LocalLookupService();   
  try {
    const address = await lookupService.lookup(LOCAL_STATION_IP);
    console.log('Found address', address);
  } catch (err) {
    console.error('Not found any address...', err);
  }
};

Not found any address... Timeout on address: 192.168.x.x

the 192.168.x.x is the local base station 2 IP address.

Could you point me in a certain direction to solve this issue?
Am I doing something wrong ?
Is it related to my router/network/pihole setup you think?
Could you explain to me what is actually broadcasted to the network?

Thanks in advance.

Jan Löbel · Answer 1 · Mon Nov 30 2020 03:32:41 GMT+0800 (China Standard Time)

@mrlux Thanks for your found. That is very strange... Have you tried to connect to the found address from the CloudLookupService? Can you describe or draw your network setup, the network way from your client running eufy-node-client to the basestation?

There is not really a broadcast, that is the reason why the variable LOCAL_STATION_IP has to be filled with the base station url., e.g.: 192.168.0.101. The client sends a udp packet to a default port to the LOCAL_STATION_IP. The station than sends a udp packet to the clients ip from the port it has open to connect to. So the only thing which could get blocked are the udp packets. The strange thing is, the cloud setup also uses an udp packet and the cloud is responding with the ip addresses.

It is very strange, please try to use the responded address and port to connect to the base station and see if that works.

Wim Meisman · Answer 2 · Tue Dec 01 2020 03:23:57 GMT+0800 (China Standard Time)

@JanLoebel
I tried the following code using the cloud lookup and I'm able to connect to the base station and send the command to the base station using the p2p connection. At the bottom of the script, I try using the local lookup with the same local IP and get the timeout on lookup. (see console output).

import { HttpService } from './http/http.service';
import { CommandType } from './p2p/command.model';
import { DeviceClientService } from './p2p/device-client.service';
import { LocalLookupService } from './p2p/local-lookup.service';
import { CloudLookupService } from './p2p/cloud-lookup.service';
import { PushMessage } from './push/push.model';
import { PushRegisterService, PushClient, sleep } from './push';
import * as fs from 'fs';

(async () => {
  var email = 'xxxxx';
  var password = 'x'
  var baseStationIp = '192.168.x.x'

  const httpService = new HttpService(email, password);
  const hubs = await httpService.listHubs();

  if (hubs && hubs.length > 0) {
    const hub = hubs[0];
    var dsk = await httpService.stationDskKeys(hub.station_sn);

    const lookupService = new CloudLookupService();
    const addresses = await lookupService.lookup(hub.p2p_did, dsk.dsk_keys[0].dsk_key);
    console.log('addresses found:', addresses);
    const address = addresses[1];

    if (baseStationIp == address.host) {
      console.log('baseStationIp is equal to address.host')
    }

    console.log('using address:', address);

    const service = new DeviceClientService(address, hub.p2p_did, hub.member.action_user_id);
    const connected = await service.connect().catch((error) => {
      console.log('error trying to connected:', error);
      return false;
    });

    if (connected) {
      console.log('p2p connected');
      console.log('CMD_SET_ARMING Start:');
      // CMD_SET_ARMING  # 0 => away 1 => home, 2 => schedule, 63 => disarmed
      service.sendCommandWithInt(CommandType.CMD_SET_ARMING, 1);
      console.log('CMD_SET_ARMING End:');
    }

    const localLookupService = new LocalLookupService();
    const localAddress = await localLookupService.lookup(baseStationIp);
    console.log('Found address', localAddress);

  } else {
    console.warn('no hub found');
  }
})();

Console output:

[nodemon] starting `ts-node src/run2.ts`
addresses found: [
  { host: 'in.ter.net.ip', port: xxxxx},
  { host: '192.168.x.x', port: xxxxx }
]
baseStationIp is equal to address.host
using address: { host: '192.168.x.x', port: xxxxx}
p2p connected
CMD_SET_ARMING Start:
CMD_SET_ARMING End:
(node:13348) UnhandledPromiseRejectionWarning: Timeout on address: 192.168.x.x
(Use `node --trace-warnings ...` to show where the warning was created)
(node:13348) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:13348) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Local setup:
Development laptop (192.168.x.x) on same subnet as base station => wifi => Unify dream machine => wired lan => unmanaged switch => wired lan => Base station (192.168.x.x)

Roflware · Answer 3 · Sat Jan 16 2021 05:01:58 GMT+0800 (China Standard Time)

Hi @JanLoebel, I'd also like to thank you for creating this great library! I have a similar issue as @mrlux where the IP times out for me, even though I can ping and port scan it, but I ran into this when running your example P2P script. I'm not as familiar with node in general, but how might I go about replicating the execution of mrlux's script second script that he posted? My end-goal here is to switch the mechanical doorbell from disabled to enabled with CMD_BAT_DOORBELL_MECHANICAL_CHIME_SWITCH = 1703.

Many thanks in advance!

Jan Löbel · Answer 4 · Mon Jan 18 2021 17:43:10 GMT+0800 (China Standard Time)

Okay, that is very strange. I will add an example to the example-project with the cloud lookup and let you know.

Jan Löbel · Answer 5 · Tue Jan 19 2021 05:51:11 GMT+0800 (China Standard Time)

@mrlux @roflware I've added an example to the https://github.com/JanLoebel/eufy-node-client-examples project, see p2p-local-cloud-lookup. Please let me know if that worked for you! Assure to call npm install first.

Roflware · Answer 6 · Tue Jan 19 2021 08:50:03 GMT+0800 (China Standard Time)

@mrlux @roflware I've added an example to the https://github.com/JanLoebel/eufy-node-client-examples project, see p2p-local-cloud-lookup. Please let me know if that worked for you! Assure to call npm install first.

Thanks for putting that together! I regrabbed the example repo, and saw the following messages:

Not found any address... [ 'Timeout on address: {"host":"54.223.148.206","port":32100}', 'Timeout on address: {"host":"18.197.212.165","port":32100}', 'Timeout on address: {"host":"13.251.222.7","port":32100}' ]

However, from this VM I can ping those addresses directly with no issue, but I see that those ports are marked as "open|filtered" via nmap. Is it possible that those ports have changed, or do you have any other idea as to why they're not responding correctly?

Also, I followed the UDP stream with wireshark and got the following request/response for all 3 IPs (edited out my info with your vars instead). If it's of any value, the P2P_DID did not include the middle set of numbers, and it did add "K" before P2P_DID_Part2.

Request: .&.@P2P_DID_PART_1....P2P_DID_PART_2.......................DS_KEY.... Response: .!......

Roflware · Answer 7 · Tue Jan 19 2021 10:20:43 GMT+0800 (China Standard Time)

It's probably best for me to make a separate comment. I learned what the issue was with the P2P-local not working, and it was kind of dumb on my part. I had my VM set to NAT as opposed to bridged. As of now when testing, I get the following:

root@user-virtual-machine:/home/user/eufy-node-client-examples# node examples/p2p-local/index.js Found address { host: '192.168.1.27', port: 13053 }

But, then it just hangs indefinitely and it doesn't seem to ever send my command. In wireshark, it looks like it's just ping/ponging back and forth. Any thoughts for this piece?

Jan Löbel · Answer 8 · Tue Jan 19 2021 20:07:41 GMT+0800 (China Standard Time)

@roflware I will have to take some time to lookup. what is going on on the cloud lookup.

I've added two more console-logs to the example you're running, so please try git pull && npm install before running the script again and attach the full log.

Roflware · Answer 9 · Tue Jan 19 2021 22:15:33 GMT+0800 (China Standard Time)

@roflware I will have to take some time to lookup. what is going on on the cloud lookup.

I've added two more console-logs to the example you're running, so please try git pull && npm install before running the script again and attach the full log.

Yeah, I put in some console.logs in the same spots and saw that it made it there, however, I never saw any changes made when pulling with my Python script to see what parameters have been changed/set (nor did I see any visual changes, such as enabling the LED status light). Here are the logs from running the recent commit you made:

`root@user-virtual-machine:/home/user/user-node-client-examples# node examples/p2p-local/index.js
Found address { host: '192.168.1.27', port: 22457 }
Connected!
Sended command...
^C

root@user-virtual-machine:/home/user/eufy-node-client-examples# node examples/p2p-local-cloud-lookup/index.js
Not found any address... [
'Timeout on address: {"host":"54.223.148.206","port":32100}',
'Timeout on address: {"host":"18.197.212.165","port":32100}',
'Timeout on address: {"host":"13.251.222.7","port":32100}'
]
`

Roflware · Answer 10 · Thu Jan 21 2021 09:30:07 GMT+0800 (China Standard Time)

@JanLoebel Let me know if the above is what you needed, and if not, let me know what other logs you might need.

Roflware · Answer 11 · Tue Jan 26 2021 10:26:04 GMT+0800 (China Standard Time)

Hey @JanLoebel, I can go the route of using Frida if that makes more sense so I can intercept the POST request that I need. Can you tell me what APK version of EufySecurity you used to get it to work with your debug.js with Frida? I'm running into a null pointer dereference issue. Thanks!

Jan Löbel · Answer 12 · Tue Jan 26 2021 16:24:10 GMT+0800 (China Standard Time)

@roflware the point is that after sended command everything should have worked. Only if you change the alarm mode and it is already on that level nothing will happen.

The communication with P2P is not working with HTTP, so you will never see a POST request. Currently I have a lot of other projects and sadly not that much time to investigate further.

You could also check: https://github.com/bropat/ioBroker.eufy-security his implementation is based on this client and seems to work pretty good so far.

Roflware · Answer 13 · Tue Jan 26 2021 21:21:28 GMT+0800 (China Standard Time)

@JanLoebel, I'll check his out, thanks. In yours, Wireshark does not show anything being sent after the send command, but I'll investigate further. But, could you please tell me which APK version you used to get Frida to work? I would appreciate knowing that, thanks. That would still be helpful to me.

Roflware · Answer 14 · Wed Jan 27 2021 05:56:35 GMT+0800 (China Standard Time)

@JanLoebel, just wanted to let you know that the Frida script worked for me once I tried via Nox. I was able to successfully intercept the traffic with Burp, and I got the POST request I needed to execute my desired command. Oddly enough (and this could be due to Eufy updating their framework), but you /can/ update parameters via HTTP, not only P2P.

Jan Löbel · Answer 15 · Wed Jan 27 2021 06:25:28 GMT+0800 (China Standard Time)

@roflware that sounds great! Sorry I wanted to check which version I was using but had no time yet. Can you describe what you wanted to do and how the post request looks like?

Roflware · Answer 16 · Wed Jan 27 2021 06:58:37 GMT+0800 (China Standard Time)

Sure, I trimmed a lot of the fat because not all of the headers are necessary. My goal was to simply reset the mechanical doorbell nightly because for some odd reason, it over time makes the solenoid hang after the first ding for 2 seconds (I suspect that this is a software issue since Eufy support couldn't figure it out even after changing the transformer -- they said that my doorbell chime wasn't supported, but it's literally just an electrical charge going to a solenoid lol). Simply requesting the chime to be mechanical again fixes this issue for me. The below is a sample POST request which would just need the new token after it expires (it can be scripted up to automatically pull a new one easily enough though in Python or whatever you want).

POST /v1/app/upload_devs_params HTTP/1.1
X-Auth-Token: TOKEN
Cache-Control: no-cache
Content-Type: application/json; charset=UTF-8
Content-Length: 115
Host: security-app.eufylife.com
Connection: close
Accept-Encoding: gzip, deflate
User-Agent: okhttp/3.12.1

{"device_sn":"DEVICE_SN","params":[{"param_type":100000,"param_value":"1"}],"station_sn":"STATION_SN"}

10000 = doorbell type
1 = mechanical
2 = digital

For fixing my problem, sending this POST request with the value of 1 resets the doorbell for me and fixes my annoying problem haha.

Roflware · Answer 17 · Fri Feb 05 2021 05:29:12 GMT+0800 (China Standard Time)

Looks like you're right, @JanLoebel. I thought I had it, but there are definitely UDP packets at play out of bounds as to what you would see from Burp, even though it does state what P2P command was being sent (such as 1709, in my case).

I then performed a tcpdump against my device and using your wireshark lua plugin and I finally got it to show the data packets. When compared to your script though, I get the error: "P2P Data Command Return Code: ERROR_INVALID_ACCOUNT (-104)". When comparing the data packets, they appear to be a bit different than when sent from the phone. This could possibly be due to Eufy changing their app a bit as to how it communicates. I'm going to rewrite it in Python and let you know how that goes.

Roflware · Answer 18 · Fri Feb 12 2021 09:25:39 GMT+0800 (China Standard Time)

Hi @JanLoebel, after a lot of troubleshooting, I was finally able to re-write your portion of code into Python with my specific payloads. It turns out with your framework, not all functions work. In my case, CMD_BAT_DOORBELL_MECHANICAL_CHIME_SWITCH and others in the same neighborhood of commands would always yield a "P2P Data Command Return Code: ERROR_INVALID_ACCOUNT (-104)". After performing a TCPDump on my phone, there was an apparent difference in the beginning portion of the data in the packet vs what you have in your framework. Not sure if it's useful to you, but the below hex dump (minus the account, all labeled with ones) is a better representation for how these commands should be sent. I also to committed my code to my GitHub page as well for future reference for those that face a similar issue to mine: https://github.com/roflware/eufy-doorbell-reset-doorbell-chime

Notes: a7 06 = 1703 mechanical chime on/off

Below command turns the doorbell off = 00

0000 f1 d0 00 9c d1 00 00 06 58 5a 59 48 a7 06 88 00
0010 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
0020 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11
0030 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11
0040 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11
0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

But at the end, I realized that for my specific problem (which also had issues being executed like 1703) was 1709, CMD_BAT_DOORBELL_SET_ELECTRONIC_RINGTONE_TIME. This switches between the digital and mechanical chime. Once I do that, it resets the doorbell and solves my issue.