HS100 Hardware V4.1 Firmware 1.1.0 - No Longer Working - No Port 9999
ghostseven opened this issue · comments
I was not aware that these had an automatic firmware update but it is only what I can assume has happened. I have some HS100 devices that present at hardware version 4.1 (I have other that are 2.0 and 2.1 and all work fine).
Suddenly they now not longer work and NMAP shows nothing on port 9999 any more.
The units work fine in the Kassa app but that is it, I can only assume a firmware update has done this.
Result for UDP nmap scan
blake@nash:~$ sudo nmap -p0-65535 192.168.0.154 -sU
Starting Nmap 7.80 ( https://nmap.org ) at 2020-11-12 08:46 UTC
Nmap scan report for BM-TVConsole.ghost7.com (192.168.0.154)
Host is up (0.0058s latency).
Not shown: 65535 closed ports
PORT STATE SERVICE
20002/udp open|filtered commtact-http
MAC Address: CC:32:E5:A6:E5:64 (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 35.37 seconds
blake@nash:~$
Result for TCP nmap scan
blake@nash:~$ sudo nmap -p0-65535 192.168.0.154 -sT
Starting Nmap 7.80 ( https://nmap.org ) at 2020-11-12 08:48 UTC
Nmap scan report for BM-TVConsole.ghost7.com (192.168.0.154)
Host is up (0.0044s latency).
Not shown: 65535 closed ports
PORT STATE SERVICE
80/tcp open http
MAC Address: CC:32:E5:A6:E5:64 (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 30.08 seconds
blake@nash:~$
The same thing happened to me yesterday morning.
Ok so I have setup a hotspot on a Raspberry Pi and the only device on it is a HS100. This is a pcap from wireshark of everything on the wlan side. I did a turn on and a turn off, to me it looks like nothing is going over port 9999 (which makes sense as it is closed). Though that said it does not make a lot of sense to me as I see nothing coming in over port 80.
I am either doing something wrong (this seems likely) or I am failing to understand the capture properly (also likely).
PCAP in a zip attached.
HS100-H4.1-F1.1.0-On-Off.pcap.zip
Happy to do more tests or run anything that would help debug.
I have a couple of users reporting the same for my OctoPrint plugins (linked above). I'm using the same code to communicate with TPLinkSmartplug devices and appears firmware has finally broken.
Thank you for sharing this. I scrambled quickly to implement some deny rules on my firewall to not allow my plugs to reach out to the internet as a result of this....
Looks like I caught it in time:
{
"system": {
"get_sysinfo": {
"sw_ver": "1.5.2 Build 180611 Rel.080914",
"hw_ver": "2.0",
"type": "IOT.SMARTPLUGSWITCH",
"model": "HS100(US)",
"mac": "zzz",
"dev_name": "Smart Wi-Fi Plug",
"alias": "TP-LINK_Smart Plug_F159",
"relay_state": 0,
"on_time": 0,
"active_mode": "none",
"feature": "TIM",
"updating": 0,
"icon_hash": "",
"rssi": -52,
"led_off": 0,
"longitude_i": 0,
"latitude_i": 0,
"hwId": "xxx",
"fwId": "00000000000000000000000000000000",
"deviceId": "xxx",
"oemId": "xxx",
"next_action": {
"type": -1
},
"err_code": 0
}
}
}
I'm also going quite a ways back here... I do remember messing with some settings quite some time back but the details escape me. A history | grep on one of my PI's did show me trying to override things at some point. I can't remember where I found this, or even if it worked as it does not show up via the getinfo call:
tplink_smartplug.py -t xxx -j '{"cnCloud":{"set_server_url":{"server":"127.0.0.1"}}}'
You can see my intent here was to blackhole any update attempts. Perhaps that's why both of my plugs are still on such older versions firmware wise.
I have a couple of users reporting the same for my OctoPrint plugins
One of these users here 🙂
Doesn't work with OctoPrint, or this library on its own, or https://github.com/python-kasa/python-kasa either. If there is anything I can do to help, I will be happy to assist.
from what I'm seeing this might only be effecting UK models.
Related: home-assistant/core#43088
@ghostseven You might need to put both the phone and the plug on the same network and block internet access to force it to communicate locally.
I may try and get a packet capture later but it will require me to reconfigure some network hardware so I can't do it at the moment.
@dragon2611 completely true, we are now seeing direct activity to port 80 on the device. See attached PCAP. It is a little noisy but just my iphone and the device on the network with internet access blocked.
I'll try and have a look when I'm on a computer, might be using some kind of multicast though
Ok there is some interesting UDP data packets on port 20002. This much better progress now its local only
k<§¿÷°u^õ2EÿÄjÀ¨7À¨7N"á·sÙ[ób{"result":{"ip":"192.168.55.2","mac":"B0-95-75-5E-F5-32","device_id":"5889F7E50D5EBAC24CD4B1829EFAF927","owner":"98E11A6B2B3C68B89EB60704BE5A57C5","device_type":"IOT.SMARTPLUGSWITCH","device_model":"HS110(UK)","hw_ver":"4.1","factory_default":true,"mgt_encrypt_schm":{"is_support_https":false,"encrypt_type":"KLAP","http_port":80}},"error_code":0}
Some initial findings from a quick peek into that pcap file, the message format for discovery is
16B header
Broadcast: 020000010000000000000000463cb5d3 on 20002/udp.
^ checksum, maybe?
Response: 02000001015b000000000000c3b3625c
^ 0x15b = 347 length of the payload excluding the header
The payload is the json encoded info as shown in @ghostseven's comment, which contains encrypt_type (no clue about what KLAP would be) and the port for the management controls. After the discovery, the communication commences using HTTP requests with binary payloads. The requests are HTTP POSTed on /app/request?seq=1307806992 (I first thought seq would be a unix timestamp, but then again this plug would be living in 2011. Seems to be just incremented integer.)
Any ideas how to communicate with udp port?
based on @ghostseven's information it seems the the kasa device is acting like a tapo device on this firmware now. A similar approach seems to be happening on them as implemented here and discussed here.
Some initial findings from a quick peek into that pcap file, the message format for discovery is
16B header Broadcast: 020000010000000000000000463cb5d3 on 20002/udp. ^ checksum, maybe? Response: 02000001015b000000000000c3b3625c ^ 0x15b = 347 length of the payload excluding the headerThe payload is the json encoded info as shown in @ghostseven's comment, which contains
encrypt_type(no clue about what KLAP would be) and the port for the management controls. After the discovery, the communication commences using HTTP requests with binary payloads. The requests are HTTP POSTed on/app/request?seq=1307806992(I first thought seq would be a unix timestamp, but then again this plug would be living in 2011. Seems to be just incremented integer.)
If you know Java then maybe decompiling the android app might yield some further clues?
@dragon2611 I have an 80Mb decompiled zip from the apk and in sources/com/tplinkra/tpcommon/tpclient/klap/TPKLAPClient.java I can see:
HttpClient httpClient = new HttpClient(requestId2, "http://" + this.iotContext.getDeviceContext().getIPAddress() + ":" + 80 + "/app/request?seq=" + sessionInfoByDeviceIdMD5.getSeq());
@dragon2611 I have an 80Mb decompiled zip from the apk and in sources/com/tplinkra/tpcommon/tpclient/klap/TPKLAPClient.java :
HttpClient httpClient = new HttpClient(requestId2, "http://" + this.iotContext.getDeviceContext().getIPAddress() + ":" + 80 + "/app/request?seq=" + sessionInfoByDeviceIdMD5.getSeq());
That looks interesting, I have grabbed the APK and if I get a chance tonight I will decompile and try and replicate some of the app behaviour.
I think we are moving forward so that is a good thing 👍
That looks interesting, I have grabbed the APK and if I get a chance tonight I will decompile and try and replicate some of the app behaviour.
That looks interesting, I have grabbed the APK and if I get a chance tonight I will decompile and try and replicate some of the app behaviour.
Thank you :)
@ghostseven @snecklifter I went through that process over the weekend and managed to get through the new two step handshake to start an encrypted session with my upgraded HS110. Here's a small proof of concept python script demonstrating discovery, handshake and sending one command to get the device info: https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55
I also successfully turned the plug on and off 🎉
In summary:
- UDP broadcast with fixed payload on port 20002 to discover devices.
- Requests are plaintext HTTP on TCP port 80 with encrypted contents
- Two step handshake to determine encryption key and base IV to use for later requests.
- Key and IV seem to be derived from the device email and password. Presumably this is for the account registered for TP-Link cloud functionality. I haven't registered and so the email and password are empty strings. See https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55#file-hs110-py-L76
- AES-128-CBC for request and response body encryption.
- HTTP requests include an incrementing sequence number as a query parameter. This is used to determine the per-request IV.
- Encrypted request and response bodies are prefixed with a HMAC.
- Pre-documented requests seem to work e.g. {"system":{"get_sysinfo":null}}
- There's some time-sensitive state in the plug that results in repeat calls in quick succession to the script generating 403 responses. Not sure what is causing that (too many connections, cookies, &c). Note that each time the script runs it generates a new session cookie with the device.
@chriswheeldon Thanks for that, script confirmed working with my HS110 - Mine's associated with a TPLink account, so I fiddled with this bit and it worked flawlessly.
Edit: formatting
@ghostseven @snecklifter I went through that process over the weekend and managed to get through the new two step handshake to start an encrypted session with my upgraded HS110. Here's a small proof of concept python script demonstrating discovery, handshake and sending one command to get the device info: https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55
I also successfully turned the plug on and off 🎉
In summary:
- UDP broadcast with fixed payload on port 20002 to discover devices.
- Requests are plaintext HTTP on TCP port 80 with encrypted contents
- Two step handshake to determine encryption key and base IV to use for later requests.
- Key and IV seem to be derived from the device email and password. Presumably this is for the account registered for TP-Link cloud functionality. I haven't registered and so the email and password are empty strings. See https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55#file-hs110-py-L76
- AES-128-CBC for request and response body encryption.
- HTTP requests include an incrementing sequence number as a query parameter. This is used to determine the per-request IV.
- Encrypted request and response bodies are prefixed with a HMAC.
- Pre-documented requests seem to work e.g. {"system":{"get_sysinfo":null}}
- There's some time-sensitive state in the plug that results in repeat calls in quick succession to the script generating 403 responses. Not sure what is causing that (too many connections, cookies, &c). Note that each time the script runs it generates a new session cookie with the device.
This is fantastic, literally just started picking through the encryption side and now I don't have to!
Thank you for putting the work in, I will have a play!
I've got a number of these and my experience is that the firmware update only applies to hardware version 4.1, which I believe is exclusively seen in the UK.
Weirdly one of my plugs got updated to Firmware revision 1.1.0 but the other which is also on hardware version 4,1 is still on an old version and even using the App to try to trigger a firmware update doesn't present it as an option. It could be that they've pulled the firmware of it could be something else entirely.
I'll be interested to see how you progress with this issue. Unfortunately it now means my plugs don't work in conjunction with Home Assistant so for the time being i've enabled the TPLink Alexa Skill so I can at least use voice control for some of my equipment.
Can confirm that HW 4.0 (sold in Germany / Central Europe) does not get the FW update just yet, most recent is v1.1.5 which still has port 9999 open (works with HA). Really looking forward to the new mechanism as I guess we will be getting it as well and I really like the TP Link switches.
I actually opened a support ticket with TP-Link because the one that had upgraded to 1.1.0 was showing the wrong time-zone (Dawson/America) and it wouldn't allow me to change it. They have asked for some more information relating to this issue, but interestingly in the same e-mail they have specifically told me to NOT upgrade the other 4.1 device I have.
For the firmware version still in 1.0.4, if the app didn't have the new firmware upgrade available, please keep it. Because the 1.1.0 version was to avoid the potential attacks and security risks with 3rd party software, so currently, you don't need to upgrade it.
Interestingly one of mine has that timezone, I thought I'd messed up readding it after resetting it to test an unpaired plug.
Support just asked me for the Mac of the plugs so I wonder if they're going to push a downgrade
Because the 1.1.0 version was to avoid the potential attacks and security risks with 3rd party software
you mean like this python module...lol.
@ghostseven @snecklifter I went through that process over the weekend and managed to get through the new two step handshake to start an encrypted session with my upgraded HS110. Here's a small proof of concept python script demonstrating discovery, handshake and sending one command to get the device info: https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55
I also successfully turned the plug on and off 🎉
- There's some time-sensitive state in the plug that results in repeat calls in quick succession to the script generating 403 responses. Not sure what is causing that (too many connections, cookies, &c). Note that each time the script runs it generates a new session cookie with the device.
Using your gist, I bound to the IP address of an interface, used a fixed IP address for a specific plug, and tried to narrow down the timing issue.
I removed the time.sleeps from Handshake.
I modified main so it has a retry loop, and tried changing things until the number of retries was 0 or less than the maximum. This code results in no retries, or an average of 2-3 (at least for my network) when the first attempt results in a 403:
if __name__ == '__main__':
s = requests.Session()
# ip = discover()
# print('Found device at {}'.format(ip))
ip = '192.168.88.138'
handshake = Handshake(ip)
retry = 0
while retry < 15:
print("retry: ", retry)
time.sleep(0.25)
encryption = handshake.perform(s)
(msg, seq) = encryption.encrypt('{"system":{"get_sysinfo":null}}')
res = s.post('http://{}:80/app/request'.format(ip), params={'seq': seq}, data=msg)
if res.status_code == 200:
break
retry += 1
assert(res.status_code == 200)
print(encryption.decrypt(res.content))When encryption = handshake.perform(s) is outside of the retry loop, the number of retries is either 0 or 14 (it either succeeds on first try, or never succeeds). The only assert I noticed failing was the last one. Assuming encryption.encrypt() is correct, that leaves s.post().
I just tried adding a 5 second sleep before s.post() and it still resulted in some failures so I think it might be something (session invalidation?) happening on the plug. Does Kasa/cloud send retries or do they always succeed first time?
- Key and IV seem to be derived from the device email and password. Presumably this is for the account registered for TP-Link cloud functionality. I haven't registered and so the email and password are empty strings. See https://gist.github.com/chriswheeldon/3b17d974db3817613c69191c0480fe55#file-hs110-py-L76
I am registered and can confirm filling in my TP-Link/Kaza e-mail address and password on line 81 works, whereas empty strings (and forgetting which password I used) resulted in 403 errors.
It might be possible to get the local API reinstated it seems: https://twitter.com/home_assistant/status/1331003814477000705
I've got a ticket open with them and they've confirmed my device MAC addresses are on their list, but the way it's worded makes it sound like they're working on another firmware which will then be sent to the MAC addresses they have recorded. No update on mine yet but i'll report back if things change.