SSL support

Question

SSL support

anteph opened this issue 9 years ago · comments

Hi!
I would like to know if you plan to include SSL support in the libraries.
I've tryed to send some https requests with no sucess.

It would be cool if it the ESP could act as a secure server too.

I've checked the some examples from the sdk and they have a code to create a secure server with digital certificate.

Thanks!

Geo commented 9 years ago

+1

Alexander · Answer 1 · Thu Apr 23 2015 04:47:28 GMT+0800 (China Standard Time)

SSL has been ~broken since 0.9.3-patch1 (and you needed a new libssl, separate from patch1, from Espressif.)

It's supposedly fixed again in 1.0.1b2 (you still need a special libssl for the moment) - http://bbs.espressif.com/viewtopic.php?f=5&t=382 - I haven't had a chance to try it yet, but I've heard good reports.

There are still some features missing for practical SSL - iirc there's no way to pin a certificate or do trust validation, it'll silently accept self-signed certs - but it's still a huge positive step.

Markus · Answer 2 · Thu Apr 23 2015 21:56:49 GMT+0800 (China Standard Time)

i have add the latest sdk + ssl patches
see here:
Links2004@af6c400
pull request is waiting for merged (#93)

Ivan Grokhotkov · Answer 3 · Thu Apr 23 2015 22:27:40 GMT+0800 (China Standard Time)

Merged that, thanks.
Just one (obvious) thing to note: adding SSL libraries (axTLS) is one thing, but to make them useful we also need to add SSL support to WiFiClient/WiFiServer.

Victor Aprea · Answer 4 · Thu Jun 18 2015 06:29:24 GMT+0800 (China Standard Time)

@igrr does this issue cover adding HTTPS support to a WebClient application for accessing secure-connection-only APIs? What's the roadmap/timeline for including this feature? Is it actually possible?

Victor Aprea · Answer 5 · Sat Jun 27 2015 08:43:59 GMT+0800 (China Standard Time)

@Links2004 @igrr Do I read this thread correctly that there is support for an HTTPS WebClient built into the core now, but WiFiClient class "just" needs that core functionality integrated into it?

Alexander · Answer 6 · Sat Jun 27 2015 09:22:32 GMT+0800 (China Standard Time)

@vicatcu:

does this issue cover adding HTTPS support to a WebClient application for accessing secure-connection-only APIs?

Yes - HTTPS is just HTTP over SSL sockets, so once SSL sockets are integrated with WiFiClient/WiFiServer, you should be able to talk to HTTPS servers.

What's the roadmap/timeline for including this feature?

I'm not very familiar with this project's codebase.

Is it actually possible?

Yes. Other projects (like https://github.com/tuanpmt/esp_bridge) are able to make HTTPS requests from an ESP8266. The primary limiting factor is certificate size and protocol support (TLS v1.2 is not yet supported and is mandatory for some servers) - and that the Espressif SDK doesn't always have working SSL built in to it (it works now!)

Do I read this thread correctly that there is support for an HTTPS WebClient built into the core now, but WiFiClient class "just" needs that core functionality integrated into it?

It's now possible to write a WiFiClient that opens a secure socket. The existing WebClient shouldn't need major changes.

Victor Aprea · Answer 7 · Sat Jun 27 2015 09:32:14 GMT+0800 (China Standard Time)

@abl cool, I need to come up to speed on on getting a dev environment set up so I can contribute...

Glenn Algie · Answer 8 · Mon Jul 20 2015 09:46:24 GMT+0800 (China Standard Time)

looking forward to SSL support as well for ESP8266. I am looking at using Golgi.io arduino stack as an option for the SSL shim as they provide other time saving and mission critical networking support functions.

Victor Aprea · Answer 9 · Sun Jul 26 2015 05:25:00 GMT+0800 (China Standard Time)

@igrr @Links2004 can you give any pointers on how to use axTLS to implement HTTPS capability into WiFiClient/WiFiServer? Any updates on progress here would be appreciated. I personally would like to see the WiFiClient capability implemented as the priority. Would it be beneficial splitting this issue into two; one for WiFiClient and one for WiFiServer?

Ivan Grokhotkov · Answer 10 · Sun Jul 26 2015 06:25:56 GMT+0800 (China Standard Time)

@vicatcu
WiFiClient needs only TLS, not HTTPS. You can use WiFiClient for things other than HTTP — SSH and SMTP to name a few. The way to do this is to create a new class, let's say WiFiSecureClient, and override read/write methods of WiFiClient to pump data through TLS engine. Some additional methods for certificate management should be added as well. I'm not sure if there is anything that needs to be done on the server side, since WiFiServer's only job is to create a WiFiClient instance for every new connection. I suppose someone more familiar with TLS can comment on this.

Victor Aprea · Answer 11 · Sun Jul 26 2015 06:30:11 GMT+0800 (China Standard Time)

@igrr ah yes of course, that (that WiFiClient is not just about HTTP) makes a lot of sense.

fmgomes · Answer 12 · Thu Jul 30 2015 23:04:19 GMT+0800 (China Standard Time)

@vicatcu , have you started to implement the TLS on the WiFiClient? I need to connect to a webserver using https, and I'm interested in this functionality, and could try to help (don't know if I am able to do it, no previous experience with HTTPS or TLS).

Best regards

Fernando

Victor Aprea · Answer 13 · Thu Jul 30 2015 23:24:01 GMT+0800 (China Standard Time)

@fmgomes honestly I haven't really gotten anywhere yet - it's a pretty steep learning curve for me so I'm kind of in the same boat as you :-/. It's a bit frustrating to know it's possible for the last few months, but that I haven't had time / know-how / etc to help make it happen. @igrr spelled out what needs to happen pretty well in his last response, but I'm not sure what "certificate management" requires...

Alexander · Answer 14 · Fri Jul 31 2015 01:39:05 GMT+0800 (China Standard Time)

When a client connects to a server and performs a TLS handshake (which is what turns a socket in to a secure socket, and HTTPS is just HTTP on a secure socket) the server sends over a certificate. The certificate is what identifies the server; without the certificate validation step, HTTPS prevents against eavesdropping but you could potentially be communicating with an attacker (aka a MitM attack.)

Validating a certificate:

Check to see if the name of the server you're trying to connect to matches the server name on the certificate.
Check to see if the certificate has expired.
Check to see if the certificate was issued by a trusted CA.
Check to see if the certificate has been revoked.

1 should be pretty easy to implement; the names should be exactly equal. Wildcard certificates exist but are considered harmful; not supporting them is probably fine for v1.

2 can probably be ignored for now; the vast majority of ESP8266 work will be bound to specific certificates because...

3 is the most complex part. On a normal computer, the OS (and sometimes the browser) maintains a list of trusted root certificates. For Debian, the size of this certificate package is 502kB (see https://packages.debian.org/sid/ca-certificates) - it's possible to shrink this down a bit but it's obviously a nonstarter for the average ESP8266 project. An alternative to this is simply maintaining a list of certificate fingerprints and checking that instead - creating an explicit whitelist of certificates. That's what I'd recommend - and when you're using certificate fingerprints, you can be lax about expiration and revocation and manage it yourself.

4 is a complicated topic - check out CRL and OCSP if you're curious - and way beyond what we'd want to do in an embedded system.

AxTLS probably has simple-ish calls to handle all of this; they'd need to be exposed via SecureWifiClient.

SecureWifiServer would need a new method that allows a user to set the public and private key to use.

mtnbrit · Answer 15 · Fri Jul 31 2015 03:16:26 GMT+0800 (China Standard Time)

does thread this also cover applying TLS to MQTT eventually?

anteph · Answer 16 · Sun Aug 02 2015 21:44:42 GMT+0800 (China Standard Time)

For what I've read, adding https support is a little bit hard. But what about simple ciphering, for direct socket communication? Something with a symmetric key, that would be a good start.

I've read in the Espressif page that ESP8266 has a built in AES engine, but they don't specify if it is hardware or software based. Has anyone tried it?

Anthony Elder · Answer 17 · Sun Aug 02 2015 23:17:41 GMT+0800 (China Standard Time)

Just a comment on that list of work from @abl - to get going i'd be quite happy to not have any of that certificate validation stuff. Wont be sending my bank account details or anything top secret just need to post to a remote service that only supports https. So just binning the certificate after its read would be fine for my uses.

drmpf · Answer 18 · Mon Aug 03 2015 05:19:39 GMT+0800 (China Standard Time)

If you just want to prevent un-authorized use rather then hid the messages, check out this page on using SipHash as a secure hash.
http://www.forward.com.au/pfod/secureChallengeResponse/index.html

Matt Kocaj · Answer 19 · Fri Aug 28 2015 10:58:51 GMT+0800 (China Standard Time)

So what's the latest?

Can I make a HTTPS request?
can I use TLS 1.2?
can I pin a certificate (avoid the CA lookups)?

Deleted user · Answer 20 · Sat Aug 29 2015 02:58:21 GMT+0800 (China Standard Time)

Is it now possible to make a HTTPS request?
Or are there any libraries out there that I could use?

Ivan Grokhotkov · Answer 21 · Mon Aug 31 2015 14:29:04 GMT+0800 (China Standard Time)

Started working on TLS support: https://github.com/igrr/axtls-8266
This is the same library that Espressif uses for their libssl, so TLS 1.0/1.1 only.

Alexander · Answer 22 · Thu Sep 03 2015 02:24:04 GMT+0800 (China Standard Time)

Going that route, it seems like wolfSSL and mbed TLS (well, especially mbed TLS) might be easier to bring in, although we know axTLS works since Espressif has it working :)

They also support TLS 1.2.

Ivan Grokhotkov · Answer 23 · Thu Sep 03 2015 02:28:20 GMT+0800 (China Standard Time)

Both wolfSSL and PolarSSL (aka mbed TLS) are GPL (not LGPL) unless you
apply for a commercial license. I would like to keep the library
LGPL-compatible.

On Wed, Sep 2, 2015, 21:24 Alexander notifications@github.com wrote:

Going that route, it seems like wolfSSL and mbed TLS (well, especially
mbed TLS) might be easier to bring in, although we know axTLS works since
Espressif has it working :)

They also support TLS 1.2.

—
Reply to this email directly or view it on GitHub
#43 (comment).

Alexander · Answer 24 · Thu Sep 03 2015 02:32:40 GMT+0800 (China Standard Time)

Ah, I missed that on WolfSSL.

For PolarSSL/mbed, they have an explicit exception list that includes the LGPL: https://tls.mbed.org/foss-license-exception

The language appears to prevent the distribution of a non-GPL fork of mbed TLS but has other odd quirks. Certainly a larger headache than axTLS's BSD-ish license.

Matt Kocaj · Answer 25 · Thu Sep 10 2015 10:44:54 GMT+0800 (China Standard Time)

I can't believe this conversation has come to licencing..

Ivan Grokhotkov · Answer 26 · Mon Sep 14 2015 15:36:51 GMT+0800 (China Standard Time)

WiFiClientSecure is mostly ready now. Things left to do:

seed PRNG properly
check host name against certificate's common name and alternative DNS names (wildcards are quite common, 4 out of 5 services I used for my tests do use wildcards).
release memory allocated for certificate storage after check is complete

Matt Kocaj · Answer 27 · Mon Sep 14 2015 16:25:43 GMT+0800 (China Standard Time)

@igrr Coz I'm new, what is the simplest and quickest way to get your changes onto my esp8266?

Victor Aprea · Answer 28 · Mon Sep 14 2015 21:59:11 GMT+0800 (China Standard Time)

@igrr that's awesome, I'm looking forward to trying it out. Thanks for your work on this!

Ivan Grokhotkov · Answer 29 · Tue Sep 15 2015 06:14:51 GMT+0800 (China Standard Time)

@cottsak You can either build IDE from source or wait a bit for the next staging version.

Matt Kocaj · Answer 30 · Tue Sep 15 2015 09:38:15 GMT+0800 (China Standard Time)

@igrr So that's it? Once built, run the IDE from the build output location and the WiFiClientSecure changes will be available to my sketch and should upload to my esp8266 no problem?

Matt Kocaj · Answer 31 · Thu Sep 17 2015 15:36:14 GMT+0800 (China Standard Time)

@igrr you mean https://github.com/esp8266/Arduino not https://github.com/arduino/Arduino/wiki/Building-Arduino right?

Ivan Grokhotkov · Answer 32 · Thu Sep 17 2015 15:38:38 GMT+0800 (China Standard Time)

That link (https://github.com/arduino/Arduino/wiki/Building-Arduino) was to instructions for building the IDE. Do a git clone of this repository and follow the instructions from that link.

Edit: note that if you have boards manager package installed in your system, it will take precedence over the core bundled with the IDE. So once you build from source, be sure to go into boards manager and remove the installed esp8266 core.

Matt Kocaj · Answer 33 · Thu Sep 17 2015 16:27:29 GMT+0800 (China Standard Time)

@igrr Now that I've built macosx/arduino-1.6.6-macosx.zip, how do i manually add the esp8266 board to my IDE installation since I can't use the Board Manager?

Ivan Grokhotkov · Answer 34 · Thu Sep 17 2015 16:32:38 GMT+0800 (China Standard Time)

If you have done git clone of github.com/esp8266/Arduino, your IDE will have ESP8266 support built-in.
Just run it with open build/macosx/work/Arduino.app.

Matt Kocaj · Answer 35 · Thu Sep 17 2015 16:33:33 GMT+0800 (China Standard Time)

@igrr I had to remove the esp8266 board from my default IDE install. Then when I started the IDE from the build output the esp8266 board was back but with the latest source changes.

Matt Kocaj · Answer 36 · Thu Sep 17 2015 16:39:22 GMT+0800 (China Standard Time)

Finally got the HTTPSRequest example working and it seemed to connect to Github. However, it then proceeds to appear to crash and dump some debugging context to serial. Anyone have any idea about this?


connecting to itsatrap
........
WiFi connected
IP address: 
10.1.1.11
connecting to api.github.com
certificate matches
requesting URL: /repos/esp8266/Arduino/commits/esp8266/status
request sent
headers received
esp8266/Arduino CI successfull!
reply was:
==========
{"state":"success","statuses":[{"url":"https://api.github.com/repos/esp8266/Arduino/statuses/89df2854c8e3e5923b600bcfd70afbaa9964cdfe","id":309793032,"state":"success","description":"The Travis CI build passed","target_url":"https://travis-ci.org/esp8266/Arduino/builds/80635331","context":"continuous-integration/travis-ci/push","created_at":"2015-09-16T13:34:51Z","updated_at":"2015-09-16T13:34:51Z"}],"sha":"89df2854c8e3e5923b600bcfd70afbaa9964cdfe","total_count":1,"repository":{"id":32969220,"name":"Arduino","full_name":"esp8266/Arduino","owner":{"login":"esp8266","id":8943775,"avatar_url":"https://avatars.githubusercontent.com/u/8943775?v=3","gravatar_id":"","url":"https://api.github.com/users/esp8266","html_url":"https://github.com/esp8266","followers_url":"https://api.github.com/users/esp8266/followers","following_url":"https://api.github.com/users/esp8266/following{/other_user}","gists_url":"https://api.github.com/users/esp8266/gists{/gist_id}","starred_url":"https://api.github.com/users/esp8266/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/esp8266/subscriptions","organizations_url":"https://api.github.com/users/esp8266/orgs","repos_url":"https://api.github.com/users/esp8266/repos","events_url":"https://api.github.com/users/esp8266/events{/privacy}","received_events_url":"https://api.github.com/users/esp8266/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/esp8266/Arduino","description":"Arduino IDE for ESP8266","fork":false,"url":"https://api.github.com/repos/esp8266/Arduino","forks_url":"https://api.github.com/repos/esp8266/Arduino/forks","keys_url":"https://api.github.com/repos/esp8266/Arduino/keys{/key_id}","collaborators_url":"https://api.github.com/repos/esp8266/Arduino/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/esp8266/Arduino/teams","hooks_url":"https://api.github.com/repos/esp8266/Arduino/hooks","issue_events_url":"https://api.github.com/repos/esp8266/Arduino/issues/events{/number}","events_url":"https://api.github.com/repos/esp8266/Arduino/events","assignees_url":"https://api.github.com/repos/esp8266/Arduino/assignees{/user}","branches_url":"https://api.github.com/repos/esp8266/Arduino/branches{/branch}","tags_url":"https://api.github.com/repos/esp8266/Arduino/tags","blobs_url":"https://api.github.com/repos/esp8266/Arduino/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/esp8266/Arduino/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/esp8266/Arduino/git/refs{/sha}","trees_url":"https://api.github.com/repos/esp8266/Arduino/git/trees{/sha}","statuses_url":"https://api.github.com/repos/esp8266/Arduino/statuses/{sha}","languages_url":"https://api.github.com/repos/esp8266/Arduino/languages","stargazers_url":"https://api.github.com/repos/esp8266/Arduino/stargazers","contributors_url":"https://api.github.com/repos/esp8266/Arduino/contributors","subscribers_url":"https://api.github.com/repos/esp8266/Arduino/subscribers","subscription_url":"https://api.github.com/repos/esp8266/Arduino/subscription","commits_url":"https://api.github.com/repos/esp8266/Arduino/commits{/sha}","git_commits_url":"https://api.github.com/repos/esp8266/Arduino/git/commits{/sha}","comments_url":"https://api.github.com/repos/esp8266/Arduino/comments{/number}","issue_comment_url":"https://api.github.com/repos/esp8266/Arduino/issues/comments{/number}","contents_url":"https://api.github.com/repos/esp8266/Arduino/contents/{+path}","compare_url":"https://api.github.com/repos/esp8266/Arduino/compare/{base}...{head}","merges_url":"https://api.github.com/repos/esp8266/Arduino/merges","archive_url":"https://api.github.com/repos/esp8266/Arduino/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/esp8266/Arduino/downloads","issues_url":"https://api.github.com/repos/esp8266/Arduino/issues{/number}","pulls_url":"https://api.github.com/repos/esp8266/Arduino/pulls{/number}","milestones_url":"https://api.github.com/repos/esp8266/Arduino/milestones{/number}","notifications_url":"https://api.github.com/repos/esp8266/Arduino/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/esp8266/Arduino/labels{/name}","releases_url":"https://api.github.com/repos/esp8266/Arduino/releases{/id}"},"commit_url":"https://api.github.com/repos/esp8266/Arduino/commits/89df2854c8e3e5923b600bcfd70afbaa9964cdfe","url":"https://api.github.com/repos/esp8266/Arduino/commits/89df2854c8e3e5923b600bcfd70afbaa9964cdfe/status"}
==========
closing connection

ctx: cont 
sp: 3ffeb720 end: 3ffeb9a0 offset: 01b0

>>>stack>>>
3ffeb8d0:  401000b4 3fff5178 3ffe9404 4022c4bb  
3ffeb8e0:  3ffe9268 3fff3180 3fff318c 4022ae7a  
3ffeb8f0:  3ffe9268 3fff6578 3fff5178 4022a060  
3ffeb900:  3ffe93f3 00000000 3fff2ed8 40204572  
3ffeb910:  402075f9 3ffe93e8 3ffeb930 40203f60  
3ffeb920:  3ffe9268 3ffe93e8 3ffeb9f8 4020245e  
3ffeb930:  3ffe95a8 00000000 000003e8 00008df3  
3ffeb940:  00000000 3fff2ea0 3fff2ed8 3fff2de0  
3ffeb950:  00000012 00000012 3fff9150 000011ad  
3ffeb960:  000011ad 3fff3248 0000002d 0000002d  
3ffeb970:  00000000 00000000 00000000 3ffeb9cc  
3ffeb980:  3fffdc20 00000000 3ffeb9c4 4020185f  
3ffeb990:  00000000 00000000 3ffea980 40100398  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(1,6)


 ets Jan  8 2013,rst cause:4, boot mode:(1,6)

wdt reset

Ivan Grokhotkov · Answer 37 · Thu Sep 17 2015 16:55:07 GMT+0800 (China Standard Time)

Uhm, it worked fine when I wrote this sample. Will check today, perhaps the commit from yesterday broke something.

Anthony Elder · Answer 38 · Thu Sep 17 2015 19:14:43 GMT+0800 (China Standard Time)

The sample crashes like that for me too, a couple of seconds after the request has completed ok, but using WiFiClientSecure in other code seems to work ok for me. Had a play around with the sample - moving the line "WiFiClientSecure client;" outside of the setup() function to the top of sketch looks like it fixes the problem. Some garbage collection problem?

Matt Kocaj · Answer 39 · Fri Sep 18 2015 09:40:06 GMT+0800 (China Standard Time)

I can verify that @torntrousers workaround also prevents the crash for me.

Yeh, sounds like a memory-related problem to me too. There's no compile time error since client isn't referenced in void loop() but maybe under the hood something else is trying to access the instance and because it's not in global scope? it's barfing.. ?? Just a guess.

Matt Kocaj · Answer 40 · Fri Sep 18 2015 15:13:56 GMT+0800 (China Standard Time)

I'm still getting more of these crashes.. they seem to be host-dependant. ie, some hosts don't trigger them and some do.

Here is another dump:

Sending alarm request..

ctx: cont 
sp: 3ffeb7b0 end: 3ffebac0 offset: 01b0

>>>stack>>>
3ffeb960:  401000b4 3fff5298 3ffebb8c 4022c86b  
3ffeb970:  40208bb7 3fff35b8 3fff35c4 4022b22a  
3ffeb980:  000001bb 3fff6698 3fff5298 4022a410  
3ffeb990:  3ffebb80 00000000 3fff2ff8 40204922  
3ffeb9a0:  40208ec4 000001bb 3ffea690 40204828  
3ffeb9b0:  3ffe92f7 40208eac 3ffea690 40203b0e  
3ffeb9c0:  dbca1032 3ffe92f7 402017ec 3ffeaaa0  
3ffeb9d0:  3ffebb18 000001bb 3ffea690 402047e0  
3ffeb9e0:  3ffe96e8 dbca1032 3ffe92f5 3ffebaec  
3ffeb9f0:  3ffebb18 000001bb 3ffea690 40203ad5  
3ffeba00:  3ffe96e8 dbca1032 3ffe96e8 dbca1032  
3ffeba10:  3ffe92f7 00000001 3ffebb18 402070a0  
3ffeba20:  3ffebb18 3ffea690 3ffea7ac 40204800  
3ffeba30:  3ffea6ac 00000000 3ffea7ac 40202370  
3ffeba40:  3ffe93ce 00000000 3ffebb18 4020651d  
3ffeba50:  3ffe9274 3ffebaec 3ffebb18 402070a0  
3ffeba60:  402010ae 000003e8 3ffebb18 4020710c  
3ffeba70:  3fffdc20 00000000 3ffebb18 3ffebaec  
3ffeba80:  3fffdc20 00000000 3ffea7ac 402025ad  
3ffeba90:  4020180d 00000000 3ffebae4 402022e0  
3ffebaa0:  00000000 00000000 3ffebae4 4020186a  
3ffebab0:  00000000 00000000 3ffeaaa0 40100398  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(1,6)


 ets Jan  8 2013,rst cause:4, boot mode:(1,6)

wdt reset

Matt Kocaj · Answer 41 · Fri Sep 18 2015 15:53:20 GMT+0800 (China Standard Time)

Also, is this a problem:

GET /a/check HTTP/1.1
Host: www.howsmyssl.com
User-Agent: ESP8266
Connection: close

HTTP/1.1 200 OK
Content-Length: 568
Access-Control-Allow-Origin: *
Connection: close
Content-Type: application/json
Date: Fri, 18 Sep 2015 07:50:29 GMT
Strict-Transport-Security: max-age=631138519; includeSubdomains; preload

{"given_cipher_suites":["TLS_RSA_WITH_AES_128_CBC_SHA","TLS_RSA_WITH_AES_256_CBC_SHA","TLS_RSA_WITH_RC4_128_SHA","TLS_RSA_WITH_RC4_128_MD5"],"ephemeral_keys_supported":false,"session_ticket_supported":false,"tls_compression_supported":false,"unknown_cipher_suite_supported":false,"beast_vuln":false,"able_to_detect_n_minus_one_splitting":false,"insecure_cipher_suites":{"TLS_RSA_WITH_RC4_128_MD5":["use RC4 which has insecure biases in its output"],"TLS_RSA_WITH_RC4_128_SHA":["use RC4 which has insecure biases in its output"]},"tls_version":"TLS 1.1","rating":"Bad"}

Particularly the RC4 stuff and that the client is only supporting TLS 1.1?

Ivan Grokhotkov · Answer 42 · Fri Sep 18 2015 16:40:14 GMT+0800 (China Standard Time)

Found the cause of the crash, fix coming a bit later today.

Regarding only TLS 1.1 being supported — please read the thread above,
axTLS doesn't support 1.2. Regarding cipher suites — there is still much
work to do, i.e. add SHA256, disable outdated stuff... Any help is
appreciated.

On Fri, Sep 18, 2015, 10:53 Matt Kocaj notifications@github.com wrote:

Also, is this a problem:

GET /a/check HTTP/1.1
Host: www.howsmyssl.com
User-Agent: ESP8266
Connection: close

HTTP/1.1 200 OK
Content-Length: 568
Access-Control-Allow-Origin: *
Connection: close
Content-Type: application/json
Date: Fri, 18 Sep 2015 07:50:29 GMT
Strict-Transport-Security: max-age=631138519; includeSubdomains; preload

{"given_cipher_suites":["TLS_RSA_WITH_AES_128_CBC_SHA","TLS_RSA_WITH_AES_256_CBC_SHA","TLS_RSA_WITH_RC4_128_SHA","TLS_RSA_WITH_RC4_128_MD5"],"ephemeral_keys_supported":false,"session_ticket_supported":false,"tls_compression_supported":false,"unknown_cipher_suite_supported":false,"beast_vuln":false,"able_to_detect_n_minus_one_splitting":false,"insecure_cipher_suites":{"TLS_RSA_WITH_RC4_128_MD5":["use RC4 which has insecure biases in its output"],"TLS_RSA_WITH_RC4_128_SHA":["use RC4 which has insecure biases in its output"]},"tls_version":"TLS 1.1","rating":"Bad"}

Particularly the RC4 stuff and that the client is only supporting TLS 1.1?

—
Reply to this email directly or view it on GitHub
#43 (comment).

Matt Kocaj · Answer 43 · Mon Sep 21 2015 09:08:18 GMT+0800 (China Standard Time)

Thanks @igrr. I'd love to help but much of this c++ networking stack stuff is over my head. I'm only just managing to consume these libs it in my code.

Tim Kay · Answer 44 · Tue Sep 29 2015 10:39:56 GMT+0800 (China Standard Time)

Running the staging version, I get the error below. Is this error the same as @cottsak reported above? It doesn't look to be.

I tried to run the latest code. I cloned the latest code, installed ant, compiled the Arduino IDE, extracted arduino-1.6.6-windows.zip. It won't run, saying,

java.lang.UnsatisfiedLinkError: C:\Users\timkay\Desktop\arduino-1.6.6\lib\AStylej.dll:
Can't load IA 32-bit .dll on a AMD 64-bit platform

Error running staging version

connected with 688 Berry Ave, channel 11
dhcp client start...
ip:192.168.8.120,mask:255.255.255.0,gw:192.168.8.254
connecting
State:  sending Client Hello (1)
State:  receiving Server Hello (2)
Error: invalid protocol message
Alert: handshake failure
Alert: close notify
connection failed
Fatal exception (28):
epc1=0x40204011, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, d
Exception (28):
epc1=0x40204011 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

ctx: cont 
sp: 3ffeb5e0 end: 3ffeb840 offset: 01a0

&gt;&gt;&gt;stack>>>
3ffeb780:  3fff2560 40202ee4 3ffeb898 40206bb0  
3ffeb790:epc=0x00000000
  00000000 3ffeb898 3ffea540 40202540  
3ffeb7a0:  6c61686d 7766686c 00736177 00000000  
3ffeb7b0:  3ffeb898 00000001 3ffe92e5 40206a70  
3ffeb7c0:  3ffeb800 00000011 3ffe9285 3ffeb86c  
3ffeb7d0:  3fffdc20 00000006 3ffeb898 40205fc1  
3ffeb7e0:  3ffe92e4 00000000 3ffeb898 3ffeb86c  
3ffeb7f0:  3fffdc20 00000000 3ffeb898 40206b44  
3ffeb800:  00000000 00000000 00000016 40101a6d  
3ffeb810:  4020180d 00000000 3ffeb898 4020249c  
3ffeb820:  00000000 00000000 3ffeb864 4020186a  
3ffeb830:  00000000 00000000 3ffea820 40100398  
<<<stack<<<

Matt Kocaj · Answer 45 · Tue Sep 29 2015 10:54:16 GMT+0800 (China Standard Time)

@timkay Staging is still too old to have included @igrr's latest changes. I would try to help with your buid adventure but I was on OS X and it seemed to work for me.

Grahame Horner · Answer 46 · Thu Oct 01 2015 03:09:49 GMT+0800 (China Standard Time)

@cottsak @igrr I've installed a build from the https://www.arduino.cc/en/Main/Software hourly build link; if staging is to old; is there a more current build of the board package available? to developers for the arduino IDE (hourly build)?

Ivan Grokhotkov · Answer 47 · Thu Oct 01 2015 03:22:16 GMT+0800 (China Standard Time)

I've just updated staging to ef26c5f, so now you can just try and pull the latest boards manager package.

Grahame Horner · Answer 48 · Thu Oct 01 2015 04:14:57 GMT+0800 (China Standard Time)

@igrr your the man! great many thanks 👍

Tim Kay · Answer 49 · Thu Oct 01 2015 23:22:27 GMT+0800 (China Standard Time)

@igrr Up and running now with TLS working, talking to Firebase. Thank you!

Ivan Grokhotkov · Answer 50 · Thu Oct 01 2015 23:24:37 GMT+0800 (China Standard Time)

Good to know! Don't put this into anything critical yet though, because random number generator is not seeded properly. I'll fix this in the next release.

Grahame Horner · Answer 51 · Thu Oct 01 2015 23:53:50 GMT+0800 (China Standard Time)

just completed my first RESTful call to Azure Service Bus over HTTPS; thanks 💃 +1

N0TB0T · Answer 52 · Sat Oct 03 2015 23:19:38 GMT+0800 (China Standard Time)

I'm sorry, but I still get a "'WiFiClientSecure' was not declared in this scope" - the most current version of Staging (packaged) seems to be 1.6.5-1160-gef26c5f, which I installed over the BoardManager. Doesn't it work when just adding the link in the IDE Settings, do I have to build it on my own? Any help appreciated, thx

N0TB0T · Answer 53 · Sat Oct 03 2015 23:20:09 GMT+0800 (China Standard Time)

And @igrr - Thanks for the effort on SSL anyway, that's exactly what I needed!

Tim Kay · Answer 54 · Sat Oct 03 2015 23:37:52 GMT+0800 (China Standard Time)

You need another include, as I show here.

#include <ESP8266WiFi.h>
#define USE_SSL 1
#if USE_SSL
#   include <WiFiClientSecure.h>
    WiFiClientSecure client;
#   define PORT 443
#else
    WiFiClient client;
#   define PORT 80
#endif

N0TB0T · Answer 55 · Sat Oct 03 2015 23:40:41 GMT+0800 (China Standard Time)

Found that out that very moment, digging into ESP8266WiFi.c - =)

Still: Thanks a lot!

Tim Kay · Answer 56 · Tue Oct 06 2015 13:22:58 GMT+0800 (China Standard Time)

I can't seem to open two WiFiClientSecure connections at the same time. The second connect() causes a WDT reset. See https://www.dropbox.com/s/i8pin2m34shhxn1/esp_test_two_tls.ino?dl=0

scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 

connected with 688 Berry Ave, channel 11
dhcp client start...
ip:192.168.8.121,mask:255.255.255.0,gw:192.168.8.254
About to connect() to www.yahoo.com port 443 uri /error
connected
About to connect() to www.yahoo.com port 443 uri /error
E:M 1048

Soft WDT reset

ctx: cont 
sp: 3ffeb410 end: 3ffeba20 offset: 01b0

>>>stack>>>
3ffeb5c0:  3fffb8e0 401000a2 3fff9a40 4022c0a7  
3ffeb5d0:  3fff9128 3fff9a60 00000081 4022cd0d

Ivan Grokhotkov · Answer 57 · Tue Oct 06 2015 13:30:12 GMT+0800 (China Standard Time)

With two connections you are running out of memory, so only one connection
at a time for now. I think Espressif's version of axTLS shares this
limitation.

On Tue, Oct 6, 2015, 08:23 Timothy Kay notifications@github.com wrote:

I can't seem to open two WiFiClientSecure connections at the same time.
The second connect() causes a WDT reset. See
https://www.dropbox.com/s/u6qma92jwaa65pr/esp_test.ino?dl=0

scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt

connected with 688 Berry Ave, channel 11
dhcp client start...
ip:192.168.8.121,mask:255.255.255.0,gw:192.168.8.254
About to connect() to www.yahoo.com port 443 uri /error
connected
About to connect() to www.yahoo.com port 443 uri /error
E:M 1048

Soft WDT reset

ctx: cont
sp: 3ffeb410 end: 3ffeba20 offset: 01b0

stack>>>
3ffeb5c0: 3fffb8e0 401000a2 3fff9a40 4022c0a7
3ffeb5d0: 3fff9128 3fff9a60 00000081 4022cd0d

—
Reply to this email directly or view it on GitHub
#43 (comment).

Tim Kay · Answer 58 · Tue Oct 06 2015 13:31:01 GMT+0800 (China Standard Time)

I also seem to have trouble reading a stream with packets larger than about 1500 bytes. I turned on DEBUGV and get error :er -9 53 1, which is ERR_RST, but I can't find the code that makes that call. See https://www.dropbox.com/s/1d43d0li1cv7io5/esp_test_big_packet.ino?dl=0

scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 

connected with 688 Berry Ave, channel 11
wifi evt: 0
dhcp client start...
wifi evt: 3
ip:192.168.8.121,mask:255.255.255.0,gw:192.168.8.254
About to connect() to demonstration.firebaseio.com port 443 uri /bigpacket.json
:ref 1
:wr
:sent 56
:ww
:rn 1460
:rd 5, 1460, 0
:rdi 1460, 5
:rd 74, 1460, 5
:rdi 1455, 74
:rd 5, 1460, 79
:rdi 1381, 5
:rd 1376, 1460, 84
:rdi 1376, 1376
:c0 1376, 1460
:rn 1460
:rd 1460, 1460, 0
:rdi 1460, 1460
:c0 1460, 1460
:rn 362
:rd 353, 362, 0
:rdi 362, 353
:rd 5, 362, 353
:rdi 9, 5
:rd 4, 362, 358
:rdi 4, 4
:c0 4, 362
:wr
:sent 267
:ww
:wr
:sent 6
:ww
:wr
:sent 69
:rn 75
:ww
:rd 5, 75, 0
:rdi 75, 5
:rd 1, 75, 5
:rdi 70, 1
:rd 5, 75, 6
:rdi 69, 5
:rd 64, 75, 11
:rdi 64, 64
:c0 64, 75
connected
:wr
:sent 53
:ww
:wr
:sent 69
:ww
:wr
:sent 53
:ww
:wr
:sent 53
:ww
:wr
:sent 85
:ww
:wr
:sent 53
:ww
:wr
:sent 53
:ww
.:rn 229
:rch 229, 1460
:rch 1689, 1460
:rch 3149, 1460
:rd 5, 4609, 0
:rdi 229, 5
.:rd 224, 4609, 5
:rdi 224, 224
:c 224, 229, 4609
*** rb: 0 + 187 = 187
[128]HTTP/1.1 200 OK
Content-Length: 12842
Strict-Transport-Security: max-age=31556926; includeSubDomains; preload
Content-Type: a:rd 5, 4380, 0
:rdi 1460, 5
:wr
:sent 53
:rch 4380, 794
:ww
:wr
:er -9 53 1
:ww
[59]pplication/json; charset=utf-8
Cache-Control: no-cache

Tim Kay · Answer 59 · Tue Oct 06 2015 22:47:57 GMT+0800 (China Standard Time)

I updated the previous comment with more information.

Using https://www.dropbox.com/s/krjrv69czlglsuh/esp_test_mid_packet.ino?dl=0, I find that it works if the data length is <= 5067, but it gives an error:

WiFiClientSecure rx overflow*** rb: 0 + 1535 = 1535

Tim Kay · Answer 60 · Wed Oct 07 2015 11:46:36 GMT+0800 (China Standard Time)

Thanks! Could we get you to update staging?

Ivan Grokhotkov · Answer 61 · Wed Oct 07 2015 12:52:26 GMT+0800 (China Standard Time)

That commit doesn't fix the issue you have reported.

Tyler Horvath · Answer 62 · Sat Oct 10 2015 11:27:57 GMT+0800 (China Standard Time)

I'm unsure of what is really happening - but it seems like any Amazon Cloudfront SSL doesn't work with axTLS.

Their supported list of ciphers are:
ECDHE-RSA-AES128-SHA
ECDHE-RSA-AES256-SHA
AES256-SHA
AES128-SHA
DES-CBC3-SHA
RC4-MD5

now the above post shows the current library supports RC4-MD5 (I believe) - but I'm not sure how to prioritize it in the handshake? That is, until SHA256 is added :)

Matt Kocaj · Answer 63 · Tue Oct 13 2015 11:25:07 GMT+0800 (China Standard Time)

@igrr Still getting crashes when making POST over HTTPS sporadically:

making POST request to Twilio for sending sms..

Soft WDT reset

ctx: cont 
sp: 3ffeb800 end: 3ffebe10 offset: 01b0

>>>stack>>>
3ffeb9b0:  4022bea3 00000030 00000010 4022c973  
3ffeb9c0:  bd92dada 3fff3f88 3fff9ed8 4022cb0c  
3ffeb9d0:  40101635 3fffb238 3fffb238 3fff3f88  
3ffeb9e0:  00000080 00000102 3fff9ba0 4022cdbe  
3ffeb9f0:  3fff5288 ffffff81 00000000 00000081  
3ffeba00:  000218f5 0000007f 00000000 00000081  
3ffeba10:  3fffaa10 3fff3f88 3fff3b88 3fff40b0  
3ffeba20:  3fffb64c ffffffff 3fff3f88 3fff3f88  
3ffeba30:  00000080 3fff3f88 3fff9ba0 3fff3f88  
3ffeba40:  00000080 3fff3f88 3fff9ba0 4022d926  
3ffeba50:  3fff4050 0000007f 5aecfcb7 00000001  
3ffeba60:  3fff3f88 3fff9b80 3fff9ba0 00000001  
3ffeba70:  00000010 3fff3f88 0000000f 4022db81  
3ffeba80:  3fff3b88 3fff4090 0000000f 00000001  
3ffeba90:  00000001 3fff40f0 00000000 4022c9d8  
3ffebaa0:  00000200 3fff5c53 3fff5c53 00000000  
3ffebab0:  00000200 3fff5c53 3fff4018 4022ed2c  
3ffebac0:  00000000 000001d0 3ffebaf0 00000030  
3ffebad0:  066ce839 ac113df1 4b66993c 00000004  
3ffebae0:  3fff7020 3fff5c20 3fff5c4d 4022ad82  
3ffebaf0:  ec980203 86ffba2d 8634bd31 842a26f6  
3ffebb00:  6a6dbffb 54a77dc7 5fd8c70d a6b83ed6  
3ffebb10:  86c3f654 3986aa77 05c35ee6 9efce561  
3ffebb20:  3fff5c4d 3fff7020 3fff5c20 4022b115  
3ffebb30:  337c38c1 5a926727 1cf0d2cd e36493c8  
3ffebb40:  e4c18e1c 29db7716 0b99662c 25956359  
3ffebb50:  2ef9f8a0 3fff3c60 3fff3c20 00001140  
3ffebb60:  00000004 3fff5c4d 00000003 3fff3c60  
3ffebb70:  3fff5c20 3fff5c4d 00000004 00000004  
3ffebb80:  3fff5c4d 3fff7020 3fff5c20 4022abaa  
3ffebb90:  00000038 00000000 3fff3ab8 402048fa  
3ffebba0:  00000038 00000001 00000033 3fff5c48  
3ffebbb0:  00000000 00020010 0000000e 00000010  
3ffebbc0:  00000004 401000a2 3ffebd10 40204958  
3ffebbd0:  3fff7020 00000033 00000016 00000000  
3ffebbe0:  3fff7020 3fff5c20 3fff5c20 4022ace6  
3ffebbf0:  3fff7020 3fff5c4d 3fff5c20 4022aebd  
3ffebc00:  00000000 3fff3af0 3fff5c20 4022af46  
3ffebc10:  3fff5600 40205822 3ffebd10 3ffead0c  
3ffebc20:  3fff3b60 3fff3848 3ffebd10 4020475a  
3ffebc30:  384ff336 3ffebd10 402017ec 3ffebd10  
3ffebc40:  3ffebd10 000001bb 3ffebd10 402049cc  
3ffebc50:  3ffe9900 384ff336 3ffec2d0 40104b54  
3ffebc60:  40102e58 000001bb 3ffebd10 40203ca9  
3ffebc70:  3ffe9900 384ff336 3ffe9900 384ff336  
3ffebc80:  3ffe93f2 00000038 00000000 7fffffff  
3ffebc90:  0000007f 3ffe93f2 3ffebe68 402049e4  
3ffebca0:  3ffebd10 40240000 3ffebe68 40202448  
3ffebcb0:  6c4e5551 6a646a59 3545324e 6a56474e  
3ffebcc0:  68686a59 684a6d5a 324d6a5a 7a4d7a59  
3ffebcd0:  3551574f 6b5a544e 7a45444e 326f6a4e  
3ffebce0:  68686a5a 6964544e 31417a4e 78597a4e  
3ffebcf0:  6946574e 32457a59 31416a59 6d5a7a4d  
3ffebd00:  33553259 3045474d 3d41414f 40202400  
3ffebd10:  3ffe9790 00000000 000003e8 00000001  
3ffebd20:  00000000 3fff3ab8 3fff3848 00000040  
3ffebd30:  00000040 00000001 3ffebd90 40207ed2  
3ffebd40:  3fff3a58 00000040 3ffebd90 40207f2a  
3ffebd50:  30312e30 00000000 3ffebd90 40207f5a  
3ffebd60:  3ffebd90 40240000 3ffebd90 4020805c  
3ffebd70:  3ffe9634 40240000 00000000 3ffe8000  
3ffebd80:  3ffeaa98 40240000 00000000 402026e3  
3ffebd90:  3fff3728 0000004e 00000040 3fff3728  
3ffebda0:  0000003f 00000036 3fff3a58 0000004f  
3ffebdb0:  00000040 3ffeaad0 3ffebe68 3ffebe3c  
3ffebdc0:  3fffdc20 3ffeaad0 3ffebe68 3ffebe3c  
3ffebdd0:  3ffea998 00000000 3ffea998 4020290e  
3ffebde0:  3fffdc20 00000000 3ffebe34 402022f8  
3ffebdf0:  00000000 00000000 3ffebe34 4020186a  
3ffebe00:  00000000 00000000 3ffeadf0 40100398  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1264, room 16 
tail 0
chksum 0x42
csum 0x42
~ld

Source is https://github.com/cottsak/opensesameseed/blob/master/iforgottocloseit/iforgottocloseit.ino

Grahame Horner · Answer 64 · Wed Oct 14 2015 22:36:20 GMT+0800 (China Standard Time)

this may help; just prior to the crash the debug output is :-

making POST request to Twilio for sending sms..
chg_B:-60
E:M 536

Soft WDT reset
...

making POST request to Twilio for sending sms..
State: sending Client Hello (1)
State: receiving Server Hello (2)
State: receiving Certificate (11)
State: receiving Server Hello Done (14)
E:M 1048

Matt Kocaj · Answer 65 · Thu Oct 15 2015 09:10:46 GMT+0800 (China Standard Time)

@grahamehorner what are those? did you mod my code and dump that?

Ivan Grokhotkov · Answer 66 · Fri Oct 16 2015 04:12:14 GMT+0800 (China Standard Time)

@timkay Sorry I haven't been able to follow this issue; still on my vacation.

I tried your sketch with a bit more debug info enabled, and it turned out axTLS is unable to process TLS record because the buffer size is not sufficient. TLS record from demonstation.firebaseio.com comes with size=8240, while local buffer is 5115 bytes (this is hard-coded in my axTLS build at the moment, RT_MAX_PLAIN_LENGTH is set to 4096). So axTLS sees that it can't receive the whole record and bails out.

I'll see if I can make this local buffer size configurable at run time. Default (16k per TLSv1 spec) is way too large, we don't have enough RAM to handle that.

Workaround: try to configure max TLS record size on you server to be less than 4k.
https://www.igvita.com/2013/10/24/optimizing-tls-record-size-and-buffering-latency/

Ivan Grokhotkov · Answer 67 · Fri Oct 16 2015 05:50:42 GMT+0800 (China Standard Time)

@timkay I have implemented Maximum Fragment Length Negotiation extenstion in my axTLS build. Sadly though, it seems that server doesn't support it and sends large records anyway.

Tim Kay · Answer 68 · Fri Oct 16 2015 06:32:23 GMT+0800 (China Standard Time)

For the record, firebaseio.com isn't my server, it is a service provided by Google. Firebase is very popular among IoT users. For example, it is the back end used by Nest. It will be a shame if we can't figure out a way for the ESP8266 to talk to it.

I will reach out to them to see if they can reconfigure at their end... I posted a question to Stack Overflow.

N0TB0T · Answer 69 · Fri Oct 16 2015 18:35:31 GMT+0800 (China Standard Time)

Just curious:

Did anybody manage to use SSL with parse.com? It's basically used the same way as firebaseio, besides mostly for apps...

I did not get a call through, unfortunately.

Matt Kocaj · Answer 70 · Sat Oct 17 2015 23:19:54 GMT+0800 (China Standard Time)

Can someone point me in the direction of doco or something that I can use to begin learning how to interpret crash dumps?


 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1264, room 16 
tail 0
chksum 0x42
csum 0x42
~ld

I have no idea what this means and anything would be useful at this point. Thanks

Ivan Grokhotkov · Answer 71 · Sun Oct 18 2015 03:49:15 GMT+0800 (China Standard Time)

Output posted by @grahamehorner indicates that there is an out-of-RAM condition.
@cottsak the ets Jan 8 2013 (and all the following) stuff isn't related to the crash, so just ignore it. The stack dump you have posted above is valuable. Could you upload your compiled sketch (called iforgottocloseit.cpp.elf) somewhere for analysis?

Edit: i gave your sketch a shot, but it seems i need an account with Twilio to actually try anything.

Sabas · Answer 72 · Wed Oct 21 2015 02:42:14 GMT+0800 (China Standard Time)

Support TLS 1.2?

Ivan Grokhotkov · Answer 73 · Wed Oct 21 2015 02:44:50 GMT+0800 (China Standard Time)

@sabas1080 as mentioned a few comments above, axTLS only supports TLS 1.1

Sabas · Answer 74 · Wed Oct 21 2015 02:58:58 GMT+0800 (China Standard Time)

Thanks @igrr , i have the next error in the API Telegram

IP address: 
192.168.1.65
GET Update Messages 
E:M 536

Soft WDT reset

ctx: cont 
sp: 3ffebc20 end: 3ffec1d0 offset: 01b0

stack>>>
3ffebdd0: 0f610597 21146ab8 e562ac99 4022c764

3ffebde0: ecfae377 c2e09b1a 694e65f8 3fff5710

3ffebdf0: 00000040 00000082 3fffa390 4022c9ee

3ffebe00: 3fffbc40 401000a2 00000000 00000041

3ffebe10: 0002df36 0000003f 00000000 00000041

3ffebe20: 7b70a77e 00000200 3fffa3b0 3fff5838

3ffebe30: 3fffbd4c ffffffff 3fff5710 3fff5710

3ffebe40: 00000040 3fff5710 3fffa390 3fff5710

3ffebe50: 00000040 3fff5710 3fffa390 4022d556

3ffebe60: 3fff57d8 0000003f cb13b5ec 00000020

3ffebe70: 3fff5320 3fff5710 3fff5858 00000020

3ffebe80: 3fff5320 3fff5710 00000010 4022d83c

3ffebe90: 3fff5878 3fff5818 3fff5858 00000001

3ffebea0: 00000001 3fff5858 00000000 4022c608

3ffebeb0: 00000100 3fff73eb 3fff73eb 00000000

3ffebec0: 00000100 3fff73eb 3fff57a0 4022e95c

3ffebed0: 00000000 000000d0 3ffebf00 00000030

3ffebee0: 23d24220 94505eb8 343d9c13 00000004

3ffebef0: 3fff87b8 3fff73b8 3fff73e5 4022a9b2

3ffebf00: d18f0203 5a0de732 a7a86b81 5a5fcdac

3ffebf10: dea207fb c0d93270 5df54da1 cdfe6d0d

3ffebf20: 5b8ad0c5 e16be47e bc1c8785 c0bc4001

3ffebf30: 3fff73e5 3fff87b8 3fff73b8 4022ad45

3ffebf40: f2fedcf8 0187f7fa 8c83b36b 3fff5228

3ffebf50: 00000000 00000004 00000004 40204dfb

3ffebf60: 3fff73e9 3fff73e5 3fff53b8 00001240

3ffebf70: 000012a0 3fff8667 0000001d 00000004

3ffebf80: 3fff73b8 3fff73e5 00000004 00000004

3ffebf90: 3fff73e5 3fff87b8 3fff73b8 4022a7da

3ffebfa0: 00000038 00000000 3fff5228 40204c56

3ffebfb0: 00000038 00000001 00000033 3fff73e0

3ffebfc0: 00000000 00020010 0000000e 00000010

3ffebfd0: 00000004 401000a2 3ffeae68 40204cb4

3ffebfe0: 3fff87b8 00000033 00000016 00000000

3ffebff0: 3fff87b8 3fff73b8 3fff73b8 4022a916

3ffec000: 3fff87b8 3fff73e5 3fff73b8 4022aaed

3ffec010: 00000000 3fff5280 3fff73b8 4022ab76

3ffec020: 3fff6d98 40205e8e 3ffeae68 3ffeb0e0

3ffec030: 3fff52f8 3fff5260 3ffeae68 40204ab6

3ffec040: c8a79a95 3ffe9701 402017ec 3ffe9701

3ffec050: 3ffec130 000001bb 3ffeae68 40204d28

3ffec060: 3ffe97e0 c8a79a95 00000000 40207afa

3ffec070: 3ffe9701 000001bb 3ffeae68 402032f5

3ffec080: 3ffe97e0 c8a79a95 3ffe97e0 c8a79a95

3ffec090: 3ffe95b6 3ffec154 3ffec154 40207bd0

3ffec0a0: 3ffe9701 3ffec154 3ffeae68 40204d40

3ffec0b0: 401000b4 3ffec130 3ffea8a4 40205426

3ffec0c0: 3fff4e98 401000a2 3ffec130 00000044

3ffec0d0: 00000043 00000001 3ffec130 40207afa

3ffec0e0: 3fff4e38 3fff4f28 0000000f 00000000

3ffec0f0: 3fff4f08 0000000f 00000002 40207b82

3ffec100: 3ffec238 3ffea8a4 3ffec130 40207c84

3ffec110: 3ffec238 3ffea8a4 00000000 3ffec1a0

3ffec120: 3ffec238 3ffea8a4 00000000 402055ab

3ffec130: 3fff4ea8 0000004f 00000044 00000001

3ffec140: 3ffec238 00000000 3ffe92dc 40207afa

3ffec150: 3fff3c98 3fff4f48 0000000f 00000000

3ffec160: 3fff4e38 0000004f 00000044 40207b82

3ffec170: 3fffdc20 00000000 3ffec1a0 40207c84

3ffec180: 4101a8c0 00ffffff 3ffea8a0 3ffec1fc

3ffec190: 3fffdc20 00000000 3ffea8a0 402022ca

3ffec1a0: 3fff4b78 0000000f 00000001 00000000

3ffec1b0: 00000000 00000000 3ffec1f4 4020186a

3ffec1c0: 00000000 00000000 3ffeb1b0 40100398

<<<stack<<<
ets Jan 8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1264, room 16 
tail 0
chksum 0x42
csum 0x42
~ld

I think is an out-of-RAM condition
can you help me a little?

Matt Kocaj · Answer 75 · Wed Oct 21 2015 10:13:05 GMT+0800 (China Standard Time)

@igrr Refer to the section Creating a Twilio account so you can send sms to get Twilio API creds. In the mean time, I'll get you that compiled sketch too.

Ivan Grokhotkov · Answer 76 · Fri Oct 23 2015 05:48:44 GMT+0800 (China Standard Time)

@cottsak I got myself an account and changed your sketch to send an SMS every 10 seconds. 50 requests done, no crashes so far. Will leave it running.

Matt Kocaj · Answer 77 · Sun Oct 25 2015 15:02:01 GMT+0800 (China Standard Time)

@igrr Perhaps there is something else different that we're doing.

Can you post the code you're using now so I can try it too?
Also, what method are you using to push that code to esp? I'm just using Arduino IDE - could there be some subtle differences there that might make a difference?

Thomas Amberg · Answer 78 · Sun Oct 25 2015 18:57:18 GMT+0800 (China Standard Time)

@abl re FOSS license exception: has been removed, PolarSSL/mbed is now available under the Apache 2.0 license. See https://tls.mbed.org/foss-license-exception

Sequoia McDowell · Answer 79 · Tue Oct 27 2015 04:22:54 GMT+0800 (China Standard Time)

Sorry to add noise but this thread is pretty long, is there a wiki page or something entitled "How to Make HTTPS Requests Using the ESP8266 Arduino IDE" that steps thru setting this up? Sounds somewhat complex with the arduino IDE recompile/board management etc.

As a side-question: I use the Sparkfun Thing with their board definition-- will it be terribly complicated to get this code to run on that board? thanks.

Matt Kocaj · Answer 80 · Tue Oct 27 2015 07:05:40 GMT+0800 (China Standard Time)

@Sequoia This tutorial I just published could help you out: http://hammerproject.com/post/130804023369/iot-intro-sms-me-when-i-leave-my-garage-door-open

John Harrison · Answer 81 · Sun Nov 08 2015 11:41:27 GMT+0800 (China Standard Time)

@mtnbrit it turns out the SSL library supports MQTT. Packets will be sent encrypted and server authenticity can be verified. However, as far as I know, we don't yet support client verification from the server.

To get MQTT to work with TLS, two lines need to be added in
https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/src/WiFiClientSecure.cpp

for function

uint8_t WiFiClientSecure::connected() {
    if (_client->state() == ESTABLISHED)
        return 1;

    if (!_ssl)
        return 0;

    return _ssl->available() > 0;
}

add at the beginning of the function (before if (_client->state() == ESTABLISHED))

    if (!_client)
      return 0;

(@igrr should these lines be pushed into the git repo? I believe it's a mistake that it isn't already there.)

After that it's just a matter of using WiFiClientSecure instead of WiFIClient in the mqtt_esp8266 example.

Ivan Grokhotkov · Answer 82 · Sun Nov 08 2015 15:25:17 GMT+0800 (China Standard Time)

Yes, it's a mistake. Thanks for spotting.

On Sun, Nov 8, 2015, 06:41 whyameye notifications@github.com wrote:

@mtnbrit https://github.com/mtnbrit it turns out the SSL library
supports MQTT. Packets will be sent encrypted and server authenticity can
be verified. However, as far as I know, we don't yet support client
verification from the server.

To get MQTT to work with TLS, two lines need to be added in

https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/src/WiFiClientSecure.cpp

for function

uint8_t WiFiClientSecure::connected() {
if (_client->state() == ESTABLISHED)
return 1;
if (!_ssl)
    return 0;

return _ssl->available() > 0;
}

add at the beginning of the function (before if (_client->state() ==
ESTABLISHED))
if (!_client)
  return 0;
(@igrr https://github.com/igrr should these lines be pushed into the
git repo? I believe it's a mistake that it isn't already there.)

After that it's just a matter of using WiFiClientSecure instead of
WiFIClient in the mqtt_esp8266 example.

—
Reply to this email directly or view it on GitHub
#43 (comment).

sticilface · Answer 83 · Wed Nov 11 2015 05:05:54 GMT+0800 (China Standard Time)

I'm trying to get my ESP to download from raw.githubusercontent.com, I get the handshake success, but then receive nothing. Is there a way for me to identify if it is due to the TLS fragment being too big?

Ivan Grokhotkov · Answer 84 · Wed Nov 11 2015 05:20:20 GMT+0800 (China Standard Time)

@sticilface Yes you can uncomment DEBUGV in debug.h and DEBUG_TLS in WiFiClientSecure.cpp. Also add Serial.setDebugOutput(true); in setup() function.

sticilface · Answer 85 · Thu Nov 12 2015 15:25:49 GMT+0800 (China Standard Time)

So here is the debug output. SNTP?

Connected to raw.githubusercontent.com
pos:0 pos:3 pos:6 pos:9 pos:12 pos:15 pos:18 pos:21 pos:24 pos:27 pos:30 pos:33 pos:36 pos:39 pos:42 pos:45 pos:48 pos:51 pos:54 pos:57 certificate matches
please start sntp first !
:wr

:sent 277

:ww

GET /sticilface/ESPmanager/fixcrashing/examples/Settingsmanager-example/data/jquery.mobile-1.4.5.min.js.gz HTTP/1.1
Host: raw.githubusercontent.com
User-Agent: BuildFailureDetectorESP8266
Accept: */*
Connection: close

Waiting for server response: 
Recieved data:
Recieve end
File /jquery.mobile-1.4.5.min.js.gz 0 Bytes
please start sntp first !
:wr

:sent 53

:ww

:ur 1

:close

sticilface · Answer 86 · Thu Nov 12 2015 20:03:39 GMT+0800 (China Standard Time)

Last debug msg is not so clear as I'm download 3 files back to back. here is one debug output from start to end.

===================== START ===================
File Created
HOST: raw.githubusercontent.com:443
:ref 1

please start sntp first !
please start sntp first !
:wr

:sent 56

:rn 1440

:ww

:rd 5, 1440, 0

:rdi 1440, 5

:rd 74, 1440, 5

:rdi 1435, 74

:rd 5, 1440, 79

:rdi 1361, 5

:rd 1356, 1440, 84

:rdi 1356, 1356

:c0 1356, 1440

:rn 1320

:rd 1311, 1320, 0

:rdi 1320, 1311

:rd 5, 1320, 1311

:rdi 9, 5

:rd 4, 1320, 1316

:rdi 4, 4

:c0 4, 1320

please start sntp first !
please start sntp first !
:wr

:sent 267

:ww

:wr

:sent 6

:ww

please start sntp first !
:wr

:sent 69

:ww

:rn 75

:rd 5, 75, 0

:rdi 75, 5

:rd 1, 75, 5

:rdi 70, 1

:rd 5, 75, 6

:rdi 69, 5

:rd 64, 75, 11

:rdi 64, 64

:c0 64, 75

Connected to raw.githubusercontent.com
pos:0 pos:3 pos:6 pos:9 pos:12 pos:15 pos:18 pos:21 pos:24 pos:27 pos:30 pos:33 pos:36 pos:39 pos:42 pos:45 pos:48 pos:51 pos:54 pos:57 certificate matches
please start sntp first !
:wr

:sent 261

:ww

Waiting for server response: 
File /jquery-1.11.1.min.js.gz 0 Bytes
please start sntp first !
:wr

:rn 677

:rch 677, 53

:rcla

:abort

:ww

:ur 1

SPIFFS_close: fd=1

SPIFFS_close: fd=1

/jquery-1.11.1.min.js.gz has been downloaded

N0TB0T · Answer 87 · Fri Nov 13 2015 01:20:26 GMT+0800 (China Standard Time)

I'm now getting completely lost. @igrr I can't find any of the comments you mentioned in "uncomment DEBUGV in debug.h and DEBUG_TLS in WiFiClientSecure.cpp" - sry

Also, could anybody tell me, whether it makes any sense to investigate how to use parse.com with esp8266 at all? Two people seem to have had success connecting, but maybe that is bullshit.

Maybe anybody could have a look at the certificate of api.parse.com and tell me whether that's supposed to work?

Spending my fourth day now, managed to get the certificate accepted, but did not get one call through... PLZ!

sticilface · Answer 88 · Fri Nov 13 2015 22:04:24 GMT+0800 (China Standard Time)

Sorry for posting yet another debug output, but i forgot to hit save when adding debugging to the secure client lib. so i have more debugging. hopefully this is useful to someone (probably only @igrr) I'm curious about the sntp thing. I've tried executing this from loop and not setup, but no change. also there is a huge drop in heap something like 7-8k, when the secureclient is used, and this occurs even it if is dynamic... using new. Are the secure client functions all held in RAM is there anyway to have them loaded from flash? (do forgive me if i'm talking out of my tree, and don't understand what is going on).

thanks for all the hard work igrr.

===================== START ===================
:urch 68, 57

:urd 19, 57, 13

File Created
:urch 57, 127

HOST: raw.githubusercontent.com:443
:ref 1

please start sntp first !
please start sntp first !
State:  sending Client Hello (1)
:wr

:sent 56

:rn 1418

:ww

:rd 5, 1418, 0

:rdi 1418, 5

:rd 74, 1418, 5

:rdi 1413, 74

State:  receiving Server Hello (2)
:rd 5, 1418, 79

:rdi 1339, 5

:rd 1334, 1418, 84

:rdi 1334, 1334

:c0 1334, 1418

:rn 1342

:rd 1333, 1342, 0

:rdi 1342, 1333

State:  receiving Certificate (11)
:rd 5, 1342, 1333

:rdi 9, 5

:rd 4, 1342, 1338

:rdi 4, 4

:c0 4, 1342

State:  receiving Server Hello Done (14)
please start sntp first !
please start sntp first !
State:  sending Client Key Exchange (16)
:wr

:sent 267

:ww

:wr

:sent 6

:ww

State:  sending Finished (16)
please start sntp first !
:wr

:sent 69

:ww

:rn 75

:rd 5, 75, 0

:rdi 75, 5

:rd 1, 75, 5

:rdi 70, 1

:rd 5, 75, 6

:rdi 69, 5

:rd 64, 75, 11

:rdi 64, 64

:c0 first !
:wr

 64, 75

State:  receiving Finished (16)
Connected to raw.githubusercontent.com
pos:0 pos:3 pos:6 pos:9 pos:12 pos:15 pos:18 pos:21 pos:24 pos:27 pos:30 pos:33 pos:36 pos:39 pos:42 pos:45 pos:48 pos:51 pos:54 pos:57 certificate matches
please start sntp:sent 261

:ww

Waiting for server response: 
File /jquery-1.11.1.min.js.gz 0 Bytes
please start sntp first !
:wr

:sent 53

:ww

Alert: close notify
:ur 1

:close

SPIFFS_close: fd=1

SPIFFS_close: fd=1

/jquery-1.11.1.min.js.gz has been downloaded

Juppit · Answer 89 · Fri Nov 13 2015 22:26:42 GMT+0800 (China Standard Time)

This sntp stuff comes probably in since
4cf72e7

Do you use any time functions or includes time.h ?

On 13.11.2015 at 15:04 wrote sticilface:

Sorry for posting yet another debug output, but i forgot to hit save
when adding debugging to the secure client lib. so i have more
debugging. hopefully this is useful to someone (probably only @igrr
https://github.com/igrr) I'm curious about the sntp thing. I've tried
executing this from loop and not setup, but no change. also there is a
huge drop in heap something like 7-8k, when the secureclient is used,
and this occurs even it if is dynamic... using new. Are the secure
client functions all held in RAM is there anyway to have them loaded
from flash? (do forgive me if i'm talking out of my tree, and don't
understand what is going on).

thanks for all the hard work igrr.

|===================== START =================== :urch 68, 57 :urd 19,
57, 13 File Created :urch 57, 127 HOST: raw.githubusercontent.com:443
:ref 1 please start sntp first ! please start sntp first ! State:
sending Client Hello (1) :wr :sent 56 :rn 1418 :ww :rd 5, 1418, 0 :rdi
1418, 5 :rd 74, 1418, 5 :rdi 1413, 74 State: receiving Server Hello (2)
:rd 5, 1418, 79 :rdi 1339, 5 :rd 1334, 1418, 84 :rdi 1334, 1334 :c0
1334, 1418 :rn 1342 :rd 1333, 1342, 0 :rdi 1342, 1333 State: receiving
Certificate (11) :rd 5, 1342, 1333 :rdi 9, 5 :rd 4, 1342, 1338 :rdi 4, 4
:c0 4, 1342 State: receiving Server Hello Done (14) please start sntp
first ! please start sntp first ! State: sending Client Key Exchange
(16) :wr :sent 267 :ww :wr :sent 6 :ww State: sending Finished (16)
please start sntp first ! :wr :sent 69 :ww :rn 75 :rd 5, 75, 0 :rdi 75,
5 :rd 1, 75, 5 :rdi 70, 1 :rd 5, 75, 6 :rdi 69, 5 :rd 64, 75, 11 :rdi
64, 64 :c0 first ! :wr 64, 75 State: receiving Finished (16) Connected
to raw.githubusercontent.com pos:0 pos:3 pos:6 pos:9 pos:12 pos:15
pos:18 pos:21 pos:24 pos:27 pos:30 pos:33 pos:36 pos:39 pos:42 pos:45
pos:48 pos:51 pos:54 pos:57 certificate matches please start sntp:sent
261 :ww Waiting for server response: File /jquery-1.11.1.min.js.gz 0
Bytes please start sntp first ! :wr :sent 53 :ww Alert: close notify :ur
1 :close SPIFFS_close: fd=1 SPIFFS_close: fd=1 /jquery-1.11.1.min.js.gz
has been downloaded |

—
Reply to this email directly or view it on GitHub
#43 (comment).

Ivan Grokhotkov · Answer 90 · Fri Nov 13 2015 22:29:05 GMT+0800 (China Standard Time)

Time functions are used by SSL internally. However there is no issue with SNTP not being initialized.
@sticilface this thread got quite long, did you post the sketch which corresponds to this debug output somewhere? Could you please link a gist or something?

Ivan Grokhotkov · Answer 91 · Fri Nov 13 2015 22:31:56 GMT+0800 (China Standard Time)

@N0TB0T i went to parse.com but honestly I'm completely lost. What is the minimal amount of stuff i need to do to try their API?

N0TB0T · Answer 92 · Fri Nov 13 2015 23:03:34 GMT+0800 (China Standard Time)

@igrr It's pretty straightforward: Get an Account (free), click "new app", done. You can find your app ID and keys under settings: https://www.parse.com/apps/yourAppName/edit#keys . From there on, it's just using their REST API, using the app-ID and the REST-API-Key... All data sent there via POST in json-format will show up at https://www.parse.com/apps/yourAppName/collections - here are the example calls: https://www.parse.com/docs/rest/guide

Thanks for trying!

N0TB0T · Answer 93 · Fri Nov 13 2015 23:08:44 GMT+0800 (China Standard Time)

@igrr code on the clientside would look somewhat like this:

    client.print("POST /1/classes/yourAppName HTTP/1.1\r\n");
    client.print("Host: api.parse.com\r\n");
    client.print("Connection: close\r\n");
    client.print("Content-Type: application/json\r\n");
    client.print("User-Agent: ESP8266\r\n");
    client.print("X-Parse-Application-Id: ");
    client.print(PARSE_APP_ID);
    client.print("\r\n");
    client.print("X-Parse-REST-API-Key: ");
    client.print(PARSE_REST_API_KEY);
    client.print("\r\n");

    client.print("\"temperature\":17.4,\"temperatureReference\":17,\"humidity\":85,\"pressure\":1038.1,\"brightness\":40,\"batteryLevel\":97}");

At least I hope so.

sticilface · Answer 94 · Fri Nov 13 2015 23:26:21 GMT+0800 (China Standard Time)

@igrr https://gist.github.com/sticilface/3795d4e2b8047c729bbe

Ivan Grokhotkov · Answer 95 · Fri Nov 13 2015 23:31:07 GMT+0800 (China Standard Time)

@sticilface it seems to me that you need to replace SecClient.connected with SecClient.available at line 101. At this point you are connected because you just did SecClient.print a few lines above. On the other hand, data may not be available yet. SecClient.connected check passes immediately and then you hit while (SecClient.available()), and you miss this loop because no data is available.

Ivan Grokhotkov · Answer 96 · Fri Nov 13 2015 23:31:58 GMT+0800 (China Standard Time)

@N0TB0T Thanks for info, I'll try on the weekend.

sticilface · Answer 97 · Fri Nov 13 2015 23:45:19 GMT+0800 (China Standard Time)

ah, i did actually have it that way round to start with... but you make total sense. However, still no jo
https://gist.github.com/sticilface/24842312767a2ed1fe1a

the clue might be this

wr

:er -9 53 1

:ww

Ivan Grokhotkov · Answer 98 · Sat Nov 14 2015 04:27:32 GMT+0800 (China Standard Time)

@sticilface this config.htm file is pretty large, so yes, you are hitting max fragment size limitation.

---------------
Waiting for server response: 
---- BODY ---
:rd 5, 1452, 0
:rdi 1452, 5
Error: invalid protocol message

Final line is debug output from axTLS (i have uncommented #define DEBUG_SSL line in WiFiClientSecure.cpp).

sticilface · Answer 99 · Sun Nov 15 2015 02:33:35 GMT+0800 (China Standard Time)

@igrr ok, thanks for that. Have to think of another solution to that. I guess HTTP hosting somewhere else