No result from github https query

Question

No result from github https query

glennswest opened this issue 2 years ago · comments

Describe the bug

Doing a query to a github api endpoint (Publically available, and works from curl), and get no response.

Steps to Reproduce

Run app(i2cgateha) on m5stack esp32 (Which has working mqttasync and webserver), query returns no result.
App github: https://github.com/glennswest/i2cgateha
Debug Text: https://raw.githubusercontent.com/glennswest/i2cgateha/main/problem.txt

Steps to reproduce the behavior. Including the MRE sketches
Run above program

Expected behavior

Returns:
Print something like:
[
{
"name": "index.html",
"path": "contents/index.html",
"sha": "9e52837f62962c18c7e214a7e1a0d70882a1c726",
"size": 11,
"url": "https://api.github.com/repos/glennswest/i2cgateha/contents/contents/index.html?ref=main",
"html_url": "https://github.com/glennswest/i2cgateha/blob/main/contents/index.html",
"git_url": "https://api.github.com/repos/glennswest/i2cgateha/git/blobs/9e52837f62962c18c7e214a7e1a0d70882a1c726",
"download_url": "https://raw.githubusercontent.com/glennswest/i2cgateha/main/contents/index.html",
"type": "file",
"_links": {
"self": "https://api.github.com/repos/glennswest/i2cgateha/contents/contents/index.html?ref=main",
"git": "https://api.github.com/repos/glennswest/i2cgateha/git/blobs/9e52837f62962c18c7e214a7e1a0d70882a1c726",
"html": "https://github.com/glennswest/i2cgateha/blob/main/contents/index.html"
}
}
]

Actual behavior

Returns no data

Debug and AT-command log (if applicable)

See serial log:
https://raw.githubusercontent.com/glennswest/i2cgateha/main/problem.txt

Information

Please ensure to specify the following:
Arduino 1.18.16
Hardware: M5Stack-Paper ESP32

Contextual information (e.g. what you were trying to achieve)
Pull a list of files on the directory on github
Simplest possible steps to reproduce
Use the supplied url to test with your example code

Glenn West · Answer 1 · Thu Mar 24 2022 08:01:21 GMT+0800 (China Standard Time)

Similar result to: "https://worldtimeapi.org/api/timezone/Europe/London.txt"

Glenn West · Answer 2 · Thu Mar 24 2022 08:02:14 GMT+0800 (China Standard Time)

When I do the London, example: I see [ATCP] _recv: tot_len = 1211
In the log, indicating a decent respond.

Glenn West · Answer 3 · Thu Mar 24 2022 08:04:11 GMT+0800 (China Standard Time)

Looks like data comes back on the github api call:
ATCP] _handle_async_event: Task Name = async_tcp_ssl
[ATCP] _handle_async_event: LWIP_TCP_SENT = 0x3F848928
[ATCP] _sent: len = 303
[ATCP] _handle_async_event: Task Name = async_tcp_ssl
[ATCP] _handle_async_event: LWIP_TCP_RECV = 0x3F848928
[ATCP] _recv: tot_len = 1436
[ATCP] _handle_async_event: Task Name = async_tcp_ssl
[ATCP] _handle_async_event: LWIP_TCP_RECV = 0x3F848928
[ATCP] _recv: tot_len = 1253

Glenn West · Answer 4 · Thu Mar 24 2022 09:50:33 GMT+0800 (China Standard Time)

Here's something to make it even more interesting:
https://raw.githubusercontent.com/glennswest/i2cgateha/main/contents/.version
This returns "1" as expected.

Khoi Hoang · Answer 5 · Fri Mar 25 2022 01:35:00 GMT+0800 (China Standard Time)

Hi @glennswest

Thanks for your interest and usage of the library.

I've just tested and found no issue at all with the libraries, using the AsyncHTTPSRequest_ESP example. Check the following Debug Terminal Output for both

I suggest you restart rewriting your code, slowly and methodologically, from the above example, then move forward gradually to identify when and where the code is broken.

I'm sorry I certainly don't have all the time in the world to read, test and debug your code. I also don't have the same hardware and/or similar settings (network, ISP, etc.).

As this is not a bug of the library, I'm closing the issue now, and only reopen when you prove this is a bug of this library.

Good Luck,

Debug Terminal Outputs

Starting AsyncHTTPSRequest_ESP using ESP32_DEV
AsyncTCP_SSL v1.2.0
AsyncHTTPSRequest_Generic v2.0.0
Connecting to WiFi SSID: HueNet1
........
AsyncHTTPSRequest @ IP : 192.168.2.93

**************************************
abbreviation: EDT
client_ip: 45.72.131.83
datetime: 2022-03-24T13:06:02.807503-04:00
day_of_week: 4
day_of_year: 83
dst: true
dst_from: 2022-03-13T07:00:00+00:00
dst_offset: 3600
dst_until: 2022-11-06T06:00:00+00:00
raw_offset: -18000
timezone: America/Toronto
unixtime: 1648141562
utc_datetime: 2022-03-24T17:06:02.807503+00:00
utc_offset: -04:00
week_number: 12
**************************************
HHH

Starting AsyncHTTPSRequest_ESP using ESP32_DEV
AsyncTCP_SSL v1.2.0
AsyncHTTPSRequest_Generic v2.0.0
Connecting to WiFi SSID: HueNet1
.......
AsyncHTTPSRequest @ IP : 192.168.2.93

**************************************
abbreviation: GMT
client_ip: 45.72.131.83
datetime: 2022-03-24T17:07:16.182664+00:00
day_of_week: 4
day_of_year: 83
dst: false
dst_from: 
dst_offset: 0
dst_until: 
raw_offset: 0
timezone: Europe/London
unixtime: 1648141636
utc_datetime: 2022-03-24T17:07:16.182664+00:00
utc_offset: +00:00
week_number: 12
**************************************

Glenn West · Answer 6 · Fri Mar 25 2022 01:47:41 GMT+0800 (China Standard Time)

Actually, the probem is "timing".
request.setTimeout(60);
Solved the problem.
Adding that made everything work.
Suggest you increase the default timeout.

Glenn West · Answer 7 · Fri Mar 25 2022 01:56:48 GMT+0800 (China Standard Time)

I've ran it multiple times, and every url I use works now, with debug off.
So Im really happy.
I need to look how to do "chunks", so I can download arbitrary files to sd card.

Khoi Hoang · Answer 8 · Fri Mar 25 2022 01:57:33 GMT+0800 (China Standard Time)

Thanks for your feedback and I'm glad you found the elusive clue from debug messages

[AHTTPS] _onPoll timeout

Will increase the DEFAULT_RX_TIMEOUT from 3s to reasonable 30s to cope with some slow network.

Glenn West · Answer 9 · Fri Mar 25 2022 02:41:10 GMT+0800 (China Standard Time)

Great. Happy I could help.
Im working on a auto update of flash and web content using your library.
Push to github, and device can update itself.
(SD card required of course)

Khoi Hoang · Answer 10 · Fri Mar 25 2022 02:50:15 GMT+0800 (China Standard Time)

Hi @glennswest

The new AsyncHTTPSRequest_Generic releases v2.0.1 has just been published. Your contribution is noted in Contributions and Thanks

Best Regards,

Releases v2.0.1

Increase DEFAULT_RX_TIMEOUT to 30s from 3s for slower networks. Check No result from github https query #8

Glenn West · Answer 11 · Fri Mar 25 2022 02:56:34 GMT+0800 (China Standard Time)

Wow, thanks.
Made my day.
I love opensource.