Request is not always captured when page sends it right after loading
andrebcmarinho opened this issue · comments
Hello! I'm loading a webpage that is always doing the same POST request and most of the times I'm able to get request body using this library. However, sometimes when shouldInterceptRequest() is called, webViewRequest.type is HTML instead of XML_HTTP and in that case POST request body is not available. Please note that webViewRequest.method is always POST as expected.
Any idea of what might be happening and how to fix this? Thank you
Hello, thanks for the report!
This means that the library's main request recorder didn't work, that is, no request was recorded. Could you please provide more details on where this happens so it can be debugged? For example, on what website does it happen?
Hello @acsbendi! I've created a simple page to be able to replicate this issue: http://ynap-debug.atwebpages.com/
If you load this page multiple times, it will eventually fail after a few time.
Would you be able to take a look on your side? Thank you
@andrebcmarinho I've tried to reproduce this issue, but I couldn't, the request were logged correctly every time for me, producing the following log output:
Sending request from WebView:
Type: XML_HTTP
URL: http://ynap-debug.atwebpages.com/redirect.php?rand=612
Method: POST
Body: {
"Id": 78912,
"Customer": "Jason Sweet",
"Quantity": 1,
"Price": 18.00
}
Headers:
referer: http://ynap-debug.atwebpages.com/
cookie:
origin: http://ynap-debug.atwebpages.com
content-type: application/json
cache-control: private
accept: application/json
user-agent: Mozilla/5.0 (Linux; Android 12; SM-A528B Build/SP1A.210812.016; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/111.0.5563.116 Mobile Safari/537.36
Trace:
at XMLHttpRequest.send (<anonymous>:68:17)
at doPostOriginal (http://ynap-debug.atwebpages.com/:28:21)
at HTMLDocument.<anonymous> (http://ynap-debug.atwebpages.com/:47:17)
Encoding type (form submissions only): null
Is for main frame? false
Is redirect? false
Has gesture? false
Can you try to create a project with a minimal reproducible example?
@andrebcmarinho Thanks for the sample app, I've managed to reproduce the issue with it.
As I've mentioned before, this problem happens because the request recorder didn't work. I've spent some time debugging why, and the reason I found is that the script that sets up the recorder doesn't get executed fast enough after the page starts loading. The only way to resolve this is to inject the recorder setup code directly into the HTML page so that we can guarantee that it gets executed before any other Javascript. This requires sending the original main GET request (e.g. http://ynap-debug.atwebpages.com/
in your example) and modifying the returned body, as described in #2. So once that functionality is implemented, we can fix this issue too.
I've edited the issue's title so that it reflects my findings.