Follow 302/301 redirects

[Sebastian Neef]

Hi,

The problem

in Net.java's toString() method there's the following check:

            if (mUrlConnection.getResponseCode() == HttpURLConnection.HTTP_OK){

That leads to issues if a the response is a 301/302 redirect to a status.json file.

For example:

> curl -L -I http://www.c-base.org/status.json
HTTP/1.1 302 Found
Cache-Control: no-cache
Content-length: 0
Location: https://www.c-base.org/status.json
Connection: close

HTTP/1.1 200 OK
Date: Sat, 17 Jan 2015 21:46:03 GMT
Server: Apache/2.2.16 (Debian)
Last-Modified: Fri, 02 May 2014 01:48:51 GMT
ETag: "1860012-316-4f860fbbf26c0"
Accept-Ranges: bytes
Content-Length: 790
Content-Type: text/plain; charset=UTF-8
Strict-Transport-Security: max-age=31536000; includeSubDomains

The error message is something like: Error: java.lang.Throwable - Found http://www.c-base.org/status.json

Fix

Check if the response code is 301/302 (redirect) and then follow the new url. Taking care of protocol switches (e.g. http -> https / https -> http) would be necessary.

Kind regards,
gehaxelt

[Noskill]

Thanks for issuing the problem !

We'll look into that and fix it in the next release :)

[Aubort Jean-Baptiste]

Oops indeed, we have to find a place to check for final response (after getInputStream() ?), HttpURLConnection supports redirections (apart in the case of protocols switching indeed..)..

[Izzy]

We hit this bug as well after switching to HTTPS. Any progress on this? :/

[Bernhard Tittelbach]

coincidence, same here. Switching to https using temp-redirects broke the app. Any progress on this?

[Stefan]

(It seems thanks to Let's Encrypt everyone it switching to HTTPS so I guess we will see this issue more often in the next weeks ...)

For the sake of completeness: In HttpURLConnection this is ... not a bug, but a feature.

After discussion among Java Networking engineers, it is felt that we shouldn't automatically follow redirect from one protocol to another, for instance, from http to https and vise versa, doing so may have serious security consequences. Thus the fix is to return the server responses for redirect. Check response code and Location header field value for redirect information. It's the application's responsibility to follow the redirect.

See http://bugs.java.com/bugdatabase/view_bug.do?bug_id=4620571

... 👎

[Stefan]

Something like this in your Net.getString() should do the job ...

String location = "http://realraum.at"; // does a 307 to https
HttpURLConnection urlConnection = null;
URL url = null;

do{
   url = new URL(location);
   urlConnection = (HttpURLConnection) url.openConnection();
   //HttpURLConnection.setFollowRedirects(true);
   InputStream in = new BufferedInputStream(urlConnection.getInputStream());

   if(urlConnection.getResponseCode() == HttpURLConnection.HTTP_MOVED_TEMP
      || urlConnection.getResponseCode() == HttpURLConnection.HTTP_MOVED_PERM
      || urlConnection.getResponseCode() == HTTP_TEMPORARY_REDIRECT
      || urlConnection.getResponseCode() == HTTP_PERMANENT_REDIRECT) {
      // TODO: ^ think about more relevant status codes

      location = urlConnection.getHeaderField("Location");
      System.out.println(location == null ? "no location found" : location);

   } else {
      // TODO: introduce boolean flag for this:
      location = null;
   }
}while(location != null);

Full demo code at:
https://gist.github.com/stefan2904/8b75d45db5816500b44a

(Also, not sure if you need those mUrlConnection.connect(); because url.openConnection(); already connects?)

[Stefan]

If you are lazy: PR #38

[Adam Hirst]

Hi there! I would like to clarify whether the issue I'm currently having is the same as this one, before I open a fresh Bug Report which ends up being found to be a duplicate.

I am using version 1.8 from F-Droid, but whenever I try to select the Hackerspace "OpenLab", I get the following error message:

Errpr:java.lang.Throwable
Moved Permanently: Moved Permanently http://api.openlab-augsburg.de/data.json

The guys over at said Hackerspace (#augsburg on Freenode, German-language) reckon things their-side (or at least at the given URL) are working fine, so it looks as if it's this redirection problem.

Let me know if you need anything else. 😄

[Aubort Jean-Baptiste]

Hello,

Thanks for the report, I'll check that asap. That's strange because I just merged something that should take care of http->https redirection in 2d102cf and it was working with the cases I tested.

Do you have access to the application log using adb ? there should be a log saying "fetching "

[Adam Hirst]

@rorist I don't at the moment but I might be able to later, we'll see how this evening goes.

[Aubort Jean-Baptiste]

Hello,

I've found the problem, it was not working at all the redirection :) It's now fixed in 3446041 and will be published soon with the ability to change the spaceapi directory (by default it will be ours).

Regards