Reading rack.input and rewinding multiple times is not possible

[Luc Heinrich]

Here's a contrived example demonstrating the problem:

require 'rubygems'
require 'rack'

class Test
  def call(env)
    input = env["rack.input"]
    puts "first read: '#{input.read}'"
    input.rewind
    puts "second read: '#{input.read}'"

    [200, {"Content-Type" => "text/plain"}, "Hello World"]
  end
end

run Test.new

Launch a mizuno server for this app and make a POST request like this:

curl -X POST -d "foo=bar" localhost:9292

Expected result is:

first read: 'foo=bar'
second read: 'foo=bar'
0:0:0:0:0:0:0:1%0 - - [24/Jul/2012 19:27:03] "POST / HTTP/1.1" 200 - 0.0070

But actually is:

first read: 'foo=bar'
second read: ''
0:0:0:0:0:0:0:1%0 - - [24/Jul/2012 19:27:03] "POST / HTTP/1.1" 200 - 0.0070

[Luc Heinrich]

Here's a simple spec for this, which might be easier to use than the full example above:

it "should read data than rewind and read again when wrapped in a Ruby IO object" do
    str = "hello world"
    input = str.unpack('c*')
    io = rewindable_input_stream(input.to_java(:byte)).to_io
    io.read.should == str
    io.rewind
    io.read.should == str
end

Adding this to spec/rewindable_input_stream_spec.rb makes the specs fail and actually points fingers at the fact that the RewindableInputStream is wrapped in a Ruby IO object by the to_io call. So this is either an incorrect use of RewindableInputStream or a bug somewhere else.

[Luc Heinrich]

And I think I'm actually going to submit this to the jruby-rack guys too to have their input on it.

(done: jruby/jruby-rack#113)

[Luc Heinrich]

So. Calling to_io on a RewindableInputStream actually breaks its "rewindability", which basically makes Mizuno broken in its current state.

The question that I have now is why not simply using Rack::RewindableInput instead? Like this:

env['rack.input'] = Rack::RewindableInput.new(request.getInputStream.to_io.binmode)

This might be less efficient than using a straight RewindableInputStream because Rack::RewindableInput always buffers to a temp file instead of in memory below a certain threshold, but at least it's not broken :)

Thoughts?

[grinser]

Changing it to the way you supposed actually works pretty well for me, otherwise Grape (https://github.com/intridea/grape) can't be used properly (can't get POST body) in conjunction with Mizuno.
I don't know the best way to fix it though - could the owner of this project fix it somehow or should we do a fork? The latest maintenance of the project was 6 months ago...
For now I'll just monkey-patch the servlet_to_rack method locally myself which breaks future changes to this part of mizuno of course :-(

[Adam Zell]

Unicorn 3.0 handles it in the following way:

Rewindable "rack.input" may be disabled via the
"rewindable_input false" directive in the configuration file.
This will violate Rack::Lint for Rack 1.x applications, but can
reduce I/O for applications that do not need a rewindable
input.

jruby-rack:

• The previous non-rewindable input behavior can be re-instated by
  setting the system property 'jruby.rack.input.rewindable' to false.
  Despite being out of spec, environments such as Google AppEngine
  require this behavior because writing to a tempfile is not possible.

Perhaps it would make sense to make it an option.

[Don Werve]

I tend to get distracted by my client work, but I'll see if I can carve out some time in the near future to fix this, with Rack::RewindableInput as a configurable option -- some users, myself included, probably don't want to buffer input streams to files by default for both security and performance reasons.

Of course, a pull request with specs that doesn't break existing functionality would get merged-and-released much more quickly. :)

[Adam Zell]

This bug should probably be marked as closed.

[L. David Varvel]

Agreed! Thanks.