ChunkedIO does not implement #gets correctly, and therefore does not parse CSV while streaming.
edlebert opened this issue · comments
edlebert commented
Hi,
First of all, thank you for writing this software. I tried fixing this myself but could not get the ChunkedIO CSV test to work. Currently, ruby's csv calls the io#gets with a nil separator and a limit. In ChunkedIO's current implementation, this causes ChunkedIO to read the entire data source, because it always does so when separator is nil.
When using CSV.new(Chunk.open('http://some/large/file.csv')).each {...}
, I used print statements to verify that the entire file is downloaded before any parsing occurs.
Janko Marohnić commented
I just released version 4.8.1 with fixes described in #30, so hopefully that should resolve the issue.