Listener crash when serving large attachments

[Alejandro Martínez]

I'm trying to serve (streaming using HTML5) a 2mb .mp4 video file from TouchDB + Phonegap, and sometimes the app crashes with:

07-03 20:08:05.970: E/dalvikvm-heap(21516): Out of memory on a 12408264-byte allocation.
07-03 20:08:05.970: E/AndroidRuntime(21516): FATAL EXCEPTION: TDListenerHandlerThread
07-03 20:08:05.970: E/AndroidRuntime(21516): java.lang.OutOfMemoryError
07-03 20:08:05.970: E/AndroidRuntime(21516):    at java.nio.CharArrayBuffer.<init>(CharArrayBuffer.java:43)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at java.nio.ReadWriteCharArrayBuffer.<init>(ReadWriteCharArrayBuffer.java:47)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at java.nio.CharBuffer.allocate(CharBuffer.java:54)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at java.nio.charset.CharsetDecoder.allocateMore(CharsetDecoder.java:236)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:195)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at java.nio.charset.Charset.decode(Charset.java:487)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at java.lang.String.<init>(String.java:174)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at java.lang.String.<init>(String.java:141)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at com.couchbase.touchdb.listener.TDHTTPServlet$1.onDataAvailable(TDHTTPServlet.java:126)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at com.couchbase.touchdb.router.TDRouter.start(TDRouter.java:409)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at com.couchbase.touchdb.listener.TDHTTPServlet$2.run(TDHTTPServlet.java:143)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at android.os.Handler.handleCallback(Handler.java:605)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at android.os.Handler.dispatchMessage(Handler.java:92)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at android.os.Looper.loop(Looper.java:137)
07-03 20:08:05.970: E/AndroidRuntime(21516):    at android.os.HandlerThread.run(HandlerThread.java:60)

[Marty Schoch]

Thanks for the report, I believe its the same as this one:

#44

The stacktrace goes back to the same line in TDHttpServlet.

[Alejandro Martínez]

That line is:

                   Log.v(TAG, String.format("Asked to write: %s", new String(data)));

https://github.com/couchbaselabs/TouchDB-Android/blob/master/TouchDB-Android-Listener/src/com/couchbase/touchdb/listener/TDHTTPServlet.java#L126

so I guess either commenting it or making it not output logs for large responses (or binary ones) would do the trick.

[Marty Schoch]

Yes, if you're looking for a really quick fix you can do this. However, I've been working on a larger change which will allow us to stream data back. This means we never have to have the whole thing in RAM, and it can also allow for greater concurrency by returning from the single-threaded TouchDB path faster. Its basically working, but taking a little longer than I planned, hopefully I can wrap it up tonight or tomorrow.

[Marty Schoch]

Attachments are now streamed in and out when using methods in the API to directly access them. This should avoid the need to have the whole file in memory and hopefully this problem is now resolved. I tested successfully uploading and downloading a 15MB movie through the listener.

[Alejandro Martínez]

This issue is fixed now indeed, now I'm suffering #49 but it's probably not related.