Saving to memory instead of disk
bsergean opened this issue · comments
Hi there,
How could the library be extended so that we just download the torrent but don't write it to disk, but to a memory buffer instead ?
Would the idea be to make a 'memory-storage' out of this class/code ?
https://github.com/cenkalti/rain/tree/master/internal/storage
Thanks !
Yes, that would be great. I think the interface is simple enough to be implemented in memory.
This may also include some overlapping work for #19 .
This project might worth checking out: https://github.com/spf13/afero
I would like to help on debugging the issue. Can you elaborate how it is being abused? Is it read or write cache that is being abused? What is the access pattern?
I'd like to give some information about the current state of the read cache.
Lines 146 to 153 in 4bda7a0
When a piece is requested from Rain, it assumes that the peer will request blocks from the same piece again. For that reason, Rain reads more than the actual requested block size. It divides the piece in larger blocks of size
ReadCacheBlockSize
and read all at once.
Example:
Let's assume that torrent piece size is 256K peer will request blocks in size of 16K.
0123456789ABCDEF
If ReadCacheBlockSize
is 128K (default value), from the perspective of read cache, torrent piece is divided to 128K blocks.
01234567 89ABCDEF
In ideal scenario, the peer starts requesting peers from the start in order. When the first block (0) is requested, Rain reads the range between 0 and 7 into memory and caches it. The following requests from 1 to 7 are served directly from cache without going to disk. For the next range between 8 and F, a second read is made from the disk.
If peer requests pieces and blocks completely random, then the read cache will not be useful. If that is the case, you can try increasing ReadCacheSize
.
I might give this a try, but I noticed that my disk problem was completely unrelated to this library, so feel free to close this ticket, or use it for reference.
This issue is stale because it has been open for 30 days with no activity.
This issue was closed because it has been inactive for 14 days since being marked as stale.