Distributed Downloading
Immortalin opened this issue · comments
An advanced feature that would speed up transfers is distributed downloading. Essentially a group of workers will download the various chunks across multiple servers allowing each server to saturate their network bandwidth. And then the main master server will concatenate the chunks together as usual. In other words, MapReduce.
MapReduce is certainly an interesting idea but do you see it as providing a significant benefit to BitTorrent? BitTorrent is a fairly lightweight protocol, so almost all time spent is usually in IO (at least based on my benchmarking). In essence your master server's disk/network bandwidth will always be a bottleneck regardless of whether or not the data comes from workers or peers.