Memory usage keeps increasing
rjclarkewp007 opened this issue · comments
Hey hey, it is me again with another little problem. hehe
So, I've noticed that when I start up my application it takes up about 35MB of RAM. After that I start my seeding/downloading process and the memory climbs to about 60MB, which still seems normal, but then it starts climbing and it never stops.
When I stop my seeding process, the RAM usage is stable, but does not clear the memory which it has already taken up.
I would have to close my application and open it again to clear the memory usage.
I did a test to see if something else was causing this so left all code in place and only commented out:
await manager.StartAsync();
When doing that, the memory just stays where it is and nothing changes. Obviously the seeding etc does not start/work at all.
My code I use:
private Thread seedingTask;
private async void StartSeeding()
{
// Give an example of how settings can be modified for the engine.
var settingBuilder = new EngineSettingsBuilder
{
AllowPortForwarding = true,
AutoSaveLoadDhtCache = true,
AutoSaveLoadFastResume = true,
AutoSaveLoadMagnetLinkMetadata = true,
AllowLocalPeerDiscovery = true,
MaximumConnections = 300,
};
string currentinstalldir = File.ReadLines("config.ini").Skip(2).Take(1).First();
Directory.SetCurrentDirectory(currentinstalldir);
startTorrentFileDownload();
var torrent = await Torrent.LoadAsync(torrentfile);
dhtEngine = new DhtEngine(new IPEndPoint(IPAddress.Any, 0));
TorrentEngine = new ClientEngine(settingBuilder.ToSettings());
manager = await TorrentEngine.AddAsync(torrent, currentinstalldir);
manager.TorrentStateChanged += Manager_TorrentStateChanged1;
await dhtEngine.StartAsync();
await manager.StartAsync();
await manager.LocalPeerAnnounceAsync();
await manager.DhtAnnounceAsync();
}
Code I use to start the process:
private void button_Seeding_Click(object sender, EventArgs e)
{
button_StopSeeding.Visible = true;
button_StopSeeding.Enabled = true;
button_Seeding.Enabled = false;
button_Seeding.Visible = false;
label_seeding.Enabled = true;
label_seeding.Visible = true;
button_removeArchive.Enabled = false;
button_setmaxupload.Visible = true;
label_mbps.Visible = true;
numericUpDown1.Visible = true;
stopwatch = new Stopwatch();
stopwatch.Start();
seedingTask = new Thread(StartSeeding);
seedingTask.Start();
}
private void Manager_TorrentStateChanged1(object sender, TorrentStateChangedEventArgs e)
{
MethodInvoker error = delegate
{
try
{
label_seeding.Text = "Error! Please restart.";
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
};
MethodInvoker hashing = delegate
{
try
{
label_seeding.Text = "Hashing... Please wait!";
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
};
MethodInvoker downloading = delegate
{
try
{
label_seeding.Text = "Downloading... Please wait!";
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
};
MethodInvoker stopping = delegate
{
try
{
label_seeding.Text = "Stopping... Please wait!";
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
};
MethodInvoker seeding = delegate
{
try
{
label_seeding.Text = manager.State + "... " + ConvertBytesToMegabytes(manager.Monitor.UploadSpeed).ToString("0.00" + " MB/s ") + "(" + stopwatch.Elapsed.TotalSeconds.ToString("0") + "s )";
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
};
if (manager.State == TorrentState.Stopping)
{
label_seeding.Invoke(stopping);
}
if (manager.State == TorrentState.Hashing)
{
label_seeding.Invoke(hashing);
}
if (manager.State == TorrentState.Downloading)
{
label_seeding.Invoke(downloading);
}
if (manager.State == TorrentState.Error)
{
label_seeding.Invoke(error);
}
if (manager.State == TorrentState.Seeding)
{
label_seeding.Invoke(seeding);
}
while (manager.State == TorrentState.Seeding)
{
label_seeding.Invoke(seeding);
Thread.Sleep(10);
}
}
Application started:
After starting seeding process:
After about 10min:
After another 10-20min:
And it goes continues to climb from here...
Any ideas for me here pretty please? :)
https://www.jetbrains.com/dotmemory
Could you use dotmemory (or any other profiler you're familiar with) to track allocations and identify what's consuming all the memory? If you use dotmemory and are comfortable zipping and sharing the data capture, I can analyse it myself .
Overall that's pretty surprising as I haven't seen this kind of problem locally and I do have a stress test which spins up 16 copies of MonoTorrent as leechers, and 1 as a seeder. The stress test redownloads the same data repeatedly and so should emulate this reasonably well.
That breaks down to be about 25MB per 'client' as there is 1 seeder and 16 leechers. After the 'seeder' uploads 11gb, and the leechers share 9GB between them, leading to a total upload/download of over 20GB each way, memory is still fairly static.
Whatever is triggering the memory leak for you is not being triggered in this testcase : https://github.com/alanmcgovern/monotorrent/blob/master/src/Samples/SampleClient/StressTest.cs
I also tweaked the testcase so it actually performed actual read/write operations on disk rather than using an in-memory 'Null' writer (I deleted the Factories.Default.WithPieceWriterCreator (maxOpenFiles => new NullWriter ())) line) and it had no effect on memory growth over time .
Hey Alan, thanks for always being so helpful. I really do appreciate it.
I downloaded DotMemory and ran the software together with my app.
I have no idea what all the info is telling me, but hopefully you will be able to shed some light on this.
I did an export.. not sure if this will work?
https://www.dropbox.com/s/4lbikp04u3r0jmt/Launcher_DotMemory.7z?dl=1
Firstly, that's perfect! This was really helpful in beginning to understand what's happening!
There are a few obvious ish issues which stand out to me. I'll start with the easiest one first - there's an awful lot of unexpected memory allocation originating from Tracker manager in that dump.
Does this memory issue happen for one specific torrent, or all torrents? If it's all torrents, is there anything in your code which is explicitly calling any of the Announce or Scrape methods in MonoTorrent? If so, does the problem go away if you stop invoking those methods?
If this is entirely implicit (i.e. it's occurring just because you start a torrent) could you share the tracker portion of the .torrent file, or just the whole .torrent file? Maybe there's an infinite loop being triggered by the set of trackers listed in that torrent.
If you want to share just a subset of the torrent, there are plenty of viewers for bencoded data, so just copy/paste the section which contains the tracker data. Alternatively, you can load the torrent using BEncodedDictionary.Load (or whatever the API is called :p ), then pull out the BEncodedList which represents the trackers, call Encode on that and write the bytes to a file. I can load that up and test things out then.
I can double check that there are no unbounded loops in there, though without a repro it's hard to know what to look for exactly :)
Thanks Alan
My code does have in:
await dhtEngine.StartAsync();
await manager.StartAsync();
await manager.LocalPeerAnnounceAsync();`
await manager.DhtAnnounceAsync();
But even if I remove the last two the same thing happens.
This happens with all the torrents not just one, but I use the same torrent tracker list for all of them.
I normally just use those huge torrent lists from the internet.. maybe there is an issue with that list?
tracker list.txt
link to torrent also in there if you want to check the torrent file itself. ;)
I can also just mention that the smaller the torrent download, the longer it takes for that memory to build up.
With the one file of 43GB it build up very quick. With another one that is only 800MB, it increases very slowly.
not sure if that is of any help hehe.
Oh and I did do a test where I only used 5 trackers. Recreated the torrent and tested again, same problem though. So I highly doubt the tracker list has anything to do with this.
Wait a second... haha..
It seemed to increase slightly and then stopped.
I tried with a bigger file now.. and so far.. memory is stable at around 54MB! :)
Maybe the issue was that huge tracker list then?
If you use custom tracker list, maybe it's related to #450
I also use a tracker list for all the torrent
If you use custom tracker list, maybe it's related to #450 I also use a tracker list for all the torrent
Thanks, I don't use that code (await AnnounceLimiter.EnterAsync ())
So don't think i have exactly same issue, but the memory leak seemed to be similar.
After using only like 5 trackers when creating my torrent file, everything seems to be going really well now. :)
I'll leave this, and the other issue, open until I can fix the underlying problem! :) With a repro this should be easy!
If you're comfortable testing from a PR/branch, i think this will resolve the issue.
This changes how rate limiting occurs and should solve the memory issue. Essentially TrackerManager.AnnounceAsync can now be invoked many times sequentially. During the first invocation, the (exclusive!) semaphore will be entered and the regular "announce to each tracker tier" logic will execute.
The second and subsequent invocations to TrackerManager.AnnounceAsync will either be immediately discarded (if they are regular recurring non-special announces), or they'll be enqueue and wait for the active announce to complete.
Previously the rate limiting was applied deeper in the stack, which could lead to multiple consecutive invocations to TrackerManager.AnnounceAsync creating a pretty deep queue of announce events, depending on how many trackers there were in the queue.
There are likely some more tweaks to be made, and some tests to add, before I merge this though!