Giters

LonamiWebs / Telethon

Pure Python 3 MTProto API Telegram client library, for bots too!

Home Page:https://docs.telethon.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Memory continues to increase

v2boardbot opened this issue · comments

commented

Code that causes the issue

message = await self.client.send_message(entity=int(chat_id), message=message_text)

Expected behavior

none

Actual behavior

Memory overflow, memory exceeds normal value.

Traceback

2024-05-23 05:27:04,785 INFO : RAM (MB): client  3.63 | _updates_queue 4 | all_tasks 55 | event_handlers []
2024-05-23 05:27:08,020 INFO : RAM (MB): client  3.45 | _updates_queue 0 | all_tasks 56 | event_handlers []
2024-05-23 05:27:10,902 INFO : RAM (MB): client  2.39 | _updates_queue 0 | all_tasks 57 | event_handlers []
2024-05-23 05:27:14,307 INFO : RAM (MB): client  3.92 | _updates_queue 7 | all_tasks 54 | event_handlers []
2024-05-23 05:27:17,382 INFO : RAM (MB): client  4.58 | _updates_queue 0 | all_tasks 56 | event_handlers []
2024-05-23 05:27:20,202 INFO : RAM (MB): client  2.77 | _updates_queue 0 | all_tasks 57 | event_handlers []
2024-05-23 05:27:24,722 INFO : RAM (MB): client  4.01 | _updates_queue 11 | all_tasks 55 | event_handlers []
2024-05-23 05:27:28,442 INFO : RAM (MB): client  4.61 | _updates_queue 0 | all_tasks 57 | event_handlers []
2024-05-23 05:27:32,025 INFO : RAM (MB): client  5.11 | _updates_queue 0 | all_tasks 60 | event_handlers []
2024-05-23 05:27:36,188 INFO : RAM (MB): client  5.42 | _updates_queue 0 | all_tasks 55 | event_handlers []
2024-05-23 05:27:40,430 INFO : RAM (MB): client  6.60 | _updates_queue 0 | all_tasks 60 | event_handlers []
2024-05-23 05:27:45,261 INFO : RAM (MB): client  7.28 | _updates_queue 0 | all_tasks 53 | event_handlers []
2024-05-23 05:27:50,962 INFO : RAM (MB): client  8.59 | _updates_queue 27 | all_tasks 56 | event_handlers []
2024-05-23 05:27:54,571 INFO : RAM (MB): client  9.68 | _updates_queue 2 | all_tasks 57 | event_handlers []
2024-05-23 05:27:58,218 INFO : RAM (MB): client 10.68 | _updates_queue 0 | all_tasks 57 | event_handlers []
2024-05-23 05:28:03,469 INFO : RAM (MB): client 11.82 | _updates_queue 12 | all_tasks 57 | event_handlers []
2024-05-23 05:28:08,933 INFO : RAM (MB): client 12.45 | _updates_queue 22 | all_tasks 56 | event_handlers []
2024-05-23 05:28:14,953 INFO : RAM (MB): client 13.99 | _updates_queue 8 | all_tasks 57 | event_handlers []
2024-05-23 05:28:19,810 INFO : RAM (MB): client 15.00 | _updates_queue 0 | all_tasks 55 | event_handlers []
2024-05-23 05:28:25,213 INFO : RAM (MB): client 15.46 | _updates_queue 0 | all_tasks 55 | event_handlers []
2024-05-23 05:28:31,987 INFO : RAM (MB): client 17.05 | _updates_queue 3 | all_tasks 57 | event_handlers []
2024-05-23 05:28:37,964 INFO : RAM (MB): client 18.52 | _updates_queue 4 | all_tasks 54 | event_handlers []
2024-05-23 05:28:43,479 INFO : RAM (MB): client 20.45 | _updates_queue 0 | all_tasks 54 | event_handlers []
2024-05-23 05:28:48,458 INFO : RAM (MB): client 21.40 | _updates_queue 3 | all_tasks 54 | event_handlers []
2024-05-23 05:28:54,251 INFO : RAM (MB): client 21.96 | _updates_queue 0 | all_tasks 55 | event_handlers []
2024-05-23 05:29:02,068 INFO : RAM (MB): client 22.55 | _updates_queue 3 | all_tasks 59 | event_handlers []
2024-05-23 05:29:10,631 INFO : RAM (MB): client 24.51 | _updates_queue 68 | all_tasks 57 | event_handlers []
2024-05-23 05:29:20,275 INFO : RAM (MB): client 26.48 | _updates_queue 77 | all_tasks 54 | event_handlers []
2024-05-23 05:29:30,420 INFO : RAM (MB): client 28.22 | _updates_queue 101 | all_tasks 56 | event_handlers []
2024-05-23 05:29:38,924 INFO : RAM (MB): client 29.32 | _updates_queue 133 | all_tasks 54 | event_handlers []
2024-05-23 05:29:47,755 INFO : RAM (MB): client 31.98 | _updates_queue 146 | all_tasks 57 | event_handlers []
2024-05-23 05:29:57,400 INFO : RAM (MB): client 34.60 | _updates_queue 153 | all_tasks 54 | event_handlers []
2024-05-23 05:30:09,141 INFO : RAM (MB): client 35.86 | _updates_queue 183 | all_tasks 52 | event_handlers []
2024-05-23 05:30:24,086 INFO : RAM (MB): client 38.38 | _updates_queue 272 | all_tasks 56 | event_handlers []
2024-05-23 05:37:52,205 INFO : RAM (MB): client 147.94 | _updates_queue 0 | all_tasks 57 | event_handlers []
2024-05-23 05:38:28,130 INFO : RAM (MB): client 157.57 | _updates_queue 123 | all_tasks 51 | event_handlers []
2024-05-23 05:39:09,490 INFO : RAM (MB): client 169.14 | _updates_queue 276 | all_tasks 52 | event_handlers []
2024-05-23 05:39:54,486 INFO : RAM (MB): client 178.32 | _updates_queue 397 | all_tasks 51 | event_handlers []
2024-05-23 05:40:33,603 INFO : RAM (MB): client 185.00 | _updates_queue 405 | all_tasks 51 | event_handlers []
2024-05-23 05:41:14,758 INFO : RAM (MB): client 190.72 | _updates_queue 503 | all_tasks 51 | event_handlers []
2024-05-23 05:42:00,839 INFO : RAM (MB): client 198.83 | _updates_queue 670 | all_tasks 51 | event_handlers []
2024-05-23 05:42:55,819 INFO : RAM (MB): client 206.99 | _updates_queue 791 | all_tasks 53 | event_handlers []
2024-05-23 05:43:49,172 INFO : RAM (MB): client 215.40 | _updates_queue 881 | all_tasks 53 | event_handlers []
2024-05-23 05:44:46,162 INFO : RAM (MB): client 222.08 | _updates_queue 1004 | all_tasks 54 | event_handlers []
2024-05-23 05:45:44,588 INFO : RAM (MB): client 229.07 | _updates_queue 1085 | all_tasks 51 | event_handlers []
2024-05-23 05:46:51,018 INFO : RAM (MB): client 237.87 | _updates_queue 1263 | all_tasks 52 | event_handlers []
2024-05-23 05:47:51,334 INFO : RAM (MB): client 245.79 | _updates_queue 1473 | all_tasks 53 | event_handlers []
2024-05-23 05:48:49,196 INFO : RAM (MB): client 253.38 | _updates_queue 1589 | all_tasks 51 | event_handlers []
2024-05-23 05:49:59,084 INFO : RAM (MB): client 262.17 | _updates_queue 1725 | all_tasks 52 | event_handlers []
2024-05-23 05:51:03,997 INFO : RAM (MB): client 269.31 | _updates_queue 1924 | all_tasks 52 | event_handlers []
2024-05-23 05:51:57,587 INFO : RAM (MB): client 277.28 | _updates_queue 2107 | all_tasks 51 | event_handlers []
2024-05-23 05:52:59,002 INFO : RAM (MB): client 287.42 | _updates_queue 2248 | all_tasks 54 | event_handlers []
2024-05-23 05:54:09,670 INFO : RAM (MB): client 297.67 | _updates_queue 2442 | all_tasks 52 | event_handlers []
2024-05-23 05:55:19,240 INFO : RAM (MB): client 307.61 | _updates_queue 2636 | all_tasks 75 | event_handlers []
2024-05-23 05:56:31,259 INFO : RAM (MB): client 318.92 | _updates_queue 2848 | all_tasks 51 | event_handlers []
2024-05-23 05:57:46,729 INFO : RAM (MB): client 331.07 | _updates_queue 3072 | all_tasks 52 | event_handlers []
2024-05-23 05:59:06,321 INFO : RAM (MB): client 341.53 | _updates_queue 3306 | all_tasks 51 | event_handlers []
2024-05-23 06:00:24,682 INFO : RAM (MB): client 358.96 | _updates_queue 3574 | all_tasks 51 | event_handlers []
2024-05-23 06:01:45,388 INFO : RAM (MB): client 367.57 | _updates_queue 3756 | all_tasks 52 | event_handlers []
2024-05-23 06:03:00,179 INFO : RAM (MB): client 380.45 | _updates_queue 3997 | all_tasks 51 | event_handlers []
2024-05-23 06:04:19,030 INFO : RAM (MB): client 388.97 | _updates_queue 4220 | all_tasks 52 | event_handlers []
2024-05-23 06:05:44,454 INFO : RAM (MB): client 401.01 | _updates_queue 4453 | all_tasks 52 | event_handlers []
2024-05-23 06:07:03,469 INFO : RAM (MB): client 410.66 | _updates_queue 4632 | all_tasks 52 | event_handlers []
2024-05-23 06:08:23,753 INFO : RAM (MB): client 421.62 | _updates_queue 4830 | all_tasks 53 | event_handlers []
2024-05-23 06:09:50,630 INFO : RAM (MB): client 434.31 | _updates_queue 5088 | all_tasks 53 | event_handlers []
2024-05-23 06:11:25,056 INFO : RAM (MB): client 449.94 | _updates_queue 5348 | all_tasks 51 | event_handlers []
2024-05-23 06:12:55,343 INFO : RAM (MB): client 459.75 | _updates_queue 5583 | all_tasks 52 | event_handlers []
2024-05-23 06:14:26,025 INFO : RAM (MB): client 475.15 | _updates_queue 5865 | all_tasks 51 | event_handlers []
2024-05-23 06:15:55,620 INFO : RAM (MB): client 485.19 | _updates_queue 6114 | all_tasks 52 | event_handlers []
2024-05-23 06:17:29,575 INFO : RAM (MB): client 497.56 | _updates_queue 6175 | all_tasks 53 | event_handlers []
2024-05-23 06:19:16,563 INFO : RAM (MB): client 508.44 | _updates_queue 6379 | all_tasks 52 | event_handlers []
2024-05-23 06:21:08,524 INFO : RAM (MB): client 525.12 | _updates_queue 6679 | all_tasks 52 | event_handlers []
2024-05-23 06:22:51,044 INFO : RAM (MB): client 537.24 | _updates_queue 6987 | all_tasks 51 | event_handlers []
2024-05-23 06:24:37,400 INFO : RAM (MB): client 551.98 | _updates_queue 7239 | all_tasks 51 | event_handlers []
2024-05-23 06:26:14,167 INFO : RAM (MB): client 563.88 | _updates_queue 7551 | all_tasks 51 | event_handlers []
2024-05-23 06:28:08,806 INFO : RAM (MB): client 576.05 | _updates_queue 7792 | all_tasks 51 | event_handlers []
2024-05-23 06:30:17,140 INFO : RAM (MB): client 593.44 | _updates_queue 8165 | all_tasks 51 | event_handlers []
2024-05-23 06:32:08,026 INFO : RAM (MB): client 613.51 | _updates_queue 8514 | all_tasks 51 | event_handlers []
2024-05-23 06:33:58,429 INFO : RAM (MB): client 625.36 | _updates_queue 8814 | all_tasks 52 | event_handlers []
2024-05-23 06:36:10,825 INFO : RAM (MB): client 639.24 | _updates_queue 9108 | all_tasks 52 | event_handlers []
2024-05-23 06:38:25,312 INFO : RAM (MB): client 659.02 | _updates_queue 9468 | all_tasks 52 | event_handlers []
2024-05-23 06:40:51,962 INFO : RAM (MB): client 678.62 | _updates_queue 9905 | all_tasks 51 | event_handlers []
2024-05-23 06:42:53,993 INFO : RAM (MB): client 699.11 | _updates_queue 10346 | all_tasks 51 | event_handlers []
2024-05-23 06:45:17,246 INFO : RAM (MB): client 712.15 | _updates_queue 10668 | all_tasks 52 | event_handlers []
2024-05-23 06:48:03,267 INFO : RAM (MB): client 726.42 | _updates_queue 11055 | all_tasks 52 | event_handlers []

Telethon version

1.34.0

Python version

3.10.12

Operating system (including distribution name and version)

Ubuntu 22.04

Other details

I have many groups, and each group receives many messages every second.

Checklist

  • The error is in the library's code, and not in my own.
  • I have searched for this issue before posting it and there isn't an open duplicate.
  • I ran pip install -U https://github.com/LonamiWebs/Telethon/archive/v1.zip and triggered the bug in the latest version.
commented

_updates_queue shouldn't grow forever, unless your handlers are taking too long. Is that the case?

If the handlers aren't being called fast enough, the only other thing the library could do is drop updates, but that won't work for everyone.

commented

I didn't add any event_handlers.
2024-05-23 05:37:52,205 _updates_queue = 0 ; client ram : 38.38MB
2024-05-23 05:37:52,205 _updates_queue = 123 ; client ram : 147.94MB

Is there a direct relationship between the size of _updates_queue and memory?

2024-05-23 05:29:38,924 INFO : RAM (MB): client 29.32 | _updates_queue 133 | all_tasks 54 | event_handlers []
2024-05-23 05:29:47,755 INFO : RAM (MB): client 31.98 | _updates_queue 146 | all_tasks 57 | event_handlers []
2024-05-23 05:29:57,400 INFO : RAM (MB): client 34.60 | _updates_queue 153 | all_tasks 54 | event_handlers []
2024-05-23 05:30:09,141 INFO : RAM (MB): client 35.86 | _updates_queue 183 | all_tasks 52 | event_handlers []
2024-05-23 05:30:24,086 INFO : RAM (MB): client 38.38 | _updates_queue 272 | all_tasks 56 | event_handlers []
2024-05-23 05:37:52,205 INFO : RAM (MB): client 147.94 | _updates_queue 0 | all_tasks 57 | event_handlers []
2024-05-23 05:38:28,130 INFO : RAM (MB): client 157.57 | _updates_queue 123 | all_tasks 51 | event_handlers []
2024-05-23 05:39:09,490 INFO : RAM (MB): client 169.14 | _updates_queue 276 | all_tasks 52 | event_handlers []
2024-05-23 05:39:54,486 INFO : RAM (MB): client 178.32 | _updates_queue 397 | all_tasks 51 | event_handlers []
commented

It seemed like there was from your first post. But if you periodically clear the queue and memory continues to increase, then I guess not.

commented

I don't need to receive new messages, and my application will actively request the Telegram server when needed. Can I delete it?
self.loop.create_task(self._update_loop())