Giters

hopfenspace / MateBot

Micro service providing an API for MateBot clients

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Improve SQL Queries

myOmikron opened this issue · comments

commented

As for the time, there are few SQL queries called when using commands like /history to determine the specific users the transaction are from or to. This could be simplified with merging the queries and query the database only once.

commented

I would prefer another way: caching results. If you look into the logs, it's clear what's happening:

matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (1,)
matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (1,)
matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (1,)
matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (1,)
matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (39,)
[...]
matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (42,)
matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (39,)
matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (42,)
matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (39,)
matebot 2266: DEBUG: database: Executing 'SELECT name FROM users WHERE id=%s' using args (1,)

This happens because the MateBotUser object is created twice for every single record. I agree, this is not a good way to do this. Therefore, I would prefer caching, either locally in the /history command's code or globally in BackendHelper.

commented

Why bother with caching when you have a database and no real reason to cache. Just improve the Queries to get your desired results in just one query instead of multiples

commented

This has been addressed in 5b8aed7 using basic memoization. Now, those database calls are reduced to exactly one call per username.

commented

I will close this issue now as "too broad". I fixed the lots of queries when using the /history command, of course. If there are other cases in which heavy load on the SQL server is found, we can still open another issue for it.