lepinkainen / pyfibot

Pyfibot the Python IRC bot

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

pyfibot sends *two* HTTP requests for each observed URL

resilar opened this issue · comments

module_urltitle.py and module_urlsize.py both request a URL separately in order to post the URL title to the channel mentioning the URL. The underlying logic and reasoning seems to be that module_urltitle.py ignores URLs with large content length, in which case module_urlisze.py summarizes the size and mime type of the URL and says it on the mentioning channel. Respectively, module_urlsize.py silently ignores the URL if the content length is not considered large, in which case module_urltitle.py extracts the URL title and says it on the channel. This is all fine and dandy, but a bit inefficient approach and looks stupid in access.log of the server hosting the URL.

As a fix, I'd suggest merging module_urlsize.py logic into module_urltitle.py since both modules essentially implement URL title posting functionality and could easily use the same Response object; thus, eliminating the superfluous second HTTP request currently performed per seen URL.

👍 permission granted.

Fix'd