Suggestion: Better logging
bokerlol opened this issue · comments
I would absolutely love for better logging in these scripts. Particularly the auto-tagger but really all of them could use it. As an example, I keep getting this error and it does nothing but tell me 'image dimensions too small'. While that is nice to know, it doesn't really help me narrow down which image in particular is the one that is too small.
Z:\Misc\szurubooru\szurubooru-scripts-main>python3 auto_tagger.py tag-count:0
Found 117 posts. Start tagging...
0%| | 0/117 [00:00<?, ?it/s]image dimensions too small...
Could not get result from SauceNAO with uploaded image: image dimensions too small...
Traceback (most recent call last):
File "auto_tagger.py", line 129, in <module>
main()
File "auto_tagger.py", line 65, in main
tags, source, rating, limit_short, limit_long = saucenao.get_metadata(post)
File "C:\Users\shrev\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\syncer.py", line 44, in run
return asyncio.get_event_loop().run_until_complete(f(*args, **kwargs))
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2800.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 616, in run_until_complete
return future.result()
File "Z:\Misc\szurubooru\szurubooru-scripts-main\classes\saucenao.py", line 64, in get_metadata
if results.results:
AttributeError: 'NoneType' object has no attribute 'results'
So some improved logging could really go a long way with these scripts. You may already be working on something but wanted to throw it out there anyways.
Thanks for the suggestion and yes, I was already planning to improve logging capabilities
I'm currently implementing loguru for logging. The user will be able to specify following settings in config.json (well, config.toml in the future): log_level, log_dir and log_retention.
While more detailed warnings are getting printed to the terminal by default, the user will also have the option to set the log_level up to debug where the output (to a file) will be something like:
[DEBUG] [25.03.2022, 21:27:47 CET] [szurubooru.__init__]: szuru_user = <szuru_user>
[DEBUG] [25.03.2022, 21:27:47 CET] [szurubooru.__init__]: szuru_url = <szuru_url>
[DEBUG] [25.03.2022, 21:27:47 CET] [szurubooru.__init__]: szuru_api_url = <szuru_api_url>
[DEBUG] [25.03.2022, 21:27:47 CET] [saucenao.__init__]: Using SauceNAO API token
[DEBUG] [25.03.2022, 21:27:47 CET] [danbooru.__init__]: Using Danbooru user <danbooru_user> with API key
[DEBUG] [25.03.2022, 21:27:47 CET] [gelbooru.__init__]: Using Gelbooru without user and API key
[DEBUG] [25.03.2022, 21:27:51 CET] [szurubooru.get_posts]: Modified input query to "id:14674"
[DEBUG] [25.03.2022, 21:27:51 CET] [szurubooru.get_posts]: Getting post from query_url: https://szuru.example.com/api/posts/?query=id:14674
[DEBUG] [25.03.2022, 21:27:51 CET] [szurubooru.get_posts]: Got a total of 1 results
[DEBUG] [25.03.2022, 21:27:51 CET] [szurubooru.get_posts]: Searching across 1 pages
[DEBUG] [25.03.2022, 21:27:51 CET] [szurubooru.parse_post]: Parsing post with input: <szuru_result_dict>
[DEBUG] [25.03.2022, 21:27:51 CET] [szurubooru.parse_post]: Returning Post object: <post_object_attrs>
[DEBUG] [25.03.2022, 21:27:51 CET] [saucenao.get_result]: Trying to get result from tmp_file: /tmp/14674_dcf2627cc2fdf294.jpg
[DEBUG] [25.03.2022, 21:27:52 CET] [saucenao.get_result]: Received response <SauceNaoResults(count=1, short_avail=24, long_avail=4615, results=[<GenericSourc...e='Danbooru')>])>
[DEBUG] [25.03.2022, 21:27:53 CET] [danbooru.get_result]: Returning result: <danbooru_result_dict>
[DEBUG] [25.03.2022, 21:27:53 CET] [danbooru.get_tags]: Returning tags: <tags>
[DEBUG] [25.03.2022, 21:27:53 CET] [danbooru.get_rating]: Returning rating: s
[DEBUG] [25.03.2022, 21:27:53 CET] [utils.convert_rating]: Converted rating s to safe
[DEBUG] [25.03.2022, 21:27:53 CET] [saucenao.get_metadata]: Limit short: 24
[DEBUG] [25.03.2022, 21:27:53 CET] [saucenao.get_metadata]: Limit long: 4615
[DEBUG] [25.03.2022, 21:27:53 CET] [saucenao.get_metadata]: Found no result in Gelbooru
[DEBUG] [25.03.2022, 21:27:53 CET] [saucenao.get_metadata]: Found no result in Sankaku
[DEBUG] [25.03.2022, 21:27:53 CET] [saucenao.get_metadata]: Returning tags: <collected_tags>
[DEBUG] [25.03.2022, 21:27:53 CET] [saucenao.get_metadata]: Returning sources: <collected_sources>
[DEBUG] [25.03.2022, 21:27:53 CET] [saucenao.get_metadata]: Returning rating: <rating>
[DEBUG] [25.03.2022, 21:27:53 CET] [szurubooru.update_post]: Updating following post: <post_object>
[DEBUG] [25.03.2022, 21:27:53 CET] [szurubooru.update_post]: Using query_url: https://szuru.example.com/api/post/14674
[DEBUG] [25.03.2022, 21:27:53 CET] [szurubooru.update_post]: Using payload: <payload_dict>
Let me know if you have any other ideas/suggestions!