Using Chatlogger, anything with Emoji's, nothing is logged (UnicodeEncodeError: 'charmap' codec)

[robklob]

Recently upgraded to version 3 due to the ds:7 error. Things are working well except whenever there's an emoji, nothing gets logged (can't get past it) and errors on encoding show up. Note the log on a test text with emoji's. Same errors if there are emoji's in the user's name fields.

C:\Python36\hangoutsbot3\hangoutsbot>python hangupsbot/hangupsbot.py
21:52:58 INFO config: C:\Users\USER\AppData\Local\hangupsbot\hangupsbot\config.json read
21:52:58 INFO root: memory = C:\Users\USER\AppData\Local\hangupsbot\hangupsbot\memory.json, failsafe = 3, delay = 1
21:52:58 INFO config: C:\Users\USER\AppData\Local\hangupsbot\hangupsbot\memory.json read
21:52:58 INFO requests.packages.urllib3.connectionpool: Starting new HTTPS connection (1): accounts.google.com
21:52:59 INFO requests.packages.urllib3.connectionpool: Starting new HTTPS connection (1): www.google.com
21:52:59 INFO tagging: refreshed
21:53:01 INFO permamem: loading 133 conversations from memory
21:53:01 INFO permamem: incomplete users: {'106626091637413336975': 'Unknown', '114620570984948619503': 'Unknown', '107676144177290501032': 'Unknown', '102475403957292218240': 'Unknown', '107595435936851686671': 'Unknown', '104280683331843532548': 'Unknown', '107721817594367868021': 'Unknown', '114912874763041858022': 'Unknown', '109138306546380829111': 'Unknown', '108260940164349407650': 'Unknown', '111526860345907373407': 'Unknown', '112023883024346134174': 'Unknown', '106676602700985955613': 'Unknown', '118161857036077076384': 'Unknown', '100901858950048284338': 'Unknown', '108772205651010133072': 'Unknown', '114792334757268834967': 'Unknown', '113258879149218617018': 'Unknown', '108982894813151333972': 'Unknown', '116447075444551726546': 'Unknown', '116615275532012307661': 'Unknown', '114169851820863199347': 'Unknown'}
21:53:03 INFO permamem: loading 86 users from hangups
21:53:03 INFO permamem: loading 97 conversations from hangups
21:53:03 INFO permamem: total conversations: 133
21:53:03 INFO permamem: total users: 87 cached: 87 definitive (at start): 65
21:53:04 INFO plugins: plugincontrol - *addplugin, *getplugins, *plugininfo, *pluginload, *pluginreload, *pluginunload, *removeplugin
21:53:04 WARNING commands.basic: resource is unavailable on your system
21:53:04 INFO plugins: basic - help, *locale, optout, ping, *resourcememory, version
21:53:04 INFO plugins: tagging - *tagdel, *tagindexdump, *tagscommand, *tagsconv, *tagset, *tagspurge, *tagsuser, *tagsuserlist
21:53:04 INFO plugins: permamem - *dumpconv, *dumpunknownusers, *makeallusersindefinite, *refreshusermemory, *removeconvrecord, *resetunknownusers
21:53:04 INFO plugins: convid has legacy hangups reference
21:53:04 INFO plugins: convid - *convecho, *convfilter, *convleave, *convrename, *convusers
21:53:04 INFO plugins: loggertochat - *logconfig, *lograise
21:53:04 INFO plugins: excluded 59: ['api', 'botaddnotif', 'botalive', 'catfact', 'chance', 'cleverbot', 'convtools', 'convtools_invitations', 'easteregg', 'forecast', 'forwarding', 'foursquare', 'hubot', 'humor_hangoutcalls', 'image', 'image.image_linker_reddit', 'image.image_links', 'image.image_memegenerator', 'image.image_screenshot', 'lookup', 'lottery', 'metar', 'monitoradds', 'namelock', 'restrictedadd', 'showme', 'simplewikipedia', 'slack', 'slackrtm', 'spawn', 'spotify', 'syncrooms', 'syncrooms_config', 'telesync', 'tldr', 'twitter', 'urbandict', 'wolframalpha', 'xkcd', '_cam_mail_intercept', '_chatbridge', '_chatbridge.chatbridge_slack', '_chatbridge.chatbridge_syncrooms', '_chatbridge.chatbridge_telegram', '_DEPRECATED_simplytranslate', '_DEPRECATED_syncrooms_autotranslate', '_example', '_example.example_argpreprocessor', '_example.example_memory', '_example.example_printrenames', '_example.example_watchmembers', '_example.example_webbridge', '_unittest', '_unittest.unittest_context', '_unittest.unittest_geticon', '_unittest.unittest_memory', '_unittest.unittest_reprocessor', '_unittest.unittest_shutup', 'unittest.unittest_statusevents']
21:53:04 INFO plugins: included 9: ['botaliases', 'default', 'mentions', 'subscribe', 'autoreply', 'dnd', 'starter', 'chatlogger', 'remind']
21:53:04 INFO plugins.botaliases: aliases: ['/bot', '/robert', '/101174431731905533335']
21:53:04 INFO plugins: botaliases - botalias
21:53:04 INFO plugins: default has legacy hangups reference
21:53:04 INFO plugins: default - *broadcast, *config, echo, *hangouts, *leave, *quit, *reload, *rename, *user, *users, *whereami, whoami
21:53:04 INFO plugins: mentions - bemorespecific, *mention, pushbulletapi, setnickname
21:53:04 INFO plugins: subscribe - subscribe, *testsubscribe, unsubscribe
21:53:04 INFO plugins: autoreply has legacy hangups reference
21:53:04 INFO plugins: autoreply - *autoreply
21:53:04 INFO plugins: dnd - dnd
21:53:04 INFO plugins: starter - *files, iamspartacus, *test_one2one_message, verifyme
21:53:04 INFO plugins: chatlogger has legacy hangups reference
21:53:05 INFO plugins.chatlogger: stored in: //SHAREDFOLDER
21:53:05 INFO plugins: chatlogger - no commands
21:53:05 INFO plugins: remind - remindall, remindme
21:53:05 INFO root: bot initialised
21:53:29 INFO event: eid/dt: 8hSk4OWR-Fl8hVGpKGTZfh/2018-06-14 21:53:29
21:53:29 INFO event: cid/cn: UgxBRdjFPTuucbA45U94AaABAagBoIzGCA/[phone]
21:53:29 INFO event: c/g/un: 117945502309581643208/117945502309581643208/[phone]
-**-- Logging error ---
Traceback (most recent call last):
File "C:\Python36\lib\logging_init.py", line 994, in emit
stream.write(msg)
File "C:\Python36\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 50-54: character maps to **
Call stack:
File "hangupsbot/hangupsbot.py", line 1009, in
main()
File "hangupsbot/hangupsbot.py", line 1005, in main
bot.run()
File "hangupsbot/hangupsbot.py", line 164, in run
loop.run_until_complete(self._client.connect())
File "C:\Python36\lib\asyncio\base_events.py", line 454, in run_until_complete
self.run_forever()
File "C:\Python36\lib\asyncio\base_events.py", line 421, in run_forever
self._run_once()
File "C:\Python36\lib\asyncio\base_events.py", line 1431, in _run_once
handle._run()
File "C:\Python36\lib\asyncio\events.py", line 145, in _run
self._callback(self._args)
File "C:\Python36\lib\site-packages\hangups\channel.py", line 197, in listen
await self._longpoll_request()
File "C:\Python36\lib\site-packages\hangups\channel.py", line 298, in _longpoll_request
await self._on_push_data(chunk)
File "C:\Python36\lib\site-packages\hangups\channel.py", line 332, in _on_push_data
await self.on_receive_array.fire(data_array)
File "C:\Python36\lib\site-packages\hangups\event.py", line 59, in fire
await gen
File "C:\Python36\lib\site-packages\hangups\client.py", line 370, in _on_receive_array
await self.on_state_update.fire(state_update)
File "C:\Python36\lib\site-packages\hangups\event.py", line 59, in fire
await gen
File "C:\Python36\lib\site-packages\hangups\conversation.py", line 831, in _on_state_update
state_update.event_notification.event
File "C:\Python36\lib\site-packages\hangups\conversation.py", line 882, in _on_event
await self.on_event.fire(conv_event)
File "C:\Python36\lib\site-packages\hangups\event.py", line 59, in fire
await gen
File "hangupsbot/hangupsbot.py", line 608, in _on_event
event = ConversationEvent(self, conv_event)
File "C:\Python36\hangoutsbot3\hangoutsbot\hangupsbot\event.py", line 81, in init
self.log()
File "C:\Python36\hangoutsbot3\hangoutsbot\hangupsbot\event.py", line 89, in log
logger.log(self.emit_log, 'len/tx: {}/{}'.format(len(self.text), self.text))
Message: 'len/tx: 12/Ggftgyy😗😝😝😝😝'
Arguments: ()
21:53:29 INFO event: len/tx: 12/Ggftgyy😗😝😝😝😝
21:53:30 ERROR handlers: allmessages: plugins.chatlogger.on_chat_message : coroutine
Traceback (most recent call last):
File "C:\Python36\hangoutsbot3\hangoutsbot\hangupsbot\handlers.py", line 412, in run_pluggable_omnibus
yield from function(_passed)
File "C:\Python36\lib\asyncio\coroutines.py", line 212, in coro
res = func(*args, **kw)
File "C:\Python36\hangoutsbot3\hangoutsbot\hangupsbot\plugins\chatlogger.py", line 78, in on_chat_message
self._append_to_file(conversation_id, text)
File "C:\Python36\hangoutsbot3\hangoutsbot\hangupsbot\plugins\chatlogger.py", line 63, in _append_to_file
logfile.write(text)
File "C:\Python36\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 75-79: character maps to

Call stack:
File "hangupsbot/hangupsbot.py", line 1009, in
main()
File "hangupsbot/hangupsbot.py", line 1005, in main
bot.run()
File "hangupsbot/hangupsbot.py", line 164, in run
loop.run_until_complete(self._client.connect())
File "C:\Python36\lib\asyncio\base_events.py", line 454, in run_until_complete
self.run_forever()
File "C:\Python36\lib\asyncio\base_events.py", line 421, in run_forever
self._run_once()
File "C:\Python36\lib\asyncio\base_events.py", line 1431, in _run_once
handle._run()
File "C:\Python36\lib\asyncio\events.py", line 145, in _run
self._callback(*self._args)
File "C:\Python36\lib\site-packages\hangups\channel.py", line 197, in listen
await self._longpoll_request()
File "C:\Python36\lib\site-packages\hangups\channel.py", line 298, in _longpoll_request
await self._on_push_data(chunk)
File "C:\Python36\lib\site-packages\hangups\channel.py", line 332, in _on_push_data
await self.on_receive_array.fire(data_array)
File "C:\Python36\lib\site-packages\hangups\event.py", line 59, in fire
await gen
File "C:\Python36\lib\site-packages\hangups\client.py", line 370, in _on_receive_array
await self.on_state_update.fire(state_update)
File "C:\Python36\lib\site-packages\hangups\event.py", line 59, in fire
await gen
File "C:\Python36\lib\site-packages\hangups\conversation.py", line 831, in _on_state_update
state_update.event_notification.event
File "C:\Python36\lib\site-packages\hangups\conversation.py", line 882, in _on_event
await self.on_event.fire(conv_event)
File "C:\Python36\lib\site-packages\hangups\event.py", line 59, in fire
await gen
File "hangupsbot/hangupsbot.py", line 608, in _on_event
event = ConversationEvent(self, conv_event)
File "C:\Python36\hangoutsbot3\hangoutsbot\hangupsbot\event.py", line 81, in init
self.log()
File "C:\Python36\hangoutsbot3\hangoutsbot\hangupsbot\event.py", line 89, in log
logger.log(self.emit_log, 'len/tx: {}/{}'.format(len(self.text), self.text))
Message: 'len/tx: 12/Ggftgyy😗😝😝😝😝'
Arguments: ()

(Note, I'm not that creative when I test . . . )

[robklob]

Solution was here: Chatlogger does not support non-ascii characters #741

From that post, nerlinger:

Fix is easy:
First, import io on line 1:
import os, logging, io
Then, change line 62 to be utf8-aware:
with io.open(conversation_log, "a", encoding="utf-8") as logfile:

Todo:
Compatibility might be an issue, as existing log files are not utf8-aware.