Strange behaviour when starting up in the morning
fritzzjan opened this issue · comments
First of all: thank You for this fantastic logger!
This week I started with the omnikdatalogger in my existing home assistant application.
I can connect by port 8899 with my inverter and can scan every 60 seconds..
It is working very well, but only in the morning I see a strange behaviour, the appdeamon log displays errors and the current_power value is 0 Watt.
2022-04-17 07:57:34.354666 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-17 06:01:55.739746+02:00.
2022-04-17 07:57:35.003055 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-17 05:57:34.977773+00:00
But when I look directly in the Omnik inverter (with a webbrowser) it show that the value is not 0 Watt, but 142 Watt. this value is also presented in the sensor AC power L1.
Is your time zone set correctly?
If you have some error logs, feel free to share them
Is your time zone set correctly?
time_zone: Europe/Amsterdam
Somehow the logger thinks the sun is not up yet, so it seams
Did you set the appdeamon timezone as well?
Did you set the appdeamon timezone as well?
I followed the instructions in your readme.md
the appdaemon.yaml config is:
appdaemon:
latitude: 52.-------
longitude: 5.-------
elevation: 0
time_zone: Europe/Amsterdam
the apps.yaml config is:
omnik_datalogger:
module: omniklogger
class: HA_OmnikDataLogger
city: Amsterdam
interval: 60
persistant_cache_file: /config/appdaemon/persistant_cache.json
Interesting, in the log should also be a message from the evening at sun set indicating the time to wait. What does that say?
Interesting, in the log should also be a message from the evening at sun set indicating the time to wait. What does that say?
Sun set for my location is 20:41 hr, starting at 20:36 hr, every 18 sec there is the following message in the log, and I have seen last night, that is going on till the morning.
2022-04-17 20:50:08.457021 INFO omnik_datalogger: Inverter is not reachable over port 8899, this is normal when it is dark. [Errno 113] Host is unreachable
An interval of 60 sec is some short, I would suggest 300 sec, 5 minutes. The error messages occur because there is no connection possible with the inverter.
I have changed the interval to 300 sec,
I know that there is no connection possible with the inverter, the error messages keep coming, I thought that the tcp_client was stopping at sunset with a message "starting again at .... hr at sunrise.
No it is a calculated interval till sun rise. If the connection fails a retry follows sooner then the interval. To allign with pvoutut 5 minutes should be fine. Some people experience issues at the end of the day if the interval is too short. But that seems not the case here
This morning I can see the following:
7:26:57 field "power_ac1" first value (64Watt)
8:01:55 field "current_power" first value (156 Watt)
Looks like the sunshine postponing causes it.
logging:
2022-04-18 07:21:56.721615 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-18 05:59:34.792947+02:00.
2022-04-18 07:21:57.398154 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 05:21:57.372196+00:00
2022-04-18 07:26:56.437764 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-18 05:59:34.792947+02:00.
2022-04-18 07:26:57.116895 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 05:26:57.091480+00:00
2022-04-18 07:31:56.153149 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-18 05:59:34.792947+02:00.
2022-04-18 07:31:56.828989 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 05:31:56.809904+00:00
2022-04-18 07:36:55.869280 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-18 05:59:34.792947+02:00.
2022-04-18 07:36:56.553345 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 05:36:56.527566+00:00
2022-04-18 07:41:55.595146 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-18 05:59:34.792947+02:00.
2022-04-18 07:41:56.269799 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 05:41:56.244340+00:00
2022-04-18 07:46:55.301749 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-18 05:59:34.792947+02:00.
2022-04-18 07:46:55.989236 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 05:46:55.962560+00:00
2022-04-18 07:51:55.037802 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-18 05:59:34.792947+02:00.
2022-04-18 07:51:55.718811 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 05:51:55.680727+00:00
2022-04-18 07:56:54.764938 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-18 05:59:34.792947+02:00.
2022-04-18 07:56:55.425478 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 05:56:55.398561+00:00
2022-04-18 08:01:55.142969 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 06:01:55.118629+00:00
2022-04-18 08:06:54.859382 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 06:06:54.834633+00:00
2022-04-18 08:11:54.579233 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-18 06:11:54.552815+00:00
Something seems wrong in the calculation:
2022-04-18 07:21:56.721615 INFO omnik_datalogger: No sunshine postponing till down next dawn 2022-04-18 05:59:34.792947+02:00.
The logging is after dawn, but is still complaining.
The dawn time seems to be calculated correctly.
May be a weird question, but is the date on system a day behind. Next dawn should be on the 19th of april.
May be a weird question, but is the date on system a day behind. Next dawn should be on the 19th of april.
The systemdate is Ma 18 apr.
Ha, I see, it changes after sunset and from then there are no more updates. During sun set the power is set to 0 W. So did the logging start correct this morning?
I have changed the way the timezone info is collected. Can not really reproduce your case. I'll create a new beta version, let me now if that helped.
I have changed the way the timezone info is collected. Can not really reproduce your case. I'll create a new beta version, let me now if that helped.
Thank you for the fast respons!
I have just installed the new beta version, I will inform you tomorrow how it's working at dawn.
If this is not working either we need to dig in some deeper.
Well unfortunately it did not work as expected , now the sun is shining it works again.
I found the following errors in the appdeamon log:
ERRORS AT NIGHT:
2022-04-18 23:19:43.142436 INFO omnik_datalogger: Inverter is not reachable over port 8899, this is normal when it is dark. [Errno 113] Host is unreachable
2022-04-18 23:20:45.255420 INFO omnik_datalogger: Inverter is not reachable over port 8899, this is normal when it is dark. [Errno 113] Host is unreachable
Exception in thread Thread-208:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 1286, in run
self.function(*self.args, **self.kwargs)
File "/config/appdaemon/apps/omnikdatalogger/omnik/init.py", line 66, in _run
self.last_update_time = self.datalogger.process(*self.args, **self.kwargs)
File "/config/appdaemon/apps/omnikdatalogger/omnik/datalogger.py", line 1353, in process
return self._process_timed_event()
File "/config/appdaemon/apps/omnikdatalogger/omnik/datalogger.py", line 1202, in _process_timed_event
next_report_at = self._sunshine_check()
File "/config/appdaemon/apps/omnikdatalogger/omnik/datalogger.py", line 1105, in _sunshine_check
f"No sunshine postponing inverter checks till down next dawn {self.dl.next_dawn(time_now)}.",
TypeError: 'datetime.datetime' object is not callable
ERRORs AFTER RESTART:
2022-04-19 06:33:47.261685 INFO omnik_datalogger: MQTT connected
Exception in thread Thread-4:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 1286, in run
self.function(*self.args, **self.kwargs)
File "/config/appdaemon/apps/omnikdatalogger/omnik/init.py", line 66, in _run
self.last_update_time = self.datalogger.process(*self.args, **self.kwargs)
File "/config/appdaemon/apps/omnikdatalogger/omnik/datalogger.py", line 1353, in process
return self._process_timed_event()
File "/config/appdaemon/apps/omnikdatalogger/omnik/datalogger.py", line 1202, in _process_timed_event
next_report_at = self._sunshine_check()
File "/config/appdaemon/apps/omnikdatalogger/omnik/datalogger.py", line 1105, in _sunshine_check
f"No sunshine postponing inverter checks till down next dawn {self.dl.next_dawn(time_now)}.",
TypeError: 'datetime.datetime' object is not callable
I'll fix that one today
There is a new beta release that should solve the exception. Besides this error, logging starts correctly now?
Thank you for the new beta release.
I just installed this release and i'll see how it works tomorrow morning.
Have not looked at the logging, because after the errors I removed the first beta.
Here are my results with the beta 2 release.
Looks like the strange behaviour is caused by a mix-up with UTC time and local time,
or something with summer-/wintertime.
WHEN THE SUN IS DOWN, EVERY MINUTE THIS MESSAGE (TILL 23:31:07)
2022-04-23 23:30:08.520597 INFO omnik_datalogger: Inverter is not reachable over port 8899, this is normal when it is dark. [Errno 113] Host is unreachable
2022-04-23 23:31:07.539314 INFO omnik_datalogger: No sunshine postponing inverter checks till down next dawn 2022-04-24 05:45:41.550074+02:00.
2022-04-23 23:31:10.637608 INFO omnik_datalogger: Inverter is not reachable over port 8899, this is normal when it is dark. [Errno 113] Host is unreachable
IN THE MORNING (AT THE "POSTPONING CHECKS TILL TIME"), EVERY MINUTE THIS MESSAGE:
2022-04-24 05:55:40.672806 INFO omnik_datalogger: No sunshine postponing inverter checks till down next dawn 2022-04-24 05:45:41.550074+02:00.
2022-04-24 05:55:43.754420 INFO omnik_datalogger: Inverter is not reachable over port 8899, this is normal when it is dark. [Errno 113] Host is unreachable
.
2022-04-24 06:25:31.472190 INFO omnik_datalogger: No sunshine postponing inverter checks till down next dawn 2022-04-24 05:45:41.550074+02:00.
2022-04-24 06:26:44.546099 INFO omnik_datalogger: Inverter is not reachable over port 8899, this is normal when it is dark. timed out
.
INVERTER RESPONDS AND NOW EVERY 300 SEC. THE FOLLOWING MESSAGE: (only "CURRENT_POWER" IS 0 W)
2022-04-24 06:27:43.562276 INFO omnik_datalogger: No sunshine postponing inverter checks till down next dawn 2022-04-24 05:45:41.550074+02:00.
2022-04-24 06:27:44.264948 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-24 04:27:44.239651+00:00
2022-04-24 06:32:43.316347 INFO omnik_datalogger: No sunshine postponing inverter checks till down next dawn 2022-04-24 05:45:41.550074+02:00.
2022-04-24 06:32:43.983095 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-24 04:32:43.958261+00:00
2022-04-24 06:37:43.019307 INFO omnik_datalogger: No sunshine postponing inverter checks till down next dawn 2022-04-24 05:45:41.550074+02:00.
2022-04-24 06:37:43.700130 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-24 04:37:43.674732+00:00
.
EVERY THING IS NORMAL (ONLY UPDATE MESSAGES AND "CURRENT_POWER" HAS A VALUE)
2022-04-24 07:47:39.753022 INFO omnik_datalogger: Update for plant rb12 UTC 2022-04-24 05:47:39.727710+00:00
Thank you, I will have a further look at this. The time zone should not interfere when comparing times as long as a timezone is set. I suppose the logtime at the left is (dutch) local time?
Yes that's correct, my timezone in appdaemon.yaml is time_zone: Europe/Amsterdam
and in config/appdaemon//apps/apps.yaml:
timezone: Europe/Amsterdam
city: Amsterdam
I've added a new beta with some additional debug logging (need to set the log level to debug to see it). Hope you can check again.
Thank you for the new beta, I have just installed it.
In the appdaemon configration I have changed the loglevel to debug and in appdaemon.yaml I have added to log to files.
Problem is that it's generating so many data, Is it possible to only log debug messages from the omnik datalogger?
Thank you for the new beta, I have just installed it. In the appdaemon configration I have changed the loglevel to debug and in appdaemon.yaml I have added to log to files. Problem is that it's generating so many data, Is it possible to only log debug messages from the omnik datalogger?
Hmm, that is not possible. Besides debug messages you will also get info and warning messaged. You could download the logfiles and analyze them in an editor. Further you could test the new code first without setting the log_level to debug.
That's exactly what I am doing.
I'll keep you informed.
Unfortunately I was too late, to see the logging this morning. The APPDAEMON debug generates so much data, that in 3 minutes all 3 logfiles where full, and the oldest messages where overwritten.
What I noticed was the following:
25 april 23:35 latest 1 minute polling to the inverter.
26 april 5:51 start with 1 min polling to inverter.
26 april 6:24 first sign that inverter is alive, with life data and a message postponing till 5:41:09, polling now every 300 sec.
26 april 7:44 life data and no postponing message, current_power has correct value.
26 april 20:58 every minute "inverter not reachable over port 8899, this is normal when it is dark"
for 26 april the sun values at my place are:
dawn: 5:50
sun rise: 6:16
sun down: 20:57
dark: 21:32
Tomorrow morning I will have a better look at the debug log.
Yesterday evening I also started with omnikdatalogger in DOCKER on my Synology NAS.
Must say it works very well, COMPLIMENTS TO YOU FOR THIS FANTASTIC WORK THAT YOU DO!
This morning I have looked at the DEBUG logfiles, but that's not doable.
1 minute generates 3200 records of data, (it are all HA supervisor logs to)
I have made a quick and dirty modification and now I only get the datalogger.py logging. (changed debug to info)
Hopefully tomorrow morning better result.
This morning I have more success. Results in the appdaemon.log:
2022-04-28 06:52:29.943942 INFO omnik_datalogger: *** local time *** : 2022-04-28 04:52:29.938027+02:00.
2022-04-28 06:52:29.950187 INFO omnik_datalogger: *** time now *** : {
'dawn': datetime.datetime(2022, 4, 28, 5, 36, 40, 903290, tzinfo=<DstTzInfo 'Europe/Amsterdam' CEST+2:00:00 DST>),
'sunrise': datetime.datetime(2022, 4, 28, 6, 15, 58, 389279, tzinfo=<DstTzInfo 'Europe/Amsterdam' CEST+2:00:00 DST>),
'noon': datetime.datetime(2022, 4, 28, 13, 37, 58, tzinfo=<DstTzInfo 'Europe/Amsterdam' CEST+2:00:00 DST>),
'sunset': datetime.datetime(2022, 4, 28, 21, 0, 56, 838965, tzinfo=<DstTzInfo 'Europe/Amsterdam' CEST+2:00:00 DST>),
'dusk': datetime.datetime(2022, 4, 28, 21, 40, 30, 147715, tzinfo=<DstTzInfo 'Europe/Amsterdam' CEST+2:00:00 DST>)}.
In the first line you can see, the wrong local time is reported,
after a little study I think I found the error on line 39 in daylight.py
return datetime.utcnow().astimezone(pytz.timezone(self._city.timezone))
After I have replaced utcnow() in now() it worked and the postponing messages stopped and current_power has a non zero value
Seems like both now() and utcnow() return a timzeone-less datetime (.tzinfo = None), and it appears astimezone() performs a conversion relative to localtime if the source does not contain a timezone: https://docs.python.org/3/library/datetime.html#datetime.datetime.astimezone. Hence utcnow() is assumed to be in local time already here, turning astimezone into behaving effectively like .replace(tz=...).
The proper solution is probably to call datetime.now(tz=pytz.timezone(self._city.timezone)) which should internally perform a conversion from local time to the desired timezone, followed by attaching it in tzinfo.
>>> datetime.now(tz=timezone.utc) # Converts from local time to UTC timezone
datetime.datetime(2022, 4, 28, 8, 16, 35, 825397, tzinfo=datetime.timezone.utc)
>>> datetime.now(tz=None) # Returns a timezone-less time, even if it should know that it's in localtime...
datetime.datetime(2022, 4, 28, 10, 16, 38, 737261)
>>> datetime.now().astimezone() # When called with None, "converts" to the local timezone and attaches it
datetime.datetime(2022, 4, 28, 10, 16, 52, 545039, tzinfo=datetime.timezone(datetime.timedelta(seconds=7200), 'CEST'))
>>> datetime.now().astimezone(tz=timezone.utc) # Converts from assumed local time to UTC
datetime.datetime(2022, 4, 28, 8, 16, 54, 4432, tzinfo=datetime.timezone.utc)EDIT: Showing again that utcnow() doesn't include tzinfo, leading to astimezone converting from assumed local time to UTC and subtracing another two hours:
>>> datetime.utcnow()
datetime.datetime(2022, 4, 28, 8, 39, 43, 997657)
>>> datetime.utcnow().astimezone(tz=timezone.utc)
datetime.datetime(2022, 4, 28, 6, 39, 44, 604952, tzinfo=datetime.timezone.utc)Thnx, I am AFK a few days, will have a look ASAP
Thnx, I am AFK a few days, will have a look ASAP
Enjoy your AFK time.
@jbouwh I can line up a PR with these changes for you, but I won't be able to test them :)
If you want, I can test it.
@fritzzjan Thanks, please try out the associated PR!
@fritzzjan Thanks, please try out the associated PR!
I have installed both of your PR's.
Everything is working fine. (note my city_name is the default)
For me all these modifications solved all my issues, so it is ok to close it.
Solved:
- unnecessary postpone messages in the morning
- to long polling the inverter in the evening
- current_power that is 0 Watt in the morning
Thank you all for your time and help!
note: is the docker version also updated?
I'll have a look at the pr's tomorrow, after merging I need to publish a new version. This will trigger a new Docker version and Pypi version as well