Scheduler error after upgrade
ksurl opened this issue · comments
Expected behaviour:
Normal daemon startup without errors
Actual behaviour:
Steps to reproduce:
- Step 1: Upgrade 3.8.5 to 3.9.2+ (not sure what version scheduler change was made)
- Step 2: start daemon
Config:
any config with schedules set
Log:
(click to expand)
2023-08-19 09:43:33 ERROR apscheduler.jobstores.default Unable to restore job "839de55fdc6695747827a775e639a918ecf6f9d9" -- removing it
Traceback (most recent call last):
File "/lsiopy/bin/flexget", line 8, in <module>
sys.exit(main())
│ │ └ <function main at 0x7f2984acc860>
│ └ <built-in function exit>
└ <module 'sys' (built-in)>
File "/lsiopy/lib/python3.11/site-packages/flexget/__init__.py", line 55, in main
manager.start()
│ └ <function Manager.start at 0x7f298103ede0>
└ <flexget.manager.Manager object at 0x7f2981040650>
File "/lsiopy/lib/python3.11/site-packages/flexget/manager.py", line 366, in start
self.handle_cli()
│ └ <function Manager.handle_cli at 0x7f298103ee80>
└ <flexget.manager.Manager object at 0x7f2981040650>
File "/lsiopy/lib/python3.11/site-packages/flexget/manager.py", line 392, in handle_cli
self.daemon_command(command_options)
│ │ └ <unprintable ScopedNamespace object>
│ └ <function Manager.daemon_command at 0x7f298103efc0>
└ <flexget.manager.Manager object at 0x7f2981040650>
File "/lsiopy/lib/python3.11/site-packages/flexget/manager.py", line 489, in daemon_command
run_daemon()
└ <function Manager.daemon_command.<locals>.run_daemon at 0x7f297c243f60>
File "/lsiopy/lib/python3.11/site-packages/flexget/manager.py", line 470, in run_daemon
fire_event('manager.daemon.started', self)
│ └ <flexget.manager.Manager object at 0x7f2981040650>
└ <function fire_event at 0x7f2981fa7a60>
File "/lsiopy/lib/python3.11/site-packages/flexget/event.py", line 108, in fire_event
result = event(*args, **kwargs)
│ │ └ {}
│ └ (<flexget.manager.Manager object at 0x7f2981040650>,)
└ <Event(name=manager.daemon.started,func=setup_scheduler,priority=128)>
File "/lsiopy/lib/python3.11/site-packages/flexget/event.py", line 20, in __call__
return self.func(*args, **kwargs)
│ │ │ └ {}
│ │ └ (<flexget.manager.Manager object at 0x7f2981040650>,)
│ └ <function setup_scheduler at 0x7f297d94e0c0>
└ <Event(name=manager.daemon.started,func=setup_scheduler,priority=128)>
File "/lsiopy/lib/python3.11/site-packages/flexget/components/scheduler/scheduler.py", line 128, in setup_scheduler
setup_jobs(manager)
│ └ <flexget.manager.Manager object at 0x7f2981040650>
└ <function setup_jobs at 0x7f297d94e160>
File "/lsiopy/lib/python3.11/site-packages/flexget/components/scheduler/scheduler.py", line 155, in setup_jobs
existing_job_ids = [job.id for job in scheduler.get_jobs()]
│ └ <function BaseScheduler.get_jobs at 0x7f297d92aca0>
└ <apscheduler.schedulers.background.BackgroundScheduler object at 0x7f297c2662d0>
File "/lsiopy/lib/python3.11/site-packages/apscheduler/schedulers/base.py", line 584, in get_jobs
jobs.extend(store.get_all_jobs())
│ │ │ └ <function SQLAlchemyJobStore.get_all_jobs at 0x7f297db0f240>
│ │ └ <SQLAlchemyJobStore (url=sqlite:////config/db-config-jobs.sqlite)>
│ └ <method 'extend' of 'list' objects>
└ []
File "/lsiopy/lib/python3.11/site-packages/apscheduler/jobstores/sqlalchemy.py", line 89, in get_all_jobs
jobs = self._get_jobs()
│ └ <function SQLAlchemyJobStore._get_jobs at 0x7f297db0f6a0>
└ <SQLAlchemyJobStore (url=sqlite:////config/db-config-jobs.sqlite)>
> File "/lsiopy/lib/python3.11/site-packages/apscheduler/jobstores/sqlalchemy.py", line 148, in _get_jobs
jobs.append(self._reconstitute_job(row.job_state))
│ │ │ │ └ ('839de55fdc6695747827a775e639a918ecf6f9d9', b'\x80\x05\x95q\x04\x00\x00\x00\x00\x00\x00}\x94(\x8c\x07version\x94K\x01\x8c\x0...
│ │ │ └ <function SQLAlchemyJobStore._reconstitute_job at 0x7f297db0f600>
│ │ └ <SQLAlchemyJobStore (url=sqlite:////config/db-config-jobs.sqlite)>
│ └ <method 'append' of 'list' objects>
└ []
File "/lsiopy/lib/python3.11/site-packages/apscheduler/jobstores/sqlalchemy.py", line 131, in _reconstitute_job
job_state = pickle.loads(job_state)
│ │ └ b'\x80\x05\x95q\x04\x00\x00\x00\x00\x00\x00}\x94(\x8c\x07version\x94K\x01\x8c\x02id\x94\x8c(839de55fdc6695747827a775e639a918e...
│ └ <built-in function loads>
└ <module 'pickle' from '/usr/lib/python3.11/pickle.py'>
ModuleNotFoundError: No module named 'pytz_deprecation_shim'
2023-08-19 09:43:33 ERROR apscheduler.jobstores.default Unable to restore job "1cb82188ae52b8cd0fd56159b7b25d14c23848d4" -- removing it
Traceback (most recent call last):
File "/lsiopy/bin/flexget", line 8, in <module>
sys.exit(main())
│ │ └ <function main at 0x7f2984acc860>
│ └ <built-in function exit>
└ <module 'sys' (built-in)>
File "/lsiopy/lib/python3.11/site-packages/flexget/__init__.py", line 55, in main
manager.start()
│ └ <function Manager.start at 0x7f298103ede0>
└ <flexget.manager.Manager object at 0x7f2981040650>
File "/lsiopy/lib/python3.11/site-packages/flexget/manager.py", line 366, in start
self.handle_cli()
│ └ <function Manager.handle_cli at 0x7f298103ee80>
└ <flexget.manager.Manager object at 0x7f2981040650>
File "/lsiopy/lib/python3.11/site-packages/flexget/manager.py", line 392, in handle_cli
self.daemon_command(command_options)
│ │ └ <unprintable ScopedNamespace object>
│ └ <function Manager.daemon_command at 0x7f298103efc0>
└ <flexget.manager.Manager object at 0x7f2981040650>
File "/lsiopy/lib/python3.11/site-packages/flexget/manager.py", line 489, in daemon_command
run_daemon()
└ <function Manager.daemon_command.<locals>.run_daemon at 0x7f297c243f60>
File "/lsiopy/lib/python3.11/site-packages/flexget/manager.py", line 470, in run_daemon
fire_event('manager.daemon.started', self)
│ └ <flexget.manager.Manager object at 0x7f2981040650>
└ <function fire_event at 0x7f2981fa7a60>
File "/lsiopy/lib/python3.11/site-packages/flexget/event.py", line 108, in fire_event
... (43 lines left)
Additional information:
- FlexGet version: 3.9.2
- Python version: 3.11.4
- Installation method: docker
- Using daemon (yes/no): yes
- OS and version: debian 12
daemon starts normally after the error and runs jobs as expected. just on startup this error appears.
Did you do something to get it working?
I'm trying to figure out what we can do and where, since the error is sorta deep in apscheduler. Maybe we need to make a PR there. I also need to see if it's easy to reproduce, or something specific about your environment
Oh, I see, the scheduler is already working fine. apscheduler is already catching the exception and continuing. That's sorta the same solution I was going to implement, and I'm fine with just leaving this. It's a bit ugly that there will be a traceback in the log right after upgrade, but I'm not sure its worth any extra effort.
no. I ran a flexget check and then flexget execute --tasks <some_task> and it worked fine. so it recovered on its own. I haven't restarted it since.
I tested restart by going 3.9.2->3.9.4 since those changes shouldn't affect it. no errors this time on startup
I'm going to call this good enough. Hopefully people who are worried will see this ticket and realize everything is normal if they only see this once right after upgrading to version 3.9.1+
EDIT: But thanks for reporting!