Syncing watchlist between Trakt and Plex always adds back removed items
rfgamaral opened this issue · comments
Confirmation
- I have read the README.md on the project homepage
- I have checked if identical issue already exists
- I have tried downgrading to find version that can be used as a workaround
The problem
Syncing my watchlist between Trakt and Plex does not preserve remove state, that is:
- If I remove an item from my Trakt watchlist, I expected the item to be removed from Plex on the next sync
- If I remove an item from my Plex watchlist, I expected the item to be removed from Trakt on the next sync
Error trace / logs
I removed 3 movies from my Trakt Watchlist and ran plextraktsync sync --sync=movies:
2022-12-01 12:19:58,946 INFO[PlexTraktSync]:PlexTraktSync [0.24.5]
2022-12-01 12:19:59,275 INFO[PlexTraktSync]:Sync Movie sections: [<PlexLibrarySection:movie:Concerts>, <PlexLibrarySection:movie:Movies>]
2022-12-01 12:20:01,890 INFO[PlexTraktSync]:Concerts processed in 2.6 seconds
2022-12-01 12:20:09,803 INFO[PlexTraktSync]:Movies processed in 7.9 seconds
2022-12-01 12:20:17,613 INFO[PlexTraktSync]:Adding 'Amsterdam' to Trakt watchlist
2022-12-01 12:20:31,097 INFO[PlexTraktSync]:Adding 'Black Panther: Wakanda Forever' to Trakt watchlist
2022-12-01 12:20:35,505 INFO[PlexTraktSync]:Adding 'Deepwater Horizon' to Trakt watchlist
2022-12-01 12:20:48,794 INFO[PlexTraktSync]:Skipping 'Untitled Ghostbusters: Afterlife Sequel' from Trakt watchlist because not found in Plex Discover
2022-12-01 12:20:52,170 INFO[PlexTraktSync]:Skipping 'The King' from Trakt watchlist because not found in Plex Discover
2022-12-01 12:20:52,185 INFO[PlexTraktSync]:Updated watchlist and/or liked list in 40.7 seconds
2022-12-01 12:20:52,322 INFO[PlexTraktSync]:Completed full sync in 53.1 seconds
I removed 3 movies from my Plext Watchlist and ran plextraktsync sync --sync=movies:
2022-12-01 12:21:57,098 INFO[PlexTraktSync]:PlexTraktSync [0.24.5]
2022-12-01 12:21:57,184 INFO[PlexTraktSync]:Sync Movie sections: [<PlexLibrarySection:movie:Concerts>, <PlexLibrarySection:movie:Movies>]
2022-12-01 12:21:58,890 INFO[PlexTraktSync]:Concerts processed in 1.7 seconds
2022-12-01 12:22:06,670 INFO[PlexTraktSync]:Movies processed in 7.8 seconds
2022-12-01 12:22:59,716 INFO[PlexTraktSync]:Skipping 'Untitled Ghostbusters: Afterlife Sequel' from Trakt watchlist because not found in Plex Discover
2022-12-01 12:23:04,296 INFO[PlexTraktSync]:Skipping 'The King' from Trakt watchlist because not found in Plex Discover
2022-12-01 12:23:04,919 INFO[PlexTraktSync]:Adding 'Amsterdam' to Plex watchlist
2022-12-01 12:23:06,450 INFO[PlexTraktSync]:Adding 'Black Panther: Wakanda Forever' to Plex watchlist
2022-12-01 12:23:07,845 INFO[PlexTraktSync]:Adding 'Deepwater Horizon' to Plex watchlist
2022-12-01 12:23:08,483 INFO[PlexTraktSync]:Updated watchlist and/or liked list in 1 min 0.1 seconds
2022-12-01 12:23:08,486 INFO[PlexTraktSync]:Completed full sync in 1 min 11.3 seconds
Expected behavior
- Removed items from Plex Watchlist should be removed from Trakt Watchlist on the next sync
- Removed items from Trakt Watchlist should be removed from Plex Watchlist on the next sync
Steps to reproduce the behavior
Any sync command results in the issue.
Inspect of problematic items
No response
Workarounds
No response
Install method
docker-compose
Config file contents
# Config File: /app/config/config.yml
cache:
path: /app/config/.cache/trakt
config:
dotenv_override: true
excluded-libraries:
- Samples
logging:
append: true
console_time: true
debug: false
filename: plextraktsync.log
filter: null
plex:
timeout: 30
sync:
plex_to_trakt:
collection: false
ratings: false
watched_status: true
watchlist: true
trakt_to_plex:
liked_lists: false
ratings: true
watched_status: true
watchlist: true
watchlist_as_playlist: false
watch:
add_collection: true
remove_collection: true
scrobble_threshold: 80
username_filter: true
xbmc-providers:
movies: imdb
shows: tvdbVersion
0.24.5
Python Version
3.11.0
Operating System and Version
Synology (Linux)
When there is a Movie in one watchlist but not in the other watchlist, how could the script kow if it should remove it from the first watchlist or add it in the second wtchlist ?
I can't work out the details right now because I don't know how exactly each API works, but it should be possible by keeping some local state for the previous sync for each watchlist, and do some condition checks.
Right now, the way it's implemented, it acts more like a "mirror" than a "sync".
FYI: After #1238 (0.24.6) watchlist sync is disabled if --sync=movies is specified.
FYI: After #1238 (0.24.6) watchlist sync is disabled if
--sync=moviesis specified.
Not relevant here because watchlist_as_playlist=false
And I only used --sync=movies in my examples because I was testing and didn't want to go through syncing my TV Shows. For actual syncing, I use the Ofelia approach described in the README.
Workaround is to sync watchlist only one-way (plex_to_trakt or trakt_to_plex).
Not really a workaround for me, I have no use for having just one of them enabled. Either 2-way sync works taking into account removed items, or I might as well just turn off both plex_to_trakt and trakt_to_plex 😔
I noticed the trakt response includes list: updated_at, could that be useful?
2022-12-27 09:39:21,429 DEBUG[PlexTraktSync]:Updated Trakt watchlist: {'deleted': {'movies': 3, 'shows': 1, 'seasons': 0, 'episodes': 0}, 'list': {'updated_at': '2022-12-27T07:39:20.000Z', 'item_count': 29}}
also, trakt side/sync/watchlist and /users/id/watchlist have listed_at timestamp. maybe useful?
"listed_at": "2014-09-01T09:10:11.000Z",
so, the trakt response has:
{'id': 773647533,
'listed_at': '2022-12-07T10:34:20.000Z',
'movie': {'ids': {'imdb': 'tt0070948',
'slug': 'zardoz-1974',
'tmdb': 4923,
'trakt': 2858},
'title': 'Zardoz',
'year': 1974},
'notes': None,
'rank': 8,
'type': 'movie'}but all data that's not inside movie gets lost, because only movie data is collected:
so would need new data class to pytrakt that has listed_at, notes, movie, rank, type properties?
actually, watchlist_shows would pass the extra data to TVShow:
but I guess a new class with extra fields would be a cleaner solution.
I noticed the trakt response includes
list: updated_at, could that be useful?
Explain how useful it is please.
I noticed the trakt response includes
list: updated_at, could that be useful?Explain how useful it is please.
please note the question form of the sentence.
the idea was to compare plex side watchlistedAt vs trakt side listed_at. for this to be useful, need the third timestamp, when was the last sync ran, such timestamp is not available.
do you see a solution how to do with only two timestamps? something like compare newest entry in plex = last update in plex side, and same for trakt?
@glensc I have a Trakt Pro subscription, and I could make a feature request on the Trakt VIP forums to improve the API to allow this use case, I just need to know the exact requirements. Let me know how I can help.
the idea was to compare plex side
watchlistedAtvs trakt sidelisted_at. for this to be useful, need the third timestamp, when was the last sync ran, such timestamp is not available.
Explain how comparing listedAt dates could help here.
What would you do if plex_listed_at > trakt_listed_at for example ?
Explain how comparing
listedAtdates could help here. What would you do ifplex_listed_at > trakt_listed_atfor example ?
You omitted the part where I said sync timestamp is needed.
in my mind, the algo would be:
- if item is missing in trakt watchlist, compare
last_sync_dateandplex_listed_at. ifplex_listed_at > last_sync_datethen add it to trakt otherwise drop from plex
the same for other direction.
yes. this could work
def plex_watchlist_updated_at(self):
from datetime import datetime, timezone
from plexapi import utils
updated_at = datetime(1970, 1, 1)
for pm in self.plex_wl.values():
listed_at = utils.toDatetime(pm._data.get("watchlistedAt"))
updated_at = max(updated_at, listed_at)
return updated_at.astimezone(timezone.utc)
def trakt_watchlist_updated_at(self, watchlist):
from datetime import datetime, timezone
updated_at = datetime(1970, 1, 1).astimezone(timezone.utc)
for tm in watchlist.values():
listed_at = datetime.strptime(tm.listed_at, "%Y-%m-%dT%H:%M:%S.%f%z")
updated_at = max(updated_at, listed_at)
return updated_at
def watchlist_sync_item(self, m: Media, dry_run=False):
if not self.sync_wl:
return
plex_watchlist_updated_at = self.plex_watchlist_updated_at()
print(plex_watchlist_updated_at)
trakt_movies_watchlist_updated_at = self.trakt_watchlist_updated_at(self.trakt_wl_movies)
print(trakt_movies_watchlist_updated_at)
if plex_watchlist_updated_at > trakt_movies_watchlist_updated_at:
print(f"Plex watchlist is newer")
else:
print(f"Trakt watchlist is newer")extra thing to consider:
- trakt removes automatically when item is played, so the "last timestamp from items" might be wrong.
https://trakt.docs.apiary.io/#reference/sync/get-watchlist/get-watchlist:
Auto Removal
When an item is watched, it will be automatically removed from the watchlist. For shows and seasons, watching 1 episode will remove the entire show or season.
could maybe compare "played_on_trakt" state and make an extra decision.
You omitted the part where I said sync timestamp is needed.
How do you get it ?
if plex_watchlist_updated_at > trakt_movies_watchlist_updated_at: print(f"Plex watchlist is newer") else: print(f"Trakt watchlist is newer")
I think those print() assertions are wrong because it forgets to read the "remove" actions. It only reads "add" actions on watchlists.
You omitted the part where I said sync timestamp is needed.
How do you get it ?
read last sentence:
if plex_watchlist_updated_at > trakt_movies_watchlist_updated_at: print(f"Plex watchlist is newer") else: print(f"Trakt watchlist is newer")I think those
print()assertions are wrong because it forgets to read the "remove" actions. It only reads "add" actions on watchlists.
also must consider trakt removes automatically from Watchlist:
Removed automatically or manually from watchlist doesn't change the problem.
Perhaps one thing to do is not to add to watchlist items that are played.
found that trakt offers a response that includes when watchlist was last updated:
This feature would be a great addition as I would like to use the watchlists in this way:
- Add a movie to Plex Watchlist (I expect it to be added to Trakt)
- I add a movie to Trakt Watchlist (I expect it to be added to Plex)
- I watch a movie on Plex and it detects the movie is watched and auto removes the movie from the Plex Watchlist (I expect that movie to also be removed from Trakt Watchlist)
- I watch a movie somewhere else and I manually mark the movie as watched in Trakt and I manually remove it from Trakt Watchlist (I expect that movie removed from Plex Watchlist also)
This feature would be a great addition as I would like to use the watchlists in this way:
* Add a movie to Plex Watchlist (I expect it to be added to Trakt) * I add a movie to Trakt Watchlist (I expect it to be added to Plex) * I watch a movie on Plex and it detects the movie is watched and auto removes the movie from the Plex Watchlist (I expect that movie to also be removed from Trakt Watchlist) * I watch a movie somewhere else and I manually mark the movie as watched in Trakt and I manually remove it from Trakt Watchlist (I expect that movie removed from Plex Watchlist also)
exactly, that is how sync should work, at the moment it's more 'tranfer' than 'sync' :-)
@KoenBoone opinions, etc trash should go to discussions. if you want to be helpful, provide solutions. your comment has been hidden.
exactly, that is how sync should work, at the moment it's more 'tranfer' than 'sync' :-)
you just commented some post, I read no solution from there.
I stand corrected, i thought it was another issue (#1080), should have followed the link instead of replying by email :-)
This feature would be a great addition as I would like to use the watchlists in this way:
- Add a movie to Plex Watchlist (I expect it to be added to Trakt)
- I add a movie to Trakt Watchlist (I expect it to be added to Plex)
- I watch a movie on Plex and it detects the movie is watched and auto removes the movie from the Plex Watchlist (I expect that movie to also be removed from Trakt Watchlist)
- I watch a movie somewhere else and I manually mark the movie as watched in Trakt and I manually remove it from Trakt Watchlist (I expect that movie removed from Plex Watchlist also)
I agree with this!
This feature would be a great addition as I would like to use the watchlists in this way:
- Add a movie to Plex Watchlist (I expect it to be added to Trakt)
- I add a movie to Trakt Watchlist (I expect it to be added to Plex)
- I watch a movie on Plex and it detects the movie is watched and auto removes the movie from the Plex Watchlist (I expect that movie to also be removed from Trakt Watchlist)
- I watch a movie somewhere else and I manually mark the movie as watched in Trakt and I manually remove it from Trakt Watchlist (I expect that movie removed from Plex Watchlist also)
Alternatively you could use a script specifically to remove watched items from plex/trakt watchlists.
Edit: After thinking about it this would be a great separate feature to add as an optional config setting. I opened a separate issue for it here #1494
Similar problem to synchronizing two filesystems, that do not know about each other. Take a look at how rclone solved with for the bisync feature: https://rclone.org/bisync/.
Initial run:
Get list from trakt and save the list with timestamp. Get list from plex and save the list with timestamp.
Decide, which list to keep. Remove missing from plex? Remove missing from trakt? Keep both, as in, add missing items to the other list?
Perform synchronisation action and save result (the two lists, with two timestamps)
Subsequent runs can then decide based on timestamp, which items where removed from each list or added to each list and replicate this action to the other list.
The only problem with this approach, is that the index needs to be protected. If the local copy of the list with the timestamps is tampered with, lost or in some other way corrupted, chaos can ensue. Worst case is, both lists get pruned of all items. Another failure case is that updates are lost, e.g. all removed items from last run are added back.
There is a lot of fun computer sciency topics to think about, e.g. https://en.wikipedia.org/wiki/Write%E2%80%93write_conflict. Don't think implementing this well would be trivial. Or maybe I am overthinking it.
If the local copy of the list with the timestamps is tampered with, lost or in some other way corrupted, chaos can ensue. Worst case is, both lists get pruned of all items. Another failure case is that updates are lost, e.g. all removed items from last run are added back.
I was thinking about ways you could implement this too and this was my conclusion as well. There is a lot of risk associated with trying to determine which items to delete. Trakt API goes down pretty frequently which can cause issues. Local database can be corrupted, deleted, etc. Sometimes Trakt API can also omit results when things are not working correctly (multiple pagination pages missing). I would assume the same risks for Plex API as well. There would need to be a lot of fail proof checks in order to accomplish this safely.
Some things I was considering in one of my projects (IMDB-Trakt-Syncer) that would apply here as well, schedule items to be removed. Periodically check all lists to verify the item should be deleted (to ensure no API errors), after a certain amount of checks or time passed then approve the scheduled items for deletion. Additionally, you could add a check so that if more than 50% of a list is to be deleted then stop the deletion (in order to prevent accidental deletion due to unforeseen issues). As you can see it would get very complicated and there is still risk of chaos ensuing as you mentioned.