Bug in v5.0 (but cannot upgrade to v7.0 because of parsing speed problem)

Question

Bug in v5.0 (but cannot upgrade to v7.0 because of parsing speed problem)

vinraspa opened this issue 2 years ago · comments

Hi.
I am currently using v5.0 because of parsing speed issues in v6.0 et v7.0 (see: #244).

The bug I show below was fixed in v6 and v7. Being stuck to v5, I need help to find a workaround in my case.

Consider this ical file original.ics:

BEGIN:VCALENDAR
X-WR-CALNAME:Perso
X-WR-TIMEZONE:Europe/Paris
PRODID:-//Google Inc//Google Calendar 70.9054//EN
VERSION:2.0
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VEVENT
DTSTART:20190522T060000Z
DTEND:20190522T133000Z
DTSTAMP:20220320T185509Z
UID:bctfpnhdaem30k2v2uflatqpag@google.com
CREATED:20190520T091318Z
DESCRIPTION:
LAST-MODIFIED:20190527T181058Z
LOCATION:IUT\, BC\, Salle Pierre Andanson
SEQUENCE:2
STATUS:CONFIRMED
SUMMARY:Garde
TRANSP:OPAQUE
BEGIN:VALARM
ACTION:NONE
TRIGGER;VALUE=DURATION:-PT1H
DESCRIPTION:ignore me
END:VALARM
END:VEVENT
END:VCALENDAR

I create a calendar from reading this file: no problem.
I write the calendar to a new file faulty.ics.
I create a calendar from faulty.ics file.

from ics import Calendar
with open('original.ics', 'r') as f:
    c = Calendar(imports=f.read())
with open('faulty.ics', 'w') as f:
    f.writelines(c)
with open('faulty.ics', 'r') as f:
    c = Calendar(imports=f.read())

I get this error :
ics.parse.ParseError: No ':' in line 'None'

Actually, here is the content of faulty.ics file:

BEGIN:VCALENDAR
X-WR-CALNAME:Perso
X-WR-TIMEZONE:Europe/Paris
PRODID:-//Google Inc//Google Calendar 70.9054//EN
VERSION:2.0
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VEVENT
CREATED:20190520T091318Z
SEQUENCE:2
DTSTAMP:20220320T185509Z
LAST-MODIFIED:20190527T181058Z
DTSTART:20190522T060000Z
DTEND:20190522T133000Z
SUMMARY:Garde
LOCATION:IUT\, BC\, Salle Pierre Andanson
TRANSP:OPAQUE
UID:bctfpnhdaem30k2v2uflatqpag@google.com
None
STATUS:CONFIRMED
END:VEVENT
END:VCALENDAR

The problem is in the fourth line from the end.

I cannot figure out the exact origin of the bug nor the way to work it around.
Again, I cannot upgrade to v7.0.

Thank you in advance.

Niko Fink · Answer 1 · Mon Mar 21 2022 04:33:31 GMT+0800 (China Standard Time)

Should be fixed once we release v0.7.1 which contains the parsing speed fixes, see #318. Alternatively, you can try upgrading directly to v0.8 which contains a lot more goodies, but which is still in development (i.e. on the current main branch). If you really want to stay on 0.5, you probably need to dig deeper into this yourself as the codebase largely changed since then and I'm not at all familiar with what was done back then. Did you check the backtrace of the exception?

Vincent Raspal · Answer 2 · Wed Mar 23 2022 01:45:41 GMT+0800 (China Standard Time)

Thank you for the quick reply.
Yes I checked the backtrace. The only problem is that the ':' is missing in the line None. But where this None comes from? I do not know. As you said later versions are quite different from v0.5.
I patched my script simply by checking the content written to/read from an ics file and by deleting any None line.

I am looking forward to the release of v0.7.1.

I tried v0.8 but many things failed in my script (maybe because the arrow package is not used anymore?) and I did not dig any further, preferring getting back to v0.5.

Vincent Raspal · Answer 3 · Wed Mar 23 2022 01:46:22 GMT+0800 (China Standard Time)

Do you know when v0.7.1 will be released?

Niko Fink · Answer 4 · Wed Mar 23 2022 18:44:18 GMT+0800 (China Standard Time)

Probably pretty quickly once #318 is resolved. But I would really recommend upgrading to 0.8. It switches back to using Python's builtin datetime because arrow is, amongst other things, unable to represent floating Events that must be interpreted in the local timezone. Instead of installing 0.8 from the master branch, you could also install the branch that will be released as 0.7.1, i.e.

pip install git+https://github.com/ics-py/ics-py.git@version-0.7

Vincent Raspal · Answer 5 · Fri Apr 01 2022 00:19:31 GMT+0800 (China Standard Time)

The problem with the latest version (obtained with pip install git+https://github.com/ics-py/ics-py.git@main) is that I get an error when I try to initiate an instance of my own subclass of Calendar, exactly in the populate method of the Component class

    def populate(self, container: Container, context: Optional[ContextDict] = None):
        from ics import initialize_converters
        initialize_converters()
        from ics.converter.component import ComponentMeta
        ComponentMeta.BY_TYPE[type(self)].populate_instance(self, container, context)

the last line raises a KeyError as type(self) != Calendar

Should I open a new issue for that?
Thanks

Niko Fink · Answer 6 · Fri Apr 01 2022 15:28:16 GMT+0800 (China Standard Time)

That is an issue with the customizability of (de)serializiation which I hope to fix before the v0.8 release. Could you try the following (which is borrowed from what is done for Calendar here) until a patch for this lands?

ComponentMeta.BY_TYPE[MyCalendar] = CalendarMeta(MyCalendar)

Vincent Raspal · Answer 7 · Fri Apr 01 2022 17:44:50 GMT+0800 (China Standard Time)

I finally worked it out with the suggested patch.
Unfortunately, all my soft needs to be des-arrowized and it will require a lot of time.
Too bad for me because I needed the extra fields to be parsed.
I will wait for stable v0.8 to be released and then I will weigh up the pros and cons.
Thank you for your help.

Niko Fink · Answer 8 · Fri Apr 01 2022 19:40:30 GMT+0800 (China Standard Time)

If it's just Event.begin and end that's problematic for you, you could also fix that in a custom subclass that automatically (un-)wraps the arrow/built-in datetime objects. But please be aware that wrapping timezone-naïve i.e. local datetime objects in arrow interprets them in UTC (and not your computer's local timezone), so floating events might get interpreted wrong if you continue to use arrow.

Vincent Raspal · Answer 9 · Fri Apr 01 2022 21:01:14 GMT+0800 (China Standard Time)

Sorry if I look stupid but, when I initiate a new MyCalendar (subclass of Calendar) instance, how can I by-pass the built-in mechanics to populate it with MyEvent (subclass of Event) instances rather than Event instances? MyCalendar.__init__ delegates a lot to its super class.
My first guess would be to create a MyEvent "clone" of every Event, then update the events set in my_calendar instance. I do not find it is smart...
These are certainly basics python and inheritance-related problems
I ask this question because, beyond the arrow/datetime problem (which can be fixed), I have been thinking for a long time to subclass Event to add some extra specific features.

I keep your advice about arrow and timezone in mind. Still need some time for that.
Thanks again

Niko Fink · Answer 10 · Sat Apr 02 2022 22:53:53 GMT+0800 (China Standard Time)

I created a proof-of-concept how this could be done here. Unfortunately, with the Converter system as it currently is, it is somewhat cumbersome to do as you need to replace the whole initialize_converters method as no changes can be made once the respective modules are loaded (you need to restart the interpreter if you want change anything about the loaded converters). I'll definitely keep this use case in the back of my head when refactoring how this can be customized. (see #355)

Vincent Raspal · Answer 11 · Mon Apr 04 2022 15:01:05 GMT+0800 (China Standard Time)

Thank you. I will try this out ASAP (certainly next week) et will let you know how it works on my side.

0xC4 · Answer 12 · Thu Dec 15 2022 12:04:36 GMT+0800 (China Standard Time)

I guess this is solved either by 0.7.2 or 0.8.0dev0. Feel free to reopen if it's not the case :)