XMLBIFReader fails parsing moderately large networks

Question

XMLBIFReader fails parsing moderately large networks

systemssoperfect opened this issue a year ago · comments

systemssoperfect commented a year ago

Subject of the issue

XMLBIFReader fails parsing moderately large networks, with tables ~1M entries.

Your environment

pgmpy 0.1.22
Python 3.8.10
Ubuntu 20.04

Steps to reproduce

Create a node TABLE with more than 1 million entries

Expected behaviour

No error.

Actual behaviour

lxml throws the exception below:

Traceback (most recent call last):
File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
File "./Inferências/Modelos/Modelo5_BIF_2022_2t.xml", line 5765
lxml.etree.XMLSyntaxError: xmlSAX2Characters: huge text node, line 5765, column 10000791

Ankur Ankan · Answer 1 · Thu May 11 2023 15:54:04 GMT+0800 (China Standard Time)

@systemssoperfect Thanks for reporting this. Would it be by any chance, possible to share the model file so that I can reproduce the issue?

systemssoperfect · Answer 2 · Thu May 11 2023 19:35:43 GMT+0800 (China Standard Time)

Dear Ankur, Your library is great and this is a minor issue regarding lxml only. As lxml has the same interface of xml.etree and you also provide it as an alternative option, I've just commented lxml and copied the import of xml.etree. And xml.etree seems much faster than lxml. Again, thanks a lot for your beautiful work. Gerson try: # from lxml import etree import xml.etree.ElementTree as etree except ImportError: try: import xml.etree.ElementTree as etree except ImportError: # try: # import xml.etree.cElementTree as etree # commented out because xml.etree.cElementTree is giving errors with dictionary attributes print("Failed to import ElementTree from any known place")

…

On Thu, May 11, 2023 at 4:54 AM Ankur Ankan ***@***.***> wrote: @systemssoperfect <https://github.com/systemssoperfect> Thanks for reporting this. Would it be by any chance, possible to share the model file so that I can reproduce the issue? — Reply to this email directly, view it on GitHub <#1634 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADUOOACX42SOHXELZRCEH33XFSLKPANCNFSM6AAAAAAX5GHXDI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

systemssoperfect · Answer 3 · Fri May 12 2023 04:50:39 GMT+0800 (China Standard Time)

Here we go.

Modelo6_BIF_2022_2t.zip

Ankur Ankan · Answer 4 · Tue May 16 2023 03:09:01 GMT+0800 (China Standard Time)

@systemssoperfect Thanks for sharing the file. I hadn't realized that lxml's parser is much slower than xml.etree. I have changed everything to work with xml.etree now.