Overflow Error with some .arktribe files (xpost from arkpy-examples)

[Derek Ries]

Original issue posted in DerekRies/arkpy-examples#1

I'm encountering the following error when attempting to read in .arktribe files.

OverflowError: Python int too large to convert to C long

Some have issues however, the majority (83%) of the .arktribe files I've run worked fine.

I can reproduce this issue with a script as simple as this. I have also attached some examples of .arktribe files that encounter this issue.

from arkpy import ark, utils

def main():
    file_path = 'Fails/1115390102.arktribe'
    tribe = ark.ArkTribe(file_path)

if __name__ == '__main__':
    main()

Fails.zip

Full error traceback below:

Traceback (most recent call last): File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\tribetest.py", line 9, in <module> main() File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\tribetest.py", line 6, in main tribe = ark.ArkTribe(file_path) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\ark.py", line 416, in __init__ self._load_from_stream(file_path) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\ark.py", line 433, in _load_from_stream struct = arktypes.load_struct(stream) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 28, in load_struct struct = STRUCTS.get(name, BaseStruct)(stream=stream) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 895, in __init__ self.load_and_set_next_property(stream) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 120, in load_and_set_next_property name, prop_type, value = load_property(stream) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 41, in load_property return (name, prop_type, PROPERTIES[prop_type](stream=stream)) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\arktypes.py", line 219, in __init__ self.value = stream.readNullTerminatedString() File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\binary.py", line 63, in readNullTerminatedString s = self.unpack(str(length - 1) + 's', length - 1) File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\binary.py", line 194, in unpack return unpack(fmt, self.readBytes(length))[0] File "C:\Users\user\Desktop\codingprojects\arktribelogproject\arkpy-master\arkpy-master\arkpy\binary.py", line 22, in readBytes return self.base_stream.read(length) OverflowError: Python int too large to convert to C long

[Derek Ries]

@Failoe I wasn't able to reproduce this issue as all those files work correctly for me. One thing they have in common though is the use of Unicode characters, which used to be a problem for the library. Are you on the latest version (0.1.3)?

It looks like you're using Windows, so I'm going to go ahead and say its probably an issue with my code for parsing Unicode strings on Windows. I haven't tested much of any of this on Windows so I'll have to set up a dev environment there for future testing. Might take me a couple days to find the time to do that though, so if you want to take a crack at fixing this go for it, I can always point you in the right direction.

[Failoe]

I am indeed using Windows (Server 2012 R2 specifically). I've verified
that the newest version of arkpy is installed (0.1.3) and that all the
modules in dependencies.txt have been installed. I'm still encountering
the OverflowError: Python int too large to convert to C long error. Do
you have any ideas as to steps I could take to troubleshoot this?

On Fri, Sep 23, 2016 at 4:00 PM, Derek Ries [e-mail]
wrote:

@Failoe https://github.com/Failoe I wasn't able to reproduce this issue
as all those files work correctly for me. One thing they have in common
though is the use of Unicode characters, which used to be a problem for the
library. Are you on the latest version (0.1.3)?

It looks like you're using Windows, so I'm going to go ahead and say its
probably an issue with my code for parsing Unicode strings on Windows. I
haven't tested much of any of this on Windows so I'll have to set up a dev
environment there for future testing. Might take me a couple days to find
the time to do that though, so if you want to take a crack at fixing this
go for it, I can always point you in the right direction.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#20 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AQVyf5VDCnlkP_Oee4_v7MLQ7v0Y6ZuAks5qtFoegaJpZM4KDK9w
.

[Failoe]

As a followup to this i replaced all instances of xrange with
itertool.count to try to solution in this SO article:
http://stackoverflow.com/questions/22114088/overflowerror-python-int-too-large-to-convert-to-c-long

The error I received was "MemoryError" and when I printed the length
variable in binary.py the value was "1953384788" which is probably my
issue. Though it seems like something is wrong here unfortunately, I do
not know enough about this to pinpoint it. If you have any advice I'd love
to hear it.

On Fri, Oct 7, 2016 at 10:53 PM, Ryan [e-mail] wrote:

I am indeed using Windows (Server 2012 R2 specifically). I've verified
that the newest version of arkpy is installed (0.1.3) and that all the
modules in dependencies.txt have been installed. I'm still encountering
the OverflowError: Python int too large to convert to C long error. Do
you have any ideas as to steps I could take to troubleshoot this?

On Fri, Sep 23, 2016 at 4:00 PM, Derek Ries [e-mail]
wrote:

@Failoe https://github.com/Failoe I wasn't able to reproduce this
issue as all those files work correctly for me. One thing they have in
common though is the use of Unicode characters, which used to be a problem
for the library. Are you on the latest version (0.1.3)?

It looks like you're using Windows, so I'm going to go ahead and say its
probably an issue with my code for parsing Unicode strings on Windows. I
haven't tested much of any of this on Windows so I'll have to set up a dev
environment there for future testing. Might take me a couple days to find
the time to do that though, so if you want to take a crack at fixing this
go for it, I can always point you in the right direction.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#20 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQVyf5VDCnlkP_Oee4_v7MLQ7v0Y6ZuAks5qtFoegaJpZM4KDK9w
.