Exception in destructor

[maxmontgmx]

Hi all,
I get sometimes an exception during my software running :

.

The exception is related to a file access.
My software executes python scripts on different threads at the same time.

So I guess 2 different root causes : concurrent access to the same script file or a HDD corruption.

Can you help me to investigate more and find the real root cause?

Thanks

[slozier]

Sounds like the underlying TextWriter might have gotten GCd which causes the failure. Probably the best solution would be to make sure your files are closed on the Python side of things so the finalizer doesn't get invoked.

If that's not an option you could also file a PR to catch the ArgumentOutOfRangeException in the finalizer:

ironpython2/Src/IronPython/Runtime/PythonFile.cs

Lines 1097 to 1106 in 1452008

 ~PythonFile() { 
     try { 
         Dispose(false); 
     } catch (ObjectDisposedException) { 
     } catch (EncoderFallbackException) { 
         // flushing could fail due to encoding, ignore it 
     } catch (IOException) { 
         // flushing could fail, especially if one half of a pipe is closed 
     } 
 } 

[maxmontgmx]

Many thanks for your answer.
In "GCd", do you mean Garbage collector?
What do you mean in "Python side"?
I'm using IronPython API just to execute a python script. I never open file in write mode.

I'm using IronPython binaries, I don't build from source code. so not easy to add a try catch.

I appraciate your help. Because my app is in production and this issue is blocking.

[slozier]

In "GCd", do you mean Garbage collector?

Yes, garbage collected.

What do you mean in "Python side"? I'm using IronPython API just to execute a python script. I never open file in write mode.

I meant using open (or file) on the Python side of things, but if you're not opening anything for writing it shouldn't be hitting this code path. I guess the other possibility is it's the console streams which are failing (SharedIO.OutputWriter/SharedIO.ErrorWriter). Are you doing any sort of IO redirects?

I'm using IronPython binaries, I don't build from source code. so not easy to add a try catch.

I meant you could submit a pull request to this repo with the change and it could be included in the next release.

[maxmontgmx]

Thanks again for your help.
I don't see in my python scripts any direct file access (by writing). I use C# libraries to write in files but not IronPython functions (like open, write, ...)
So I don't see where instance of PythonFile is created.
I'm using version 2.7.7 (with .NET 3.5). It would be great to have a debug version witch displays the exception (in dialog box) with the name of the file which causes the error.
I don't manage building version 2.7.7 with .NET 3.5

[maxmontgmx]

I'm working hardly on this issue and I 'm doing IO redirection like that :

var ScriptOutputStream = new ScriptOutputStream();
ScriptOutputStream.StringWritten += OutputEventCallback;
m_Engine.Runtime.IO.SetOutput(ScriptOutputStream, Encoding.GetEncoding(1252));
m_Engine.Runtime.IO.SetErrorOutput(ScriptOutputStream, Encoding.GetEncoding(1252));

But in my case , OutputEventCallback do anything (empty function)

[slozier]

Hmm, you could try setting your IO before the engine is created so the standard console outputs are never initialized:

var runtime = Python.CreateRuntime();
runtime.IO.SetOutput(ScriptOutputStream, Encoding.GetEncoding(1252));
runtime.IO.SetErrorOutput(ScriptOutputStream, Encoding.GetEncoding(1252));
m_Engine = Python.GetEngine(runtime);

[maxmontgmx]

I can do your suggestion.
In parallel, I manage compiling with .NET3.5 and I make modification you suggested by catching exception.
But I get the same error. Maybe, not the right place.
What do you think? Thanks

[slozier]

Based on your screenshot it should definitely be the finalizer I linked. Not sure why it wouldn't work, maybe you app is not loading the DLL you compiled (maybe it's using a DLL from the GAC or something)? Not sure...

[maxmontgmx]

If I compile in Release mode, the generated DLL doesn't take in account my modification (checked with dotPeek). If I compile in Debug Mode, I see modification in dotPeek.
In parallel, I tried your suggested modification (IO setting). For moment, I don't obtain any crash (but it could be happen in the future)

[maxmontgmx]

Hi Solzier,
I come back again on this issue.
I put in place your suggested modification. And I don't get any error.
But I get a big memory leak. If I revert the modification, memory leak is removed.
I guess that this modification makes GC doesn't delete IronPython objects.
I remember I used CreateEngine with Option "LightweightScopes" enable :
m_Engine = Python.CreateEngine(new Dictionary<string, object> { { "LightweightScopes", true } });

If I keep my code without thisq option, I get back memory leak. So this option is required for the engine.

So How can I put I place in place your modification and enable this option at the same time?

Thanks again this your great support

[maxmontgmx]

Hi Solzier,
I come back again on this issue.
I put in place your suggested modification. And I don't get any error.
But I get a big memory leak. If I revert the modification, memory leak is removed.
I guess that this modification makes GC doesn't delete IronPython objects.
I remember I used CreateEngine with Option "LightweightScopes" enable :
m_Engine = Python.CreateEngine(new Dictionary<string, object> { { "LightweightScopes", true } });

I guess that with your modification, the engine creation doesn't use this option.
Is it the cause of my problem?

Thanks again this your great support

[slozier]

I have no idea if LightweightScopes is related to your memory leak, but you can use the same options with CreateRuntime:

var runtime = Python.CreateRuntime(new Dictionary<string, object> { { "LightweightScopes", true } });