Huge memory leak when running a Pipe inside the Managed Monad

[Stephan Schiffels]

Consider this program that simply reads a file and outputs to standard out.

import Control.Monad.Managed (runManaged, managed)
import Control.Monad.IO.Class (liftIO)
import Pipes ((>->), runEffect)
import qualified Pipes.Prelude as P
import System.Environment (getArgs)
import System.IO (withFile, IOMode(..))

main = runManaged $ do
    [filename] <- liftIO getArgs
    handle <- managed (withFile filename ReadMode)
    runEffect $ P.fromHandle handle >-> P.stdoutLn

Note that I am actually running the core loop itself inside the Managed monad (there is no liftIO in the last line). When applied to a large file (and redirecting the output to /dev/null), the resulting program has a massive space leak. After a few seconds the program consumes hundreds of Megabytes in memory. Now consider a revised program, where the last line is replaced by

liftIO . runEffect $ P.fromHandle handle >-> P.stdoutLn

Now the loop itself runs in the IO monad, and there is no space leak.

In the first example, I may be using the Managed monad in a naive way, but there is nothing I found in the documentation of either Managed or Pipes that gives me a hint that this is a problem. I am not expert enough to understand exactly what causes this leak, but I assume that the implicit recursion in the Pipes Monad somewhat becomes non-tail-recursive inside the Managed monad. Anyway, it's easy to avoid when you know it, but I'd like to know whether this is a bug that can be fixed inside Pipes or Managed, or whether this is expected to happen. If the latter, it might be wise to warn against this more explicitly. Thanks!

[Gabriella Gonzalez]

Yeah, this is a problem with the managed library and not pipes. Here's a simpler reproducing example:

import Control.Monad
import Control.Monad.Managed

main = runManaged (forever (liftIO (print 1)))

That will also leak space, whereas main = forever (print 1) will not leak space.

I believe the issue here is that ContT (which Managed is a special case of) does not work well for infinite loops because, like you pointed out, the loop is not tail recursive. I'll update the documentation of managed to indicate this issue.

[Gabriella Gonzalez]

Alright, I added a warning to the documentation of managed in this commit: Gabriella439/managed@88e8b50