Support interactive environment in PyCharm where there is no source file

[Hannes Bretschneider]

For my workflow, the greatest problem with this package at the moment is that it throws an error when running handout.Handout() in an interactive shell:

handout.Handout('/tmp')
Traceback (most recent call last):
  File "/h/hannes/anaconda3/envs/microexon-code-tf2/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3325, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-4-d909ee1709cf>", line 1, in <module>
    handout.Handout('/tmp')
  File "/h/hannes/anaconda3/envs/microexon-code-tf2/lib/python3.6/site-packages/handout/handout.py", line 25, in __init__
    self._source_text = inspect.getsource(module)
  File "/h/hannes/anaconda3/envs/microexon-code-tf2/lib/python3.6/inspect.py", line 973, in getsource
    lines, lnum = getsourcelines(object)
  File "/h/hannes/anaconda3/envs/microexon-code-tf2/lib/python3.6/inspect.py", line 955, in getsourcelines
    lines, lnum = findsource(object)
  File "/h/hannes/anaconda3/envs/microexon-code-tf2/lib/python3.6/inspect.py", line 768, in findsource
    file = getsourcefile(object)
  File "/h/hannes/anaconda3/envs/microexon-code-tf2/lib/python3.6/inspect.py", line 684, in getsourcefile
    filename = getfile(object)
  File "/h/hannes/anaconda3/envs/microexon-code-tf2/lib/python3.6/inspect.py", line 666, in getfile
    'function, traceback, frame, or code object'.format(object))
TypeError: None is not a module, class, method, function, traceback, frame, or code object

This is a problem for my (Jupyter-less) workflow as I like to use #%% in PyCharm to create run-able cells and send them to a shell with Ctrl+Enter. This way I can create my figures and do my analysis interactively and simply run the script at the end to obtain a rendered report. However, the error above throws a wrench into that workflow.

I think a workable simple solution would be for handout to simply do nothing when it detects an interactive shell.

[Danijar Hafner]

Thanks for reporting! I'm wondering what source file you'd expect the handout to be built from? If there is no source file, there won't be any comment cells or code cells. In principle, it would still be possible to allow to add media to a doc if this is a reasonable use case.

Also see #11, which seems to imply that Handout is working inside an interactive IPython environment.

[Evgeny Pogrebnyak]

Not for immediate implementation, but just to consider a workflow:

# a user creates an `InteractiveHandout` instance 
doc = InteractiveHandout('/tmp')
# alternative: 
doc = Handout('/tmp', interactive=True)

# user adds text, graphs, video, etc in interactive session calls 
doc.add_text("Print this text") 

# a user creates an artefact
doc.show()

This way there is no source to build from, but a user can try different things in console before writing a script to persist the report. Is this something close to your workflow, @hannes-brt?

I like the idea because it gives more food for thought for #9 on structuring the classes for different sources and outputs. This workflow suggests there may be no source as text file, just calls to handout instance.

[Hannes Bretschneider]

I don't necessarily think the package needs to work in terms of producing a report in a shell, but at least it shouldn't raise errors. I think the closest model from existing packages is pypublish from Pweave (http://mpastell.com/pweave/pypublish.html). Since it only uses special comment syntax to define markdown text, the code can run in a shell, as a script (python myscript.py), or in pypublish (pypublish myscript.py). The first two don't produce a report, but still run the code which you sometimes want to do. Pypublish has some problems of its own and Pweave is no longer being developed, so I'd hope handout can step into that gap, but I think that it's important running the code in a shell doesn't break things.

@epogrebnyak I don't think it's good to introduce different options or objects for interactive mode, because that again requires changing the code when running in different context. I think the simplest and easiest solution is for handout to simply do nothing when an interactive shell is detected.

@danijar Since you're local - I'd be happy to swap some ideas if you're ever at the Vector Institute.

[Danijar Hafner]

There a lot of good points here. Instead of an interactive mode, we could have a flag to not include any source code. That would work in all environments and would also cover an additional use case people have asked for. Would that be suitable here?

@epogrebnyak Yes, let's think about making the main class independent of the input and output types. Can you think of any inputs other than "current file" and "nothing"?

@hannes-brt I'm happy to chat, could you shoot me an email so we can coordinate a time, please?

[Evgeny Pogrebnyak]

Under the hood in Handout class we merge a chain of blocks derived from static analysis of a source file and a chain of blocks derived from add_x() сalls, in principle they can be separated. Answering @danijar question - for the source file there is either current file or nothing that can be used.

[Danijar Hafner]

@epogrebnyak Do you think we should separate them?