Incompatibility between `execute=False` and `tb.patch()`

[James Marchant]

It appears that is impossible to use the execute=False capability of testbook and tb.patch() together due to patched code getting run twice.

Consider this example with a simple two cell notebook; the first cell defines a simple function, the second calls it and prints out some results:

import io

from testbook import testbook

notebook_str = """
{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f92b5b71",
   "metadata": {},
   "outputs": [],
   "source": [
    "def test():\\n",
    "    return int()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ece15770",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(test())\\n",
    "print(test)"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "hide_notebook_metadata": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
"""

# Execute is False as we want to patch the target method before execution
with testbook(io.StringIO(notebook_str), execute=False) as tb:
    
    # Patch `int()` but could be any imported function
    with tb.patch("__main__.int", return_value="Hello World!") as mock_test:
        tb.execute()

        # Print cell outputs nicely so we can see what was printed
        # Mocked "Hello World!" is output as expected
        for idx, cell in enumerate(tb.cells):
            outputs_texts = [o["text"].strip() for o in cell.outputs]
            outputs_texts = "\n".join(outputs_texts)
            outputs_texts = outputs_texts.split("\n")

            if outputs_texts:
                for o_idx, output in enumerate(outputs_texts):
                    print(f"{idx}.{o_idx}: {output}")
            else:
                print(f"{idx}: No output")

        # I would expect this to work as executed in one cell
        mock_test.assert_called_once()  #  AssertionError: Expected 'int' to have been called once. Called 0 times.

Output:

0.0: 
1.0: Hello World!
1.1: <function test at 0x10b04e430>
2.0: 
AssertionError: Expected 'int' to have been called once. Called 0 times.

The underlying issue seems to be that .patch() works by injecting a new cell at the end, executing it immediately, but leaving it in place (rather than popping it off). This means that the patch-cell gets called twice: once at injection, once at .execute() which results in a new Mock instance being created and assigned to the same variable name.

Ideally there should be a way to specify to pop off the patch-cell (by passing through the appropriate kwarg to .inject()) or even for this to be the default behaviour.