Tests embeddeded in Markdown
mattvonrocketstein opened this issue · comments
Per this stack-overflow and my own experiments, doctest actually works with tests embedded in README.md. I can't figure out how to get xdoctest to do this.
This is a bug if the goal for xdoctest is to be completely compatible with classic doctest. Also, I do understand that markdown is an edge-case and generally working with modules is more useful. Still.. markdown support is extremely convenient for small projects
Markdown (and RST) file support is currently not implemented.
The place to add this would be in xdoctest/core.py in parse_calldefs. You can detect that the input path ends with ".md" and write the appropriate parser for them. The output needs to be a dictionary that maps "callnames" to CallDefNode objects.
A CallDefNode is fairly easy to construct. The line numbers aren't necessary for anything except reporting and can be spoofed for quick tests. The main thing to get right is that docstr should have a parsable doctest inside of it. This could just be the content of each markdown code block with a python prefix. Not sure how the original doctest handles it.
I'd be happy to merge a PR for this feature.
Note, looking at old issues, this is related to #49 and it looks like you can work around the issue by using pytest. E.g.
echo "
# My Markdown File
This is not a doctest
```python
>>> print("hello I am a doctest")
```
Neither is this.
```python
print('Do I count? Nope.')
```
```python
def foo():
"""
>>> print('Do I count? Yes. But be careful. This file is parsed as text. No markdown tags are considered.')
"""
```
" >foo.md
Then: pytest --xdoctest-glob foo.md
But this just parses the files as text and not markdown. Looking at the doctest implementation it seems like that's what it does as well. I suppose I consider this behavior as broken by default, which is why I don't count it against backwards compatability. The doctest package isn't markdown aware, it just takes whatever text you throw at it and runs it through a regex. If xdoctest is going to support this, I'd want to support it for real, and parse out where the markdown (or RST) code blocks are, treat them like docstrings and then pass them off to xdoctest. I'm not a fan of blindly globbing text files for >>> patterns.
parse out where the markdown (or RST) code blocks are
There may not be any! For full backwards compatibility with doctest you should accept any files, which just happen to have some lines starting with >>> . For example https://github.com/jalanb/pysyte/blob/__main__/pysyte/types/test/paths.test (Note the extension)
That file can be tested with
$ cd ..../pysyte
$ python -m doctest pysyte/types/test/paths.testOr can be included in pytest like
$ cd .../pysyte
$ pytest --doctest-glob=*.test pysyteI'm a big fan of doctests, and was considering switching hence. But I've hundreds of tests in *.test and *.md, and xdoctest's inability to handle those is (unfortunately) a hard stop for me.
@jalanb You can use xdoctest on text files with pytest. Install xdoctest and then change your invocation to:
pytest --xdoctest-glob=*.test pysyteIt's just the native runner that doesn't currently handle them, but it shouldn't be that hard to add support for it. As previously mentioned you could modify parse_calldefs to detect if the input file is a non python text file and then you could probably treat the entire file as a docstring and pass it to the existing freeform parser. I don't think it's very clean, but it's essentially what the stdlib doctest is doing.