Pandoc hangs when using +sourcepos on markdown files ending with ordered or unordered lists

[Christian Buffin]

Problem

Test files

a.md

# a

- one
- two

b.md

# b

Command used

pandoc --from commonmark+sourcepos --to native *.md

Explanations

The command above hangs.

• if I remove +sourcepos from the command, it works well
• if I add a blank line and a dot at the end of a.md last line, it works well
• if I use no list at the end of the file in a.md, it works well

Environment

• pandoc 2.14.2
• Ubuntu 18.04.5 LTS

[John MacFarlane]

This seems to be a commonmark-hs bug. I can reproduce it with the cli program commonmark that is built with commonmark-hs:

commonmark --sourcepos a.md b.md

[John MacFarlane]

Simpler test case:
a.md:

- a

b.md

b

[John MacFarlane]

Result of --tokenize:

[Tok {tokType = Symbol '-', tokPos = "a.md" (line 1, column 1), tokContents = "-"},Tok {tokType = Spaces, tokPos = "a.md" (line 1, column 2), tokContents = " "},Tok {tokType = WordChars, tokPos = "a.md" (line 1, column 3), tokContents = "a"},Tok {tokType = LineEnd, tokPos = "a.md" (line 1, column 4), tokContents = "\n"},Tok {tokType = WordChars, tokPos = "b.md" (line 1, column 1), tokContents = "b"},Tok {tokType = LineEnd, tokPos = "b.md" (line 1, column 2), tokContents = "\n"}]

[John MacFarlane]

With my example, adding blank lines to the end of a.md or beginning of b.md makes no difference; it still hangs.

[John MacFarlane]

Lab note: changing the show instance for SourceRange so it doesn't use prettyShow stops this from hanging. So the hanging seems due to a bug in prettyShow.

[Christian Buffin]

Thank you for your quick answer. I'll wait for the next pandoc version to remove my workaround from my code.