[BUG][v2.7.4] _pygit2.GitError: trailing data at line 3734

Question

[BUG][v2.7.4] _pygit2.GitError: trailing data at line 3734

edunad opened this issue 7 months ago · comments

I just updated the version of the linter to the latest (v2.7.4) and it's giving out python errors

Traceback (most recent call last):
    File "/runner/_work/_actions/cpp-linter/cpp-linter-action/v2.7.4/venv/bin/cpp-linter", line 8, in <module>
      sys.exit(main())
               ^^^^^^
    File "/runner/_work/_actions/cpp-linter/cpp-linter-action/v2.7.4/venv/lib/python3.11/site-packages/cpp_linter/run.py", line 705, in main
      get_list_of_changed_files()
    File "/runner/_work/_actions/cpp-linter/cpp-linter-action/v2.7.4/venv/lib/python3.11/site-packages/cpp_linter/run.py", line 153, in get_list_of_changed_files
      Globals.FILES = parse_diff(Globals.response_buffer.text)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/runner/_work/_actions/cpp-linter/cpp-linter-action/v2.7.4/venv/lib/python3.11/site-packages/cpp_linter/git.py", line 89, in parse_diff
      diff_obj = Diff.parse_diff(diff_obj)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  _pygit2.GitError: trailing data at line 3734

https://github.com/edunad/rawrbox/pull/108/checks#step:9:641

Brendan · Answer 1 · Sat Dec 23 2023 04:27:09 GMT+0800 (China Standard Time)

the diff being parsed
full_pr.diff.txt

Brendan · Answer 2 · Sat Dec 23 2023 04:30:58 GMT+0800 (China Standard Time)

the diff chunk in question is:

diff --git a/render/content/fonts/SIL Open Font License.txt b/rawrbox.render/assets/fonts/SIL Open Font License.txt
similarity index 100%
rename from render/content/fonts/SIL Open Font License.txt
rename to rawrbox.render/assets/fonts/SIL Open Font License.txt

It looks like pygit2 (python wrapper for libgit2) doesn't like the spaces in this line:

rename from render/content/fonts/SIL Open Font License.txt

Brendan · Answer 3 · Sat Dec 23 2023 04:47:34 GMT+0800 (China Standard Time)

My first instinct is to massage the diff before passing it to pygit2.Diff.Parse_diff(), but that seems labor intensive (because of brute force using regular expression substitution).

I'm curious how git diff would output for commit edunad/rawrbox@efbcb07
I'm wondering

Is github excluding quotes for file paths with spaces in the diff?
Is libgit2 actually ill prepared to deal with filenames that have spaces?
Is this problem even related to spaces in the file path/name? I need the git diff output to play with this more.

FailCake · Answer 4 · Sat Dec 23 2023 06:04:55 GMT+0800 (China Standard Time)

My first instinct is to massage the diff before passing it to pygit2.Diff.Parse_diff(), but that seems labor intensive (because of brute force using regular expression substitution).

I'm curious how git diff would output for commit edunad/rawrbox@efbcb07 I'm wondering

Is github excluding quotes for file paths with spaces in the diff?

Is libgit2 actually ill prepared to deal with filenames that have spaces?

Is this problem even related to spaces in the file path/name? I need the git diff output to play with this more.

Here's the git diff of that commit, hope it helps

efbcb07664e4fdce2249d1a15bbaded39454b6ce.txt

Brendan · Answer 5 · Sat Dec 23 2023 06:29:16 GMT+0800 (China Standard Time)

@edunad This might take me a while to diagnose and resolve. You can roll back to v2.7.2 as we switched to pygit2 in v2.7.3.

Caution

Any version before v2.7.3 might not work well with custom docker images. It usually shows as a pip error about installing to "system packages" in Linux...

FailCake · Answer 6 · Sat Dec 23 2023 06:34:22 GMT+0800 (China Standard Time)

@edunad This might take me a while to diagnose and resolve. You can roll back to v2.7.2 as we switched to pygit2 in v2.7.3.

[!CAUTION]
Any version before v2.7.3 might not work well with custom docker images. It usually shows as a pip error about installing to "system packages" in Linux...

Will do, thanks for the quick support! <3

Brendan · Answer 7 · Sat Dec 23 2023 11:48:10 GMT+0800 (China Standard Time)

I couldn't find that line in a local git diff output (for edunad/rawrbox@efbcb07). So, I just cloned the repo and did (in bash)

git diff bbb25e4474a56c6ffdcd0c85a8f79c16fc3ef5e5...HEAD > renamed.diff

The resulting did not add quotes around that line's filename, so the GitError was raised again.

I raised this issue upstream at libgit2/pygit2#1260.

Brendan · Answer 8 · Sat Dec 23 2023 12:05:56 GMT+0800 (China Standard Time)

I tried to parse the diff in the rust binding git2-rs and got the same error, so I think this a problem with libgit2 C library itself.

FailCake · Answer 9 · Sat Dec 23 2023 20:51:21 GMT+0800 (China Standard Time)

Does it only happen with renamed files? I can also confirm that downgrading to v2.7.2 works

Brendan · Answer 10 · Sat Dec 23 2023 22:40:03 GMT+0800 (China Standard Time)

As far as I can tell, yes.

Brendan · Answer 11 · Sat Dec 23 2023 22:49:17 GMT+0800 (China Standard Time)

v2.7.2 and earlier parsed the diff manually using pure python regex patterns. I don't want to go back to that (as a fall back approach), but if the response from libgit2 devs is too slow (or nonexistent) then I will.

Brendan · Answer 12 · Mon Dec 25 2023 01:43:16 GMT+0800 (China Standard Time)

v2.7.5 reintroduces the legacy approach (pure python regex) for parsing diff when pygit2 fails.

The workflow's logs will show when pygit2 fails and legacy parsing is used as a fallback.

I'm still tracking this bug... I have a unit test will fail when the bug is fixed. And there's the issue I raised upstream.