Pointer files are not replaced by the original files
Natsuspace opened this issue · comments
My project bundles together several submodules into a framework. Both the project and the submodules use Git LFS to store small reference recordings and various input files for unit and integration tests. And I am running a Jenkins build.
In the Windows builds the LFS files from submodules are not downloaded and all the tests that require such files fail.
Weirdly, the same build on Ubuntu18/20 does not have this issue.
The Windows specific errors can be seen from messages like:
Caught exception, test will exit: 'Not a zip archive'
Caught exception, test will exit: 'the recording file '2021.01.27_at_07.50.35_radar-mi_193.rec' is corrupted at offset 0'
etc
My Git config ->
[2024-05-06T04:36:32.507Z] diff.astextplain.textconv=astextplain
[2024-05-06T04:36:32.507Z] filter.lfs.clean=git-lfs clean -- %f
[2024-05-06T04:36:32.507Z] filter.lfs.smudge=git-lfs smudge -- %f
[2024-05-06T04:36:32.507Z] filter.lfs.process=git-lfs filter-process
[2024-05-06T04:36:32.507Z] filter.lfs.required=true
[2024-05-06T04:36:32.507Z] http.sslbackend=openssl
[2024-05-06T04:36:32.507Z] http.postbuffer=524288000
[2024-05-06T04:36:32.507Z] core.autocrlf=true
[2024-05-06T04:36:32.507Z] core.fscache=true
[2024-05-06T04:36:32.507Z] core.symlinks=false
[2024-05-06T04:36:32.507Z] core.longpaths=true
[2024-05-06T04:36:32.507Z] pull.rebase=false
[2024-05-06T04:36:32.507Z] init.defaultbranch=master
My suspicion is that, is it due to the smudge option?
Hey,
I don't think this is due to the smudge option; that appears normal. Can you provide a printout of the environment on that machine while Jenkins is running that task? The reason I ask is that if it contains GIT_LFS_SKIP_SMUDGE=1, then that will cause the pointer files to remain in the working tree and cause this problem.
Hey,
I don't think this is due to the smudge option; that appears normal. Can you provide a printout of the environment on that machine while Jenkins is running that task? The reason I ask is that if it contains
GIT_LFS_SKIP_SMUDGE=1, then that will cause the pointer files to remain in the working tree and cause this problem.
Yah, it does have this, How do I disable this?
Name Value
17:40:09 GIT_LFS_SKIP_SMUDGE 1
I'm sorry, I don't use Jenkins and don't know how it can be configured. Just as a note, I think, from some quick Googling, that there is a Jenkins task to check out LFS objects which invokes something like git lfs pull, which you might be able to use. git lfs pull, assuming it has the right credentials, should be sufficient to get the pointer files replaced by large files.
The Git plugin for Jenkins defines the "Git LFS pull after checkout" behaviour that you can add to a branch source in a pipeline project. I don't remember whether it covers submodules too.
Alternatively, you can use the sh or bat or pwsh pipeline step to run the git lfs pull command, perhaps combined with git submodule foreach. However, Jenkins does not automatically pass the Git repository access credentials to such commands. So if your repository is private, then you may have to place the command within a withCredentials step and configure the credential in Jenkins so that the pipeline is authorised to use it.