vcpkg_from_gitlab fails to download but project exists / URL is correct

Question

vcpkg_from_gitlab fails to download but project exists / URL is correct

patrikhuber opened this issue 24 days ago · comments

Describe the bug
vcpkg-custom-overlay/my-test-package/portfile.cmake with vcpkg_from_gitlab:

vcpkg_from_gitlab(
    GITLAB_URL https://gitlab.com
    OUT_SOURCE_PATH SOURCE_PATH
    REPO myorg/mysubgroup/my-test-package
    REF "${VERSION}"
    SHA512 0
    HEAD_REF main
)

vcpkg_cmake_configure(
    SOURCE_PATH "${SOURCE_PATH}"
)

vcpkg_cmake_install()

vcpkg_cmake_config_fixup(PACKAGE_NAME "my-test-package")

file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/debug/include")

# file(INSTALL "${SOURCE_PATH}/LICENSE" DESTINATION "${CURRENT_PACKAGES_DIR}/share/${PORT}" RENAME copyright)
configure_file("${CMAKE_CURRENT_LIST_DIR}/usage" "${CURRENT_PACKAGES_DIR}/share/${PORT}/usage" COPYONLY)

Then installing it from a custom overlay with:

.\vcpkg.exe install my-test-package --overlay-ports=C:\Users\MyUser\vcpkg-custom-overlay --head

It fails at:

Downloading https://gitlab.com/api/v4/projects/%6d%79%6f%72%67%2f%6d%79%73%75%62%67%72%6f%75%70%2f%6d%79%2d%74%65%73%74%2d%70%61%63%6b%61%67%65/repository/branches/main
error: Failed to download from mirror set
error: https://gitlab.com/api/v4/projects/%6d%79%6f%72%67%2f%6d%79%73%75%62%67%72%6f%75%70%2f%6d%79%2d%74%65%73%74%2d%70%61%63%6b%61%67%65/repository/branches/main: failed: status code 404

and

Downloading https://gitlab.com/api/v4/projects/%6d%79%6f%72%67%2f%6d%79%73%75%62%67%72%6f%75%70%2f%6d%79%2d%74%65%73%74%2d%70%61%63%6b%61%67%65/repository/branches/main
error: Failed to download from mirror set
error: https://gitlab.com/api/v4/projects/%6d%79%6f%72%67%2f%6d%79%73%75%62%67%72%6f%75%70%2f%6d%79%2d%74%65%73%74%2d%70%61%63%6b%61%67%65/repository/branches/main: failed: status code 404

BUT: If I put the above "encoded" URL into the browser (https://gitlab.com/api/v4/projects/%6d%79%6f%72%67%2f%6d%79%73%75%62%67%72%6f%75%70%2f%6d%79%2d%74%65%73%74%2d%70%61%63%6b%61%67%65/repository/branches/main), it resolves to https://gitlab.com/api/v4/projects/%6d%79%6f%72%67%2f%6d%79%73%75%62%67%72%6f%75%70%2f%6d%79%2d%74%65%73%74%2d%70%61%63%6b%61%67%65/repository/branches/main, and this results in a valid reply, not a 404:

(Note this is a private repo and I've changed the names, so if you run the above line in your browser, you will get a 404 in the browser too obviously as you won't have access to our internal project - but I don't get a 404 in the browser if I try it with the real internal name of the project.)

This should be easily reproducible by creating your own (even empty) GitLab repo and trying vcpkg_from_gitlab. It looks like something within vcpkg_from_gitlab is going wrong?

Environment

OS: Windows 11, latest vcpkg (today)

To Reproduce
See above.

Expected behavior
As the URL is clearly correct when manually putting it into the browser, I'd expect vcpkg_from_gitlab to be able to not display a 404, or give me a more descriptive error message.

Failure logs
See above & below output.

Additional context
Full cmdline output:

C:\Users\MyUser\vcpkg-clean [master ≡]> .\vcpkg.exe install my-test-package --overlay-ports=C:\Users\MyUser\Projects\vcpkg-custom-overlay --head
Computing installation plan...
The following packages will be built and installed:
    my-test-package:x64-windows@1.0.0 (+HEAD) -- C:\Users\MyUser\Projects\vcpkg-custom-overlay\my-test-package
Installing 1/1 my-test-package:x64-windows@1.0.0...
Building my-test-package:x64-windows@1.0.0 from HEAD...
C:\Users\MyUser\Projects\vcpkg-custom-overlay\my-test-package: info: installing overlay port from here
-- Downloading https://gitlab.com/api/v4/projects/%6d%69%6d%65%74%72%69%6b%2f%66%61%63%65%62%6f%77%2f%66%61%63%65%62%6f%77%66%69%6c%65%72%65%61%64%65%72/repository/branches/main -> myorg-mysubgroup-my-test-package-main.tar.gz.version...
[DEBUG] To include the environment variables in debug output, pass --debug-env
[DEBUG] Trying to load bundleconfig from C:\Users\MyUser\vcpkg-clean\vcpkg-bundle.json
[DEBUG] Failed to open: C:\Users\MyUser\vcpkg-clean\vcpkg-bundle.json
[DEBUG] Bundle config: readonly=false, usegitregistry=false, embeddedsha=nullopt, deployment=Git, vsversion=nullopt
[DEBUG] Metrics enabled.
[DEBUG] Feature flag 'binarycaching' unset
[DEBUG] Feature flag 'compilertracking' unset
[DEBUG] Feature flag 'registries' unset
[DEBUG] Feature flag 'versions' unset
[DEBUG] Feature flag 'dependencygraph' unset
Downloading https://gitlab.com/api/v4/projects/%6d%79%6f%72%67%2f%6d%79%73%75%62%67%72%6f%75%70%2f%6d%79%2d%74%65%73%74%2d%70%61%63%6b%61%67%65/repository/branches/main
error: Failed to download from mirror set
error: https://gitlab.com/api/v4/projects/%6d%79%6f%72%67%2f%6d%79%73%75%62%67%72%6f%75%70%2f%6d%79%2d%74%65%73%74%2d%70%61%63%6b%61%67%65/repository/branches/main: failed: status code 404
[DEBUG] D:\a\_work\1\s\src\vcpkg\base\downloads.cpp(997):
[DEBUG] Time in subprocesses: 0us
[DEBUG] Time in parsing JSON: 2us
[DEBUG] Time in JSON reader: 0us
[DEBUG] Time in filesystem: 872us
[DEBUG] Time in loading ports: 0us
[DEBUG] Exiting after 233 ms (216853us)

CMake Error at scripts/cmake/vcpkg_download_distfile.cmake:32 (message):

      Failed to download file with error: 1
      If you are using a proxy, please check your proxy setting. Possible causes are:

      1. You are actually using an HTTP proxy, but setting HTTPS_PROXY variable
         to `https://address:port`. This is not correct, because `https://` prefix
         claims the proxy is an HTTPS proxy, while your proxy (v2ray, shadowsocksr
         , etc..) is an HTTP proxy. Try setting `http://address:port` to both
         HTTP_PROXY and HTTPS_PROXY instead.

      2. If you are using Windows, vcpkg will automatically use your Windows IE Proxy Settings
         set by your proxy software. See https://github.com/microsoft/vcpkg-tool/pull/77
         The value set by your proxy might be wrong, or have same `https://` prefix issue.

      3. Your proxy's remote server is out of service.

      If you've tried directly download the link, and believe this is not a temporary
      download server failure, please submit an issue at https://github.com/Microsoft/vcpkg/issues
      to report this upstream download server failure.


Call Stack (most recent call first):
  scripts/cmake/vcpkg_download_distfile.cmake:270 (z_vcpkg_download_distfile_show_proxy_and_fail)
  scripts/cmake/vcpkg_from_gitlab.cmake:97 (vcpkg_download_distfile)
  C:/Users/MyUser/Projects/vcpkg-custom-overlay/my-test-package/portfile.cmake:1 (vcpkg_from_gitlab)
  scripts/ports.cmake:175 (include)


error: building my-test-package:x64-windows failed with: BUILD_FAILED
Elapsed time to handle my-test-package:x64-windows: 1.2 s

Patrik Huber · Answer 1 · Sat May 11 2024 06:32:50 GMT+0800 (China Standard Time)

In case this is an authentication issue (as our repository is private):

I can successfully do: git clone https://gitlab.com/myorg/mysubgroup/my-test-package.git from the same command-line I'm running vcpkg, without having to enter any username/password. I'm using Windows 11 and git-for-Windows with the (git/Windows) credentials manager. So presumably vcpkg's git should be able to have access to this repo, or does it use a separate git or credentials store?

I've read through https://learn.microsoft.com/en-us/vcpkg/users/authentication as well. I've tried "pre-seeding" (not sure if this is needed in my setup...):

"url=https://gitlab.com`npath=myorg/mysubgroup/my-test-package`nusername=myusername`npassword=mypassword`n"
 | git credential approve

but this didn't help either.

Osyotr · Answer 2 · Sat May 11 2024 17:44:24 GMT+0800 (China Standard Time)

vcpkg_from_gitlab does not use git. Use AUTHORIZATION_TOKEN argument or fall back to vcpkg_from_git.

Unfortunately, AUTHORIZATION_TOKEN is not documented yet.