[develop2] Issues with cmake-conan + CMakeToolchain + VS

Question

[develop2] Issues with cmake-conan + CMakeToolchain + VS

simonimpey opened this issue 10 months ago · comments

Hi,

I'm not certain if this is an issue or user error.

Having migrated a number of our project to conan2/cmake-conan successfully I am trying to fully migrate the last one, which is also used as a dependency, through Conan, by our other projects. It works fine in this use case and is correctly built on demand by Conan when installing it in another project.

However when trying to develop the project itself using Visual Studio I run into an issue whereby the CMake generation step seems to get into an infinite loop. I have attached the output below:

Command line: "C:\WINDOWS\system32\cmd.exe" /c "%SYSTEMROOT%\System32\chcp.com 65001 >NUL && "C:\PROGRAM FILES\MICROSOFT VISUAL STUDIO\2022\PROFESSIONAL\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\CMAKE\CMake\bin\cmake.exe"  -G "Ninja"  -DCMAKE_C_COMPILER:STRING="cl" -DCMAKE_CXX_COMPILER:STRING="cl" -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES:STRING="Conan.cmake" -DPROTO_SOURCE_DIR:STRING="C:/Code/Data.Interfaces/CppImpl/../net" -DCMAKE_BUILD_TYPE:STRING="Release" -DCMAKE_INSTALL_PREFIX:STRING="C:/Code/Data.Interfaces/CppImpl/out/install/x64/Release"  -DCMAKE_MAKE_PROGRAM="C:\PROGRAM FILES\MICROSOFT VISUAL STUDIO\2022\PROFESSIONAL\COMMON7\IDE\COMMONEXTENSIONS\MICROSOFT\CMAKE\Ninja\ninja.exe" "C:\Code\Data.Interfaces\CppImpl" 2>&1"
1> Working directory: C:/Code/Data.Interfaces/CppImpl/out/build/x64/Release
1> [CMake] -- CMake-Conan: first find_package() found. Installing dependencies with Conan
1> [CMake] -- CMake-Conan: Checking if a default profile exists
1> [CMake] C:\ProgramData\Conan\profiles\default
1> [CMake] -- CMake-Conan: cmake_system_name=Windows
1> [CMake] -- CMake-Conan: cmake_system_processor=x86_64
1> [CMake] -- CMake-Conan: CMake compiler=MSVC
1> [CMake] -- CMake-Conan: CMake compiler version=19.36.32535.0
1> [CMake] -- CMake-Conan: [settings] compiler=msvc
1> [CMake] -- CMake-Conan: [settings] compiler.version=193
1> [CMake] -- CMake-Conan: Creating profile C:/Code/Data.Interfaces/CppImpl/out/build/x64/Release/conan_host_profile
1> [CMake] -- CMake-Conan: Profile: 
1> [CMake] include(default)
1> [CMake] [settings]
1> [CMake] arch=x86_64
1> [CMake] os=Windows
1> [CMake] compiler=msvc
1> [CMake] compiler.version=193
1> [CMake] compiler.cppstd=20
1> [CMake] build_type=Release
1> [CMake] [conf]
1> [CMake] tools.cmake.cmaketoolchain:generator=Ninja
1> [CMake] 
1> [CMake] -- CMake-Conan: Installing single configuration Release
1> [CMake] -- CMake-Conan: conan install C:/Code/Data.Interfaces/CppImpl -of=C:/Code/Data.Interfaces/CppImpl/out/build/x64/Release/conan -pr;C:/Code/Data.Interfaces/CppImpl/out/build/x64/Release/conan_host_profile;--build=missing;-g;CMakeDeps
1> [CMake] 
1> [CMake] ======== Input profiles ========
1> [CMake] Profile host:
1> [CMake] [settings]
1> [CMake] arch=x86_64
1> [CMake] build_type=Release
1> [CMake] compiler=msvc
1> [CMake] compiler.cppstd=20
1> [CMake] compiler.runtime=dynamic
1> [CMake] compiler.runtime_type=Release
1> [CMake] compiler.version=193
1> [CMake] os=Windows
1> [CMake] [conf]
1> [CMake] tools.cmake.cmaketoolchain:generator=Ninja
1> [CMake] 
1> [CMake] Profile build:
1> [CMake] [settings]
1> [CMake] arch=x86_64
1> [CMake] build_type=Release
1> [CMake] compiler=msvc
1> [CMake] compiler.cppstd=14
1> [CMake] compiler.runtime=dynamic
1> [CMake] compiler.runtime_type=Release
1> [CMake] compiler.version=193
1> [CMake] os=Windows
1> [CMake] [conf]
1> [CMake] 
1> [CMake] 
1> [CMake] 
1> [CMake] ======== Computing dependency graph ========
1> [CMake] Graph root
1> [CMake]     conanfile.py (cm-data-interfaces/0.1.2): C:/Code/Data.Interfaces/CppImpl\conanfile.py
1> [CMake] Requirements
1> [CMake]     fmt/9.1.0#44302d39c5a4bf7de8a39adc50bb4568 - Cache
1> [CMake]     parallel-hashmap/1.37#2e2c32fc7b50350290e19d7365f10bdd - Cache
1> [CMake]     protobuf/3.21.9#7325b70171a0225ff4f4b3bd1cc46ae9 - Cache
1> [CMake]     zlib/1.2.13#e377bee636333ae348d51ca90874e353 - Cache
1> [CMake] Test requirements
1> [CMake]     gtest/cci.20210126#dafbdf84b58cd687075ace7314651c1a - Cache
1> [CMake] 
1> [CMake] ======== Computing necessary packages ========
1> [CMake] fmt/9.1.0: Checking 3 compatible configurations:
1> [CMake] fmt/9.1.0: 'bed280a9c51d41820ca64294cf373083ad852aa8': compiler.cppstd=14
1> [CMake] fmt/9.1.0: Main binary package '81000d7a34fb02600d7a1deb46c478bd09216966' missing. Using compatible package 'bed280a9c51d41820ca64294cf373083ad852aa8'
1> [CMake] Requirements
1> [CMake]     fmt/9.1.0#44302d39c5a4bf7de8a39adc50bb4568:bed280a9c51d41820ca64294cf373083ad852aa8#b03f37cdecc6ef2d5a23fa92890a0133 - Cache
1> [CMake]     parallel-hashmap/1.37#2e2c32fc7b50350290e19d7365f10bdd:da39a3ee5e6b4b0d3255bfef95601890afd80709#852cc589d44e7bb9307c4168f3dfe8d9 - Cache
1> [CMake]     protobuf/3.21.9#7325b70171a0225ff4f4b3bd1cc46ae9:a209bbfecc765c1f8b396c95b54aafa5de1d6d7c#4b40a8cb20418700d3ce5249c7a2f24f - Cache
1> [CMake]     zlib/1.2.13#e377bee636333ae348d51ca90874e353:7bfde258ff4f62f75668d0896dbddedaa7480a0f#6612117c7b5ed9d05294c47602331be1 - Cache
1> [CMake] Test requirements
1> [CMake]     gtest/cci.20210126#dafbdf84b58cd687075ace7314651c1a:de6d049385a7644322dadbb4b78ae7a98e538a65#376afae32fce7717c00c7551df338630 - Cache
1> [CMake] 
1> [CMake] ======== Installing packages ========
1> [CMake] fmt/9.1.0: Already installed! (1 of 5)
1> [CMake] gtest/cci.20210126: Already installed! (2 of 5)
1> [CMake] parallel-hashmap/1.37: Already installed! (3 of 5)
1> [CMake] zlib/1.2.13: Already installed! (4 of 5)
1> [CMake] protobuf/3.21.9: Already installed! (5 of 5)
1> [CMake] WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
1> [CMake] WARN: deprecated:     'cpp_info.names' used in: fmt/9.1.0, gtest/cci.20210126, protobuf/3.21.9, zlib/1.2.13, parallel-hashmap/1.37
1> [CMake] WARN: deprecated:     'cpp_info.build_modules' used in: parallel-hashmap/1.37, protobuf/3.21.9
1> [CMake] WARN: deprecated:     'cpp_info.filenames' used in: protobuf/3.21.9
1> [CMake] WARN: deprecated:     'env_info' used in: protobuf/3.21.9
1> [CMake] 
1> [CMake] ======== Finalizing install (deploy, generators) ========
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): Writing generators to C:/Code/Data.Interfaces/CppImpl/out/build/x64/Release/conan
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): Generator 'CMakeToolchain' calling 'generate()'
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): CMakeToolchain generated: conan_toolchain.cmake
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): Preset 'conan-release' added to CMakePresets.json. Invoke it manually using 'cmake --preset conan-release' if using CMake>=3.23
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): If your CMake version is not compatible with CMakePresets (<3.23) call cmake like: 'cmake <path> -G Ninja -DCMAKE_TOOLCHAIN_FILE=C:\Code\Data.Interfaces\CppImpl\out\build\x64\Release\conan\conan_toolchain.cmake -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_BUILD_TYPE=Release'
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): CMakeToolchain generated: CMakePresets.json
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): CMakeToolchain generated: ..\..\..\..\..\CMakeUserPresets.json
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): Generator 'CMakeDeps' calling 'generate()'
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): Generating aggregated env files
1> [CMake] conanfile.py (cm-data-interfaces/0.1.2): Generated aggregated env files: ['conanbuild.bat', 'conanrun.bat']
1> [CMake] Install finished successfully
1> [CMake] -- CMake-Conan: CONAN_GENERATORS_FOLDER=C:/Code/Data.Interfaces/CppImpl/out/build/x64/Release/conan
1> [CMake] -- CMake-Conan: CONANFILE=C:/Code/Data.Interfaces/CppImpl/conanfile.py (cm-data-interfaces/0.1.2)
1> [CMake] -- Conan: Component target declared 'fmt::fmt'
1> [CMake] -- CMake-Conan: find_package(phmap) found, 'conan install' already ran
1> [CMake] -- Conan: Target declared 'phmap'
1> [CMake] -- CMake-Conan: find_package(protobuf) found, 'conan install' already ran
1> [CMake] -- Conan: Component target declared 'protobuf::libprotobuf'
1> [CMake] -- Conan: Component target declared 'protobuf::libprotoc'
1> [CMake] -- Conan: Target declared 'protobuf::protobuf'
1> [CMake] -- CMake-Conan: find_package(ZLIB) found, 'conan install' already ran
1> [CMake] -- Conan: Target declared 'ZLIB::ZLIB'
1> [CMake] -- Conan: Including build module from 'C:/ProgramData/Conan/p/b/proto80d133688d5b5/p/lib/cmake/protobuf/protobuf-generate.cmake'
1> [CMake] -- Conan: Including build module from 'C:/ProgramData/Conan/p/b/proto80d133688d5b5/p/lib/cmake/protobuf/protobuf-module.cmake'
1> [CMake] -- Conan: Including build module from 'C:/ProgramData/Conan/p/b/proto80d133688d5b5/p/lib/cmake/protobuf/protobuf-options.cmake'
1> [CMake] -- Looking for RiskModel protobuf definitions in C:/Code/Data.Interfaces/net
1> [CMake] -- CMake-Conan: find_package(GTest) found, 'conan install' already ran
1> [CMake] -- Conan: Component target declared 'GTest::gtest'
1> [CMake] -- Conan: Component target declared 'GTest::gtest_main'
1> [CMake] -- Conan: Component target declared 'GTest::gmock'
1> [CMake] -- Conan: Component target declared 'GTest::gmock_main'
1> [CMake] -- Conan: Target declared 'gtest::gtest'
1> [CMake] -- Configuring done (2.7s)
1> [CMake] -- Generating done (0.1s)

As soon as the CMake generation is done it restarts immediately and continues to repeat the same tasks over and over.

If I run CMake from the command line:

cmake -G Ninja -S CppImpl -B CppImpl\Windows\x64\Release -DCMAKE_BUILD_TYPE=Release -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=C:\Code\Data.Interfaces\CppImpl\Conan.cmake

then everything works as expected.

Several points/questions:

The issue appears as soon as I add CMakeToolchain to the list of generators. I believe this is required in order for Conan to be able to build the package when Conan is installing it as a dependency, but maybe this is incorrect.
I suspect the issue is related to the fact that in VS we pass the CMAKE_PROJECT_TOP_LEVEL_INCLUDES variable through the CMakePresets.json file. I'm wondering if during the CMakeToolchain generate step this CMake variable is being picked up and causing the install to run again and thus triggering the infinite loop. Are you aware of another/better way to pass this variable?
Separately, it seems odd to me that the process of installing the dependencies for a package should cause Conan to generate the build files for the package itself. A side effect of this process seems to be the unwanted creation of a CMakeUserPresets.json which will be picked up by Visual Studio even without the infinite loop issue. Indeed if I cance

I would be grateful for any suggestions on how to solve this issue...

James · Answer 1 · Fri Aug 04 2023 16:14:25 GMT+0800 (China Standard Time)

Hi @simonimpey

The issue appears as soon as I add CMakeToolchain to the list of generators. I believe this is required in order for Conan to be able to build the package when Conan is installing it as a dependency, but maybe this is incorrect.

yes, you are right, this CMakeToolchain will be important if you ever build this package as a dependency, with a conan create or any other build in the cache, driven by Conan instead of driven by CMake.

I suspect the issue is related to the fact that in VS we pass the CMAKE_PROJECT_TOP_LEVEL_INCLUDES variable through the CMakePresets.json file. I'm wondering if during the CMakeToolchain generate step this CMake variable is being picked up and causing the install to run again and thus triggering the infinite loop. Are you aware of another/better way to pass this variable?

Yes, this can be related. CMakeToolchain is generating some presets files, and there could be some interaction.

Can you please trying to set toolchain.user_presets_path = "" like:

def generate(self):
     tc = CMakeToolchain(self)
     tc.user_presets_path = ""
     tc.generate()

And see if it still happens?

Simon · Answer 2 · Fri Aug 04 2023 17:34:19 GMT+0800 (China Standard Time)

Can you please trying to set toolchain.user_presets_path = "" like:
def generate(self):
     tc = CMakeToolchain(self)
     tc.user_presets_path = ""
     tc.generate()
And see if it still happens?

Yes that seems to solve the issue parsing the CMakeLists file from Visual Studio.

If I might inquire again, why does installing the dependencies of a project via conan install trigger the generation of build files for the project itself? These seem to be two separate steps to me, as in this case, where we simply want the dependencies as the project itself will be built via Visual Studio rather than Conan in this use case.

James · Answer 3 · Fri Aug 04 2023 22:08:54 GMT+0800 (China Standard Time)

Thanks for the feedback.
Could we then consider tc.user_presets_path = "" a solution?
We are also considering adding a conf to disable the generation of the CMakeUserPresets.json, which could also help in this case.

If I might inquire again, why does installing the dependencies of a project via conan install trigger the generation of build files for the project itself? These seem to be two separate steps to me, as in this case, where we simply want the dependencies as the project itself will be built via Visual Studio rather than Conan in this use case.

Because the dependencies must match the current project settings too. This is why the generate() process of preparing the project typically has 2 parts:

XXXDeps to generate information about dependencies
XXXToolchain to generate the mapping from the current settings + options to the build system of the project.

So doing conan install ... + cmake ... (or calling the native build system), should work. But if the first conan install is passing a profile that changes the compiler, compiler version, libcxx or cppstd, the build_type, etc, the cmake should have that information too so it can use it to align, and not require developers to have to manually specify the full configuration that matches the Conan one of dependencies. Doing cmake .... -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake is easy, but having to pass all the toolchain defined variables in command line would be really unergonomic and problematic for developers (we know it is painful, the early Conan 1.X cmake integration was this way).

Simon · Answer 4 · Fri Aug 04 2023 22:19:17 GMT+0800 (China Standard Time)

Thanks for the explanation, very useful.

Thanks for the feedback. Could we then consider tc.user_presets_path = "" a solution? We are also considering adding a conf to disable the generation of the CMakeUserPresets.json, which could also help in this case.

I'm happy to consider this a solution, the package seems to build happily in both Visual Studio and from Conan itself with the generation disabled. Having a conf section to disable the generation seems 'neater' than explicitly setting the path to nothing in the conanfile.py itself.

Are there any significant side effects of not generating this file when actually creating a package from the conanfile.py recipe?

James · Answer 5 · Fri Aug 04 2023 22:21:22 GMT+0800 (China Standard Time)

Are there any significant side effects of not generating this file when actually creating a package from the conanfile.py recipe?

No, the only utility is for developers that can call cmake --preset to that generated file conveniently, but it is not used internally by Conan