Bitcode serialization fails with unset Python_EXECUTABLE variable

[Tim Grant]

Problem

The custom command to serialize the bitcode for liboslexec fails if the Python_EXECUTABLE CMake variable is not set. This is due to a slight change in the invocation of the serialization script:

-        COMMAND python "${CMAKE_SOURCE_DIR}/src/build-scripts/serialize-bc.py" ${llvm_bc} ${llvm_bc_cpp} ${prefix}
+        COMMAND ${Python_EXECUTABLE} "${CMAKE_SOURCE_DIR}/src/build-scripts/serialize-bc.py" ${llvm_bc} ${llvm_bc_cpp} ${prefix}

This can be worked around by setting Python_EXECUTABLE or setting USE_PYTHON=ON at configure time.

Perhaps for the sake of compatibility it would be better to either set a default value so that the build will work as before, or to issue a warning or error if the variable is not set?

Versions

• OSL branch/version: main (033dae3)
• OS: Ubuntu 20.04
• C++ compiler: clang 12.0.0
• LLVM version: 12.0.0
• OIIO version: N/A

[Christopher Kulla]

Ah apologies, I thought python was already being loaded in since it worked in my setup and in CI.

I'm guessing we just need an appropriately placed find_package(Python REQUIRED)?

Alternatively -- is there a more lightweight way of serializing that bytecode into a cpp?

[Larry Gritz]

@tgrant-nv were you building with USE_PYTHON=OFF?

Python is needed for the python bindings to the oslquery library, so there is a find_package(Python) already happening at some point. And also python must be available in order to run the testsuite (which are all driving by python scripts).

[Tim Grant]

@fpsunflower, no problem at all. Yes, adding a find_package does work as you would expect. Perhaps the find_python macro in pythonutils.cmake can be made aware of the Python requirement when the tests or bitcode generation are enabled.

My previous minimal configuration was setting USE_PYTHON=OFF, though of course I have Python (2.7 and 3.8) installed. The testsuite runs fine without enabling Python in the build, probably because runtest.py is executable and has the proper shebang.

[Christopher Kulla]

Makes sense. For windows, it seems that using Python_EXECUTABLE is preferable, at least in my setup the one in the path that run when you just use python is the wrong one, while cmake finds the right one via the pybind dependency.

I have yet to get the testuite fully working on windows - but I'm wondering if we should use the same strategy there ... (since I don't believe the #! will be picked up).