arvidn / libtorrent

Please provide the following information

libtorrent version (or branch): 2.0.5

platform/architecture: any

compiler and compiler version: gcc 7.5

I'm trying to build libtorrent from sources with boost. I am wondering if it is possible to produce a wheel by calling pip directly with options? My current code runs quite far but eventually fails and this seems to be due to trying to build in static instead of shared mode. Wonder how I can pass arguments through p2p to fix that?

Ref: SynoCommunity/spksrc#5180

If you are using 2.0.5 you will need to manually apply a patch before building against boost 1.78 can work since the patch is in RC_2_0 for 2.0.6 release, i could not see it applied in your link.

https://raw.githubusercontent.com/userdocs/qbittorrent-nox-static/master/qbt-build/patches/libtorrent/RC_2_0/patch

I also don't really deal with the python stuff but I am pretty sure the libtorrent lib can be/is statically linked via boost python and -fpic and it should not be an issue. It's not trying to build the whole thing static.

But you might want a second opinion on that.

Thnx for that, I'll do a bit more digging on it.

@userdocs I was able to get the build to work out on 64-bit archs using master in crossenv using pip, linked against boost 1.78. Although it fails on all 32-bit archs and my guess is address-model being set at 64 (not verified yet). Is there any way for me to pass arguments to pip to enforce it to 32 ?

Probably need this if you are using python to build it and not using b2 directly.

https://libtorrent.org/python_binding.html

Something like this example I modified.

python setup.py build_ext --b2-args="address-model=32" install

Thnx, and indeed it throws this on 32-bit:

      subprocess.CalledProcessError: Command '['b2', 'deprecated-functions=on', 'boost-link=static', 'libtorrent-link=static', 'crypto=openssl', 'variant=release', 'address-model=64', 'python=3.10', 'libtorrent-python=on', 'python-install-path=/tmp/pip-wheel-eawrxaw8/libtorrent_15105973efca45eb81da008d4056f086/bindings/python/build/lib.arm-unknown-linux-gnueabi-3.10', 'install_module', '--project-config=/tmp/tmpfsrvzdh_']' returned non-zero exit status 1.

but build_ext --b2-args="address-model=32" seems to have no effect... I tried variations of --global-option='build_ext' --global-option='--b2-args="address-model=32"' using single/double quotes but it had no effect.

On the other hand I also see this message which probably explains it:

WARNING: Disabling all use of wheels due to the use of --build-option / --global-option / --install-option.
...
Building wheels for collected packages: libtorrent
  WARNING: Ignoring --global-option when building libtorrent using PEP 517
  Building wheel for libtorrent (pyproject.toml): started

The full call to crossenv pip for cross-compiling armv7 target from x86_64 host is:

_PYTHON_HOST_PLATFORM=arm-unknown-linux-gnueabi \
CFLAGS=-I/home/spksrc/master/spksrc/toolchain/syno-armv7-6.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include -D__ARM_PCS_VFP=1 -I/home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/install//var/packages/python310/target/include  -I/home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/install/var/packages/python310/target/include/python3.10  \
CPPFLAGS=-I/home/spksrc/master/spksrc/toolchain/syno-armv7-6.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include -D__ARM_PCS_VFP=1 -I/home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/install//var/packages/python310/target/include  -I/home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/install/var/packages/python310/target/include/python3.10  \
CXXFLAGS=-I/home/spksrc/master/spksrc/toolchain/syno-armv7-6.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/usr/include -D__ARM_PCS_VFP=1 -I/home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/install//var/packages/python310/target/include  -I/home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/install/var/packages/python310/target/include/python3.10  \
LDFLAGS=-L/home/spksrc/master/spksrc/toolchain/syno-armv7-6.1/work/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi/sysroot/lib -D__ARM_PCS_VFP=1 -L/home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/install//var/packages/python310/target/lib -Wl,--rpath-link,/home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/install//var/packages/python310/target/lib -Wl,--rpath,/var/packages/python310/target/lib   \
/home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/../../../native/python310/work-native/install/usr/local/bin/pip wheel \
--disable-pip-version-check \
--no-binary :all: \
--cache-dir /home/spksrc/master/spksrc/spk/python310/../../distrib/pip \
--no-deps \
--wheel-dir /home/spksrc/master/spksrc/spk/python310/work-armv7-6.1/wheelhouse \
--global-option='build_ext' --global-option='--b2-args="address-model=32"' \
--no-build-isolation \
git+https://github.com/arvidn/libtorrent.git@master#egg=libtorrent==2.0.6

I was finally able to get it to work. Should I be using master or RC_2_0 and any timeframe as of when 2.0.6 will get released?

you would use RC_2_0 not master.

No idea on the timeframe but you can either patch 2.0.5 whilst it the current release or use RC_2_0 as that is where all v2 releases are from.

Usually as the time for a new release comes you see the dev merging version changes into that relevant RC branches.

Thnx. btw as I am closing this issue, note that the version in RC_2_0 is still 2.0.5 when building through pip. I thought it would make sense for it to be bumped. Thnx, cheers.

This works for me:

$ pip wheel --no-use-pep517 --global-option=build_ext --global-option=--b2-args=cxxstd=20 .

--no-use-pep517 seems required to avoid the behavior you found. I guess pip assumes a non-legacy build-backend if pyproject.toml is present at all, which seems wrong.

it fails on all 32-bit archs and my guess is address-model being set at 64 (not verified yet). Is there any way for me to pass arguments to pip to enforce it to 32 ?

libtorrent/bindings/python/setup.py

Lines 378 to 379 in 07a5305

    
           bits = 64 if sys.maxsize > 2 ** 32 else 32 
        
           self._maybe_add_arg(f"address-model={bits}")

Does this test not work in crossenv? If not, how would we test the address model of the "invoking" python?

note that the version in RC_2_0 is still 2.0.5 when building through pip. I thought it would make sense for it to be bumped.

Arvid only bumps the version on a release.

I've experimented with setuptools_scm in the python bindings. It gets ugly since libtorrent's compile-time version number must be a 4-int tuple, and can't reasonably hold python (or other arbitrary) version strings, and this mismatch gets awkward at various places in the build system. This could be resolved, but there are higher priorities.

Thnx for your reply, in fact I actually implemented the --no-use-pep517 and made a few other adjustments to our build framework and now all is good.

As for the address-model it was missing the BOOST_BUILD_PATH so it can find the user-config.jam file used to build boost.

As for the version I thought it would already be showing 2.0.6 or 2.1.0 (perhaps with an rc). But that really is more estetic than anything.

My last remaining bug really is #6792 and is a major blocker for multiple archs.

	bits = 64 if sys.maxsize > 2 ** 32 else 32
	self._maybe_add_arg(f"address-model={bits}")

Build directly through pip?