How to build iceberg extension
TSP-wengle opened this issue · comments
I tried to compile the iceberg extension following the steps in the readme.md, but I ran into a problem, the problem is as follows:
mkdir -p build/release && \
cmake -DDUCKDB_EXTENSION_NAMES="iceberg" -DDUCKDB_EXTENSION_ICEBERG_PATH="/home/gpadmin/duckdb_iceberg/" -DDUCKDB_EXTENSION_ICEBERG_SHOULD_LINK=1 -DDUCKDB_EXTENSION_ICEBERG_INCLUDE_PATH="/home/gpadmin/duckdb_iceberg/src/include" -DEXTENSION_STATIC_BUILD=1 -DCMAKE_BUILD_TYPE=Release -DEXTENSION_STATIC_BUILD=1 -DBUILD_EXTENSIONS="httpfs" -DOSX_BUILD_ARCH= -DVCPKG_MANIFEST_DIR='/home/gpadmin/duckdb_iceberg/' -DVCPKG_BUILD=1 -DCMAKE_TOOLCHAIN_FILE='/home/gpadmin/vcpkg/scripts/buildsystems/vcpkg.cmake' -S ./duckdb/ -B build/release && \
cmake --build build/release --config Release
-- Running vcpkg install
warning: Embedding `vcpkg-configuration` in a manifest file is an EXPERIMENTAL feature.
Detecting compiler hash for triplet x64-linux...
The following packages will be built and installed:
avro-cpp[core,snappy]:x64-linux -> 2022-11-07 -- /home/gpadmin/duckdb_iceberg/./vcpkg_ports/avro-cpp
* boost-algorithm:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-algorithm/6a184d48bcf1a89a12f66255625340ce9cbfda92
* boost-align:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-align/6dcfc6f69223f8fffff4c82db31be7495f17062d
* boost-any:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-any/798ddb57d8d3bb7a77e424cd5d60f88cc4e6bee7
* boost-array:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-array/8503faaf38485a913d4daf3efd2076b67bebdc0b
* boost-assert:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-assert/f07d2980139502309087821ca9a78a9a43a766ec
* boost-atomic:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-atomic/ed19c1a3b7f57d00917b260e35cd662b28c1f6eb
* boost-bind:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-bind/1d9372948d3c0fe1da8965e76ded151513f16a52
* boost-build:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-build/9d4a35930b7414144551dc9c2e00b31bcb0ca8b2
* boost-concept-check:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-concept-check/c266d37e02d494dcf4c3623bae07775c2a7baa45
* boost-config:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-config/af2f41ef14357f44466182d783e8cb218d677b1f
* boost-container:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-container/54ac399bdf18e9c6ab8c9df4d81df4bd86a547d9
* boost-container-hash:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-container-hash/c5bde693ed8971a4e0ba6e22854386d4054898a8
* boost-conversion:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-conversion/e99972dc970d3a2229996b240d5dc4362204b6dc
* boost-core:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-core/f406b6bfcf32cb14d17d1df5e57a1162a2f2d703
* boost-crc:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-crc/7146ce31ceb0cdce252cdf556502fbd358eb30d5
* boost-describe:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-describe/631c0781e5719b44d906954a57f3a9a42bb37e8a
* boost-detail:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-detail/ea40bd02426af4c5ce99995762c452e3a70b5189
* boost-dynamic-bitset:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-dynamic-bitset/d350f9d70da182a7abddd0cadc649394fb144113
* boost-exception:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-exception/ef4cfcde16100435562ccebf90edca8cfc4e204c
* boost-filesystem:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-filesystem/9e51e733f0ddb9e287a6ee03bf9cbb1f3f65f081
* boost-format:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-format/9562adeb23d81f0bfb89adb5c920ead09b302ec5
* boost-function:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-function/dce4e73b054f0b611a8d1e0f9efc9dd8cd79071f
* boost-function-types:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-function-types/1da109d6c53f82fdac9bdf6efa17af3d47dacb54
* boost-functional:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-functional/3c428ee1b1238a5fbf337fd549d3b3c3afc17930
* boost-fusion:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-fusion/0c32122b9244e041424874f6657451169ab9522d
* boost-integer:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-integer/ed95b549c8719f65d4599fb6a612db94043c033f
* boost-intrusive:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-intrusive/c6ef0a6523433599c5a6e0bebedd4b2caa6b56cf
* boost-io:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-io/21da7534c21a1dfd1ff5833e330837ad8fa26341
* boost-iostreams[bzip2,core,lzma,zlib,zstd]:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-iostreams/9a39f5835d29b23ad73b036797cb4f6f000e83c2
* boost-iterator:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-iterator/06e57442234e51fd436cc31ac285319baaf83e26
* boost-lexical-cast:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-lexical-cast/54b82af182eb01789c463148630e7e897ae3f3b7
* boost-math:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-math/61b0ddfd8a1a5e18f97584b57143de200770bd56
* boost-modular-build-helper:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-modular-build-helper/5c1a238cc20c885e785bc05a36b974deb5e02db7
* boost-move:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-move/1ce5f627ac0c30d041b9f2981fd3c11e4b504825
* boost-mp11:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-mp11/80e104d697b7ff7a283c5f7ee83ae25fe1192bd7
* boost-mpl:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-mpl/e08d5a2eb5d5edfc5a62469dbaa283cf56570d07
* boost-numeric-conversion:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-numeric-conversion/97842146c6a87912141dd1d4654e1576d12e5cef
* boost-optional:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-optional/100ff3094e97d23516718679d59d9f86821074b8
* boost-predef:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-predef/3a0c5194d7a4d8825efff8d67e0c75cae0b058d7
* boost-preprocessor:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-preprocessor/85819cdd004323b0afe1a361b9e05fdb55ab6838
* boost-program-options:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-program-options/c39f818a3734e996eb777117eb8d1492d7b1b26f
* boost-random:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-random/0c97951b0f38d6c9b469b367ac136249e1fa23fd
* boost-range:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-range/fc4c6c85df6efb4afe27a53499abdcd17da71a69
* boost-regex:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-regex/d31af2d45fe496fd2d89b734027e72c92d23f1d1
* boost-smart-ptr:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-smart-ptr/a61a6534966bd4a70b051538a287de38be4345ba
* boost-static-assert:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-static-assert/e6e55a999a301eb62d7c4da853a6aeb296713943
* boost-system:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-system/48fb125e52a85de87f4ff03475742d79f84b42cb
* boost-throw-exception:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-throw-exception/c51c9fa76f925c5824688c6ca9f451716373b5d5
* boost-tokenizer:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-tokenizer/4a53c730b20cacfbd8e6bbbd8faaeee0720c9fc1
* boost-tuple:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-tuple/c9173d5394c4b31fdf698b156b542c29031d03fe
* boost-type-index:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-type-index/d02bcbc9c4ee9a4b27e90ab03010bea7b6c70eb3
* boost-type-traits:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-type-traits/34773fec432e120d3f2d9dd482461da7c36ef897
* boost-typeof:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-typeof/ab6e440573d5f3ac964c50b3efc24055500a772f
* boost-uninstall:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-uninstall/d96807810fa612b038f9268c8df545d421fda1a6
* boost-unordered:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-unordered/ee2a3d19df6c91036cb772b4747bc7244307cda1
* boost-utility:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-utility/b94486558c58f29bb5165d952196d448719caae5
* boost-variant2:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-variant2/3acfc977da209b860fd7cf1365180af919be0eeb
* boost-vcpkg-helpers:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-vcpkg-helpers/f99477ea8dcfb463d2f0a2cce3c60914c2e5adb4
* boost-winapi:x64-linux -> 1.82.0#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-winapi/db66242e796c29abd3622cf41d506bee622b4d10
* bzip2[core,tool]:x64-linux -> 1.0.8#4 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/bzip2/6165360d15e6de08dff3a5f079d51e69908cc55d
* liblzma:x64-linux -> 5.4.1#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/liblzma/9a3ebbf75e1c3adb76d7e71717c1f03bb291036a
openssl:x64-linux -> 3.0.8 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/openssl/9cd36489fc65db87c4cd94ac606abd05c1db8f2d
* snappy:x64-linux -> 1.1.9#5 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/snappy/a97f6aac039dc2b5e6fdac753d66b94cfb408d30
* vcpkg-cmake:x64-linux -> 2022-12-22 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/vcpkg-cmake/1913f86defd2140d0a6751be2d51952e4090efa4
* vcpkg-cmake-config:x64-linux -> 2022-02-06#1 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/vcpkg-cmake-config/8d54cc4f487d51b655abec5f9c9c3f86ca83311f
* vcpkg-cmake-get-vars:x64-linux -> 2023-03-02 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/vcpkg-cmake-get-vars/2e624c2cf12a97a7a802e31ff1d28b9fa6ba9bde
* zlib:x64-linux -> 1.2.13 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/zlib/ad5a49006f73b45b715299515f31164131b51982
* zstd:x64-linux -> 1.5.4#2 -- /home/gpadmin/vcpkg/buildtrees/versioning_/versions/zstd/fcc4fe2738d903bc347d20d2d4c836699e273509
Additional packages (*) will be modified to complete this operation.
Restored 24 package(s) from /home/gpadmin/.cache/vcpkg/archives in 55.1 ms. Use --debug to see more details.
Installing 1/69 vcpkg-cmake-config:x64-linux...
Elapsed time to handle vcpkg-cmake-config:x64-linux: 333 us
Installing 2/69 vcpkg-cmake:x64-linux...
Elapsed time to handle vcpkg-cmake:x64-linux: 249 us
Installing 3/69 snappy:x64-linux...
Elapsed time to handle snappy:x64-linux: 484 us
Installing 4/69 zlib:x64-linux...
Elapsed time to handle zlib:x64-linux: 369 us
Installing 5/69 boost-uninstall:x64-linux...
Elapsed time to handle boost-uninstall:x64-linux: 262 us
Installing 6/69 boost-vcpkg-helpers:x64-linux...
Elapsed time to handle boost-vcpkg-helpers:x64-linux: 283 us
Installing 7/69 boost-config:x64-linux...
Elapsed time to handle boost-config:x64-linux: 2.84 ms
Installing 8/69 boost-static-assert:x64-linux...
Elapsed time to handle boost-static-assert:x64-linux: 395 us
Installing 9/69 boost-type-traits:x64-linux...
Elapsed time to handle boost-type-traits:x64-linux: 3.48 ms
Installing 10/69 boost-assert:x64-linux...
Elapsed time to handle boost-assert:x64-linux: 580 us
Installing 11/69 boost-throw-exception:x64-linux...
Elapsed time to handle boost-throw-exception:x64-linux: 526 us
Installing 12/69 boost-core:x64-linux...
Elapsed time to handle boost-core:x64-linux: 1.39 ms
Installing 13/69 boost-tuple:x64-linux...
Elapsed time to handle boost-tuple:x64-linux: 493 us
Installing 14/69 boost-preprocessor:x64-linux...
Elapsed time to handle boost-preprocessor:x64-linux: 4.41 ms
Installing 15/69 boost-io:x64-linux...
Elapsed time to handle boost-io:x64-linux: 667 us
Installing 16/69 boost-utility:x64-linux...
Elapsed time to handle boost-utility:x64-linux: 837 us
Installing 17/69 boost-predef:x64-linux...
Elapsed time to handle boost-predef:x64-linux: 2.06 ms
Installing 18/69 boost-winapi:x64-linux...
Elapsed time to handle boost-winapi:x64-linux: 1.56 ms
Installing 19/69 boost-mp11:x64-linux...
Elapsed time to handle boost-mp11:x64-linux: 863 us
Installing 20/69 boost-variant2:x64-linux...
Elapsed time to handle boost-variant2:x64-linux: 668 us
Installing 21/69 vcpkg-cmake-get-vars:x64-linux...
Elapsed time to handle vcpkg-cmake-get-vars:x64-linux: 685 us
Installing 22/69 boost-modular-build-helper:x64-linux...
Elapsed time to handle boost-modular-build-helper:x64-linux: 704 us
Installing 23/69 boost-build:x64-linux...
Elapsed time to handle boost-build:x64-linux: 9.99 ms
Installing 24/69 boost-system:x64-linux...
Building boost-system:x64-linux...
-- Installing port from location: /home/gpadmin/vcpkg/buildtrees/versioning_/versions/boost-system/48fb125e52a85de87f4ff03475742d79f84b42cb
-- Downloading https://github.com/boostorg/system/archive/boost-1.82.0.tar.gz -> boostorg-system-boost-1.82.0.tar.gz...
-- Cleaning sources at /home/gpadmin/vcpkg/buildtrees/boost-system/src/ost-1.82.0-6fabddb1d7.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source /home/gpadmin/vcpkg/downloads/boostorg-system-boost-1.82.0.tar.gz
-- Using source at /home/gpadmin/vcpkg/buildtrees/boost-system/src/ost-1.82.0-6fabddb1d7.clean
-- Getting CMake variables for x64-linux
-- Configuring x64-linux
-- Building x64-linux-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:134 (message):
Command failed: /home/gpadmin/vcpkg/downloads/tools/cmake-3.27.1-linux/cmake-3.27.1-linux-x86_64/bin/cmake --build . --config Debug --target install -- -v -j17
Working Directory: /home/gpadmin/vcpkg/buildtrees/boost-system/x64-linux-dbg
See logs for more information:
/home/gpadmin/vcpkg/buildtrees/boost-system/install-x64-linux-dbg-out.log
Call Stack (most recent call first):
/home/gpadmin/duckdb_iceberg/build/release/vcpkg_installed/x64-linux/share/vcpkg-cmake/vcpkg_cmake_build.cmake:74 (vcpkg_execute_build_process)
/home/gpadmin/duckdb_iceberg/build/release/vcpkg_installed/x64-linux/share/vcpkg-cmake/vcpkg_cmake_install.cmake:16 (vcpkg_cmake_build)
/home/gpadmin/duckdb_iceberg/build/release/vcpkg_installed/x64-linux/share/boost-build/boost-modular-build.cmake:140 (vcpkg_cmake_install)
buildtrees/versioning_/versions/boost-system/48fb125e52a85de87f4ff03475742d79f84b42cb/portfile.cmake:14 (boost_modular_build)
scripts/ports.cmake:168 (include)
error: building boost-system:x64-linux failed with: BUILD_FAILED
Elapsed time to handle boost-system:x64-linux: 3.1 s
Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+boost-system
You can submit a new issue at:
https://github.com/microsoft/vcpkg/issues/new?title=[boost-system]+Build+error&body=Copy+issue+body+from+%2Fhome%2Fgpadmin%2Fduckdb_iceberg%2Fbuild%2Frelease%2Fvcpkg_installed%2Fvcpkg%2Fissue_body.md
-- Running vcpkg install - failed
CMake Error at /home/gpadmin/vcpkg/scripts/buildsystems/vcpkg.cmake:899 (message):
vcpkg install failed. See logs for more information:
/home/gpadmin/duckdb_iceberg/build/release/vcpkg-manifest-install.log
Call Stack (most recent call first):
/usr/local/share/cmake-3.18/Modules/CMakeDetermineSystem.cmake:93 (include)
CMakeLists.txt:19 (project)
CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
make: *** [Makefile:52: release] Error 1
hey @TSP-wengle the VCPKG version used by the iceberg extension is a bit behind, if you checkout your vcpkg to 501db0f17ef6df184fcdbfbe0f87cde2313b6ab1 it should work.
This is something that should be fixed for next duckdb release