Mac OS X compilation error
halsten opened this issue · comments
Hi,
I have cloned rVMI and was trying to compile on a Mac OSX, but came across a couple of problems. I am posting it in rVMI-QEMU and not rVMI is because the main compilation problems exist in this module specifically. As a suggestion first I would consider doing a lookup on required libraries on the system first before failing the building process completely. For example, it's a good idea to check first for the existence for glib and gthread before doing any building. Here is some of the information about my system and build tool-chain:
$: uname -a
Darwin halsten 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64
$: clang --version
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$: gcc --version
gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$: g++ --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$: python --version
Python 2.7.10
I will document the problems that I faced during the compilation of the rVMI-QEMU sub-module. So following the instructions on the README file, I did the following.
$: ./configure --target-list=x86_64-softmmu
Install prefix /usr/local
BIOS directory /usr/local/share/qemu
binary directory /usr/local/bin
library directory /usr/local/lib
module directory /usr/local/lib/qemu
libexec directory /usr/local/libexec
include directory /usr/local/include
config directory /usr/local/etc
local state directory /usr/local/var
Manual directory /usr/local/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path /Users/halsten/Documents/rvmi/rvmi-qemu
C compiler cc
Host C compiler cc
C++ compiler c++
Objective-C compiler clang
ARFLAGS rv
CFLAGS -O2 -g
QEMU_CFLAGS -I$(SRC_PATH)/pixman/pixman -I$(BUILD_DIR)/pixman/pixman -I$(SRC_PATH)/dtc/libfdt -DHAS_LIBSSH2_SFTP_FSYNC -D_REENTRANT -I/usr/local/Cellar/glib/2.54.0/include/glib-2.0 -I/usr/local/Cellar/glib/2.54.0/lib/glib-2.0/include -I/usr/local/opt/gettext/include -I/usr/local/Cellar/pcre/8.41/include -m64 -mcx16 -DOS_OBJECT_USE_OBJC=0 -arch x86_64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wno-string-plus-int -Wno-initializer-overrides -Wendif-labels -Wno-shift-negative-value -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-definition -Wtype-limits -fstack-protector-strong -I/usr/local/Cellar/libpng/1.6.32/include/libpng16 -I/usr/local/include -I/usr/local/Cellar/libusb/1.0.21/include/libusb-1.0
LDFLAGS -m64 -framework CoreFoundation -framework IOKit -arch x86_64 -g
make make
install install
python python -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu
tcg debug enabled no
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
Cocoa support yes
pixman internal
SDL support no
GTK support no
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support no
GNUTLS rnd no
libgcrypt no
libgcrypt kdf no
nettle no
nettle kdf no
libtasn1 no
curses support no
virgl support no
curl support yes
mingw32 support no
Audio drivers coreaudio
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
VNC support yes
VNC SASL support yes
VNC JPEG support yes
VNC PNG support yes
xen support no
brlapi support no
bluez support no
Documentation yes
PIE no
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support no
Install blobs yes
KVM support no
COLO support yes
RDMA support no
TCG interpreter no
fdt support yes
preadv support no
fdatasync no
madvise yes
posix_madvise yes
libcap-ng support no
vhost-net support no
vhost-scsi support no
vhost-vsock support no
Trace backends log
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb yes
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend sigaltstack
coroutine pool yes
debug stack usage no
GlusterFS support no
Archipelago support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support yes
TPM passthrough no
QOM debugging yes
lzo support no
snappy support yes
bzip2 support yes
NUMA host support no
tcmalloc support no
jemalloc support no
avx2 optimization no
replication support yes
$: make
GEN x86_64-softmmu/config-devices.mak.tmp
GEN x86_64-softmmu/config-devices.mak
GEN config-all-devices.mak
GEN config-host.h
/Applications/Xcode.app/Contents/Developer/usr/bin/make all-recursive
Making all in pixman
I am cutting down this part as there were a lot of warnings and it wasn't problematic and will only list the part where the error occurs.
pixman-mmx.c:100:20: error: constraint 'K' expects an integer constant expression
: "y" (__A), "K" (__N)
^~~
6 warnings and 1 error generated.
make[3]: *** [libpixman_mmx_la-pixman-mmx.lo] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all] Error 2
make: *** [subdir-pixman] Error 2
If there is any vital information that was necessary, but I missed, please do let me know and I will provide it.
--halsten
Unfortunately, I cannot repro this. However, we can try to avoid having to compile pixman. This will add some additional dependencies, like pixman dev package, etc. Please try the following steps:
$ git clone https://github.com/fireeye/rvmi-qemu.git rvmi-qemu
$ cd rvmi-qemu
$ ./configure --target-list=x86_64-softmmu
$ make
This clone without the qemu submodules. Please let me know if this works as I will update the install script if this fixes your issue.
Hi,
What is the version of clang version that you used?
I already did what you just suggested before I open this issue, I even explicitly explained it above that this is what I did in the rEVMI-QEMU sub-module. The same output is absolutely guaranteed to be the same.
To save other people time in case someone falls in this problem, I will compile all the necessary information regarding the problem and solution in here for future references. It is a bit lengthy, but I will try to shorten it down as much as possible.
I pointed out that the error is in pixman itself. And this seems to be a legit problem with the source itself. Obviously, it's directly related to pixman, and not a problem with the compiler at all. We can argue about which part is the guilty one, but that would be a waste of time. Basically, you are using pixman v0.12 which dates back to 2008. The root of the problem with using a clang compiler that dates after revision r225244 will always fail to compile. This goes back to a couple of points:
-
The test performed in
configure.acdoes not match the code with sufficient accuracy. In particular, the constant is hardcoded in theconfigure.actest, while in the code it is only available after inlining. Ideally, the configure tests should try doing exactly the same things as the actual code. Realistically it's difficult to argue on whether the code is valid or not, to be honest. This is because the code is relying on the compiler-specific extensions there. The__always_inline__attribute implementation is not a part of the official C standard. GCC however just implements it because this is a useful feature. Whilst, clang is trying its best to be GCC compatible. -
Initially, GCC has constraints on inlined assembly.
Here's the code with the problem:
extern __inline __m64 __attribute__((__gnu_inline__, __always_inline__, __artificial__))
_mm_shuffle_pi16 (__m64 __A, int8_t const __N)
{
__m64 ret;
asm ("pshufw %2, %1, %0\n\t"
: "=y" (ret)
: "y" (__A), "K" (__N)
);
return ret;
}The code has a const integer function argument, whereas clang expects a constexpr, which is a higher bar. What happened before was that the constraint was not checked, but once instruction selection rolled around, everything had been inlined and constant-folded, and there was an immediate value to select, so everything worked out.
The original problem is that certain compilers, in which GCC was amongst, did not support some intrinsics when compiling MMX code like _mm_movemask_pi8, _mm_mulhi_pu16 or _mm_shuffle_pi16 for that matter. These instructions were not available as part of the original MMX, but only got introduced later with AMD Extended 3DNow! and Intel SSE1.
The compilers are not perfect and sometimes fail our expectations. At the end there are 2 ways to solve this problem:
- Use a clang compiler that is prior to revision
r225244. - Patching
pixman-mmx.csource file to circumvent this issue when using a more recent version of clang compiler, more specifically after revisionr225244.
Finally, I resolved the issue and the issue can be considered closed.
--halsten