hello-1 app won't run under QEMU

Question

hello-1 app won't run under QEMU

statusfailed opened this issue 7 years ago · comments

Error running app "hello-1" with QEMU.

I've tried to follow the tutorial documentation, but things go wrong when
I get to running qemu with the image I built.

In particular, when I run

qemu-system-i386 -nographic -m 512 -kernel images/kernel-ia32-pc99 -initrd images/hello-1-image-ia32-pc99

I get the following error:

qemu: fatal: Trying to execute code outside RAM or ROM at 0xe01207a0

Along with a dump of register state.

Have I done something wrong, or is the tutorial project broken? How should I
fix it?

Below are all the commands I used to build the project.

Steps to reproduce

Get repositories

mkdir sel4-tutorials-manifest
cd sel4-tutorials-manifest
repo init -u https://github.com/SEL4PROJ/sel4-tutorials-manifest -m sel4-tutorials.xml
repo sync

From this point on, I've run all commands within the sel4-tutorials-manifest directory.

Open the "howto" slides

I tried to open the tutorial PDF, but it doesn't seem to exist in the repository:

xpdf $(realpath projects/sel4-tutorials/docs/seL4-APILib-details.pdf)

I instead am using the version on the master branch of sel4-tutorials:

https://github.com/SEL4PROJ/sel4-tutorials/blob/master/docs/seL4-APILib-details.pdf

From the sel4-tutorials-manifest root directory, I run:

make ia32_hello-1_defconfig

Output looks fine:

[GEN] /my/home/dir/sel4-tutorials-manifest/Makefile
make -f tools/kbuild/Makefile.build obj=tools/kbuild/basic
#
# configuration written to .config
#

Now I run

make

This fails (as expected) with the error:

`undefined reference to 'main'`

So I fix this by adding a main function:

int main(void) {
  printf("Hello, World\n");
  return 0;
}

(I later copied this from the solution to make sure it was exactly correct and
I didn't have any subtle problem)

Running make again seems to work fine. The last lines of output are:

[STAGE] hello-1
[apps/hello-1] done.
[GEN_IMAGE] hello-1-image

Running QEMU

Now I run QEMU to test my image:

qemu-system-i386 -nographic -m 512 -kernel images/kernel-ia32-pc99 -initrd images/hello-1-image-ia32-pc99

I get the following error:

qemu: fatal: Trying to execute code outside RAM or ROM at 0xe01207a0

EAX=2badb002 EBX=00009500 ECX=0010003e EDX=00000511
ESI=00009500 EDI=2badb002 EBP=00000000 ESP=00100ff0
EIP=e01207a0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     000caa68 00000027
IDT=     00000000 000003ff
CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 
DR6=ffff0ff0 DR7=00000400
CCS=00000000 CCD=00950000 CCO=EFLAGS  
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
zsh: abort (core dumped)  qemu-system-i386 -nographic -m 512 -kernel images/kernel-ia32-pc99 -initrd

Peter Chubb · Answer 1 · Tue May 02 2017 06:48:47 GMT+0800 (China Standard Time)

Which version of qemu are you using? When I follow the steps you took, using GCC 5 and qemu 2.8.95 I get the expected results.

Paul Wilson · Answer 2 · Tue May 02 2017 15:11:50 GMT+0800 (China Standard Time)

I'm using default packages on Ubuntu 16.10, which have versions:

QEMU: QEMU emulator version 2.6.1 (Debian 1:2.6.1+dfsg-0ubuntu5.4)
gcc: gcc (Ubuntu 6.2.0-5ubuntu12) 6.2.0 20161005
arm-none-eabi-gcc: arm-none-eabi-gcc (15:4.9.3+svn231177-1) 4.9.3 20150529 (prerelease)

I'll try using a newer QEMU and report back

Paul Wilson · Answer 3 · Wed May 03 2017 06:18:01 GMT+0800 (China Standard Time)

I tried two versions of QEMU: the latest master branch from git, and 2.9.0. Neither worked.

Next, I tried upgrading Ubuntu to 17.04, because I had a problem building seL4test with ld 2.27 (the version shipping with Ubuntu 16.10)

I'm now running these versions:

QEMU emulator version 2.8.0
gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 20170406
GNU ld (GNU Binutils for Ubuntu) 2.28

Unfortunately this didn't help either, and I still get the same error message. I did try using the Virtualbox appliance however, and that worked as expected.

It seems unlikely that gcc/ld would produce code that QEMU can't run, so perhaps they are not producing the right kind of file.

Indeed, I did discover a difference in the output files, namely the kernel image produced by my box (not the .ova) is dynamically linked! I'm assuming this is very wrong.

For clarity, here's the output of file images/* for my box, and the test OVA:

My box:

hello-1-image-ia32-pc99: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
kernel-ia32-pc99:        ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, not stripped

Virtualbox appliance:

hello-1-image-ia32-pc99: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
kernel-ia32-pc99:        ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped

I'm guessing this has happened due to a change in the ld command line flags. I'm not sure how to fix this however - I would appreciate any tips!

I messed about with LDFLAGS in kernel/Makefile, but to no avail. I tried adding -Bstatic and changing -m,elf_i386 to -melf_i386 but neither worked.

I'm not sure how to tell GCC to static link the kernel ELF image, so I kinda guessed: I changed kernel/Makefile line 582 to add the -static flag. My version now reads:

$(Q)${CHANGED} $@ ${CC} ${LDFLAGS} -static -Wl,-T -Wl,linker.lds_pp \

Unfortunately, while my kernel image file now says "statically linked", it still gives the same error when I run it in QEMU.

Copying the kernel image from the virtual machine to my machine works, so it seems like it's only the kernel image that isn't building correctly (I can run the .ova-produced kernel image alongside the application code built on my machine)

Any thoughts on what I can try now?

Peter Chubb · Answer 4 · Wed May 03 2017 06:46:23 GMT+0800 (China Standard Time)

The problem appears to be gcc version 6 which is creating dynamically linked programs even when you request static ones.
I installed gcc 6 and got the same problem you're seeing. I'll see what can be done...

Peter Chubb · Answer 5 · Wed May 03 2017 07:33:30 GMT+0800 (China Standard Time)

Please try this patch.

project kernel/
diff --git a/Makefile b/Makefile
index 5e31f2cd..0deee694 100644
--- a/Makefile
+++ b/Makefile
@@ -389,7 +389,7 @@ WARNINGS = all error strict-prototypes missing-prototypes nested-externs \
 CFLAGS += --std=c99 -nostdlib -nostdinc -ffreestanding \
 	${WARNINGS:%=-W%} ${INCLUDES}
 CPPFLAGS += -nostdinc
-LDFLAGS += -nostdlib -nostdinc
+LDFLAGS += -nostdlib -nostdinc -static
 LDFLAGS += -Wl,--build-id=none
 ASFLAGS += ${INCLUDES}
 


project projects/tools/
diff --git a/common-tool/Makefile.flags b/common-tool/Makefile.flags
index 0ddb7ad..54aacbb 100644
--- a/common-tool/Makefile.flags
+++ b/common-tool/Makefile.flags
@@ -25,6 +25,9 @@ NK_ASFLAGS =
 NK_CCFLAGS =
 
 ldflags-y += $(call cc-ldoption, -Wl$(comma)--build-id=none)
+ldflags-y += $(call cc-ldoption, -Wl$(comma)--no-pie)
+
+ccflags-y += $(call cc-option, -fno-pie)
 
 ifeq ($(CONFIG_ARCH_X86),y)
 	ccflags-y += $(call cc-option,-fno-stack-protector)
diff --git a/common-tool/common.mk b/common-tool/common.mk
index 321bf1c..e17badc 100644
--- a/common-tool/common.mk
+++ b/common-tool/common.mk
@@ -113,8 +113,8 @@ ifeq (${CONFIG_USER_CFLAGS},)
 
     CFLAGS += $(NK_CFLAGS)
     CXXFLAGS += $(NK_CXXFLAGS)
-    CFLAGS += -fno-stack-protector
-    CXXFLAGS += -fno-stack-protector
+    CFLAGS += -fno-stack-protector $(call cc-option,-fno-pie)
+    CXXFLAGS += -fno-stack-protector $(call, cc-option,-fno-pie)
 else
     # Override the cflags with Kconfig-specified flags
 	CFLAGS += ${CONFIG_USER_CFLAGS}

Peter Chubb · Answer 6 · Wed May 03 2017 08:12:59 GMT+0800 (China Standard Time)

Looks like this patch (or an equivalent) is already in later versions of seL4.

Peter Chubb · Answer 7 · Wed May 03 2017 08:32:11 GMT+0800 (China Standard Time)

To try a later version of seL4, do repo sync -m sel4-tutorials-master.xml -- this will pick up the bleeding edge.

Paul Wilson · Answer 8 · Wed May 03 2017 15:33:48 GMT+0800 (China Standard Time)

Hooray, both your suggestions work!

With patch:

Starting node #0
Hello, World
Ignoring call to sys_exit_group

Without patch:

Hello, World

Either way: thank you very much for your help!

p.s.: I'm curious why the repo sync -m sel4-tutorials-master.xml is different to the initial mkdir ... cd ... repo init ... repo sync dance. What is repo sync -m ... doing that the original repo sync isn't?