Wrong date?

Question

Wrong date?

HimbeersaftLP opened this issue 4 years ago · comments

Using Raspbian 32bit on a Pi 4, the date inside the docker does not match the system date.

pi@rpi:~ $ docker run --rm --privileged aptman/qus -s -- -p i386
cat ./qemu-binfmt-conf.sh | sh -s -- --path=/qus/bin -p i386 --suffix -static
Setting /qus/bin/qemu-i386-static as binfmt interpreter for i386

pi@rpi:~ $ docker run --rm -it debian /bin/bash
Unable to find image 'debian:latest' locally
latest: Pulling from library/debian
5c0fdcca2cbb: Pull complete
Digest: sha256:8414aa82208bc4c2761dc149df67e25c6b8a9380e5d8c4e7b5c84ca2d04bb244
Status: Downloaded newer image for debian:latest
root@1b8f5ee7c679:/# date
Sun Nov  1 19:48:44 UTC 2020
root@1b8f5ee7c679:/# exit

pi@rpi:~ $ docker run --rm -it i386/debian /bin/bash
Unable to find image 'i386/debian:latest' locally
latest: Pulling from i386/debian
c303ed7c76e7: Pull complete
Digest: sha256:c0a5cdcd732dba815f37c0942c668b299b9a80cef949e1e4e6d3401faa5b8d8f
Status: Downloaded newer image for i386/debian:latest
root@6bf30e989f54:/# date
Sat Oct 30 08:53:36 UTC 2004

Unai Martinez-Corral · Answer 1 · Tue Nov 03 2020 00:00:24 GMT+0800 (China Standard Time)

Interesting... Unfortunately, I have no clue about what it could be. I just tried arm32v7/debian on Windows 10 (Docker Desktop includes QEMU for ARM targets), and results are correct:

# docker run --rm -it debian date
Mon Nov  2 15:58:42 UTC 2020

# docker run --rm -it i386/debian date
Mon Nov  2 15:58:50 UTC 2020

# docker run --rm -it arm32v7/debian date
Mon Nov  2 15:58:56 UTC 2020

# docker run --rm -it arm64v8/debian date
Mon Nov  2 15:59:11 UTC 2020

Is the mismatch you found consistent using different images?

Himbeer · Answer 2 · Tue Nov 03 2020 01:40:10 GMT+0800 (China Standard Time)

Yes, Alpine shows the same problem:

pi@rpi:~ $ docker run --rm -it alpine /bin/sh
/ # date
Mon Nov  2 17:36:59 UTC 2020
/ # pi@rpi:~ $ docker run --rm -it i386/alpine /bin/sh
/ # date
Mon Jan 19 12:19:25 UTC 2004

Here is a thing that I noticed that might help find the issue?

/ # wget -O - http://google.com
Connecting to google.com (216.58.214.206:80)
Connecting to www.google.com (216.58.214.228:80)
writing to stdout
wget: clock_gettime(MONOTONIC) failed

Struan Bartlett · Answer 3 · Sat Nov 07 2020 22:09:52 GMT+0800 (China Standard Time)

I have the same issue. I've noticed that the date inside the container is different according to the distribution launched, and it doesn't change over time.

root@rpi4-4:/home/pi# docker run --rm -it amd64/debian bash -c 'while true; do date; sleep 1; done'
Thu Jan  1 00:00:00 UTC 1970
Thu Jan  1 00:00:00 UTC 1970
Thu Jan  1 00:00:00 UTC 1970
^C

root@rpi4-4:/home/pi# docker run --rm -it i386/debian bash -c 'while true; do date; sleep 1; done'
Sat Oct 30 08:53:36 UTC 2004
Sat Oct 30 08:53:36 UTC 2004
Sat Oct 30 08:53:36 UTC 2004

All sorts of things, from ping to apt, fail to run properly because of this problem.

Unai Martinez-Corral · Answer 4 · Wed Nov 11 2020 09:24:13 GMT+0800 (China Standard Time)

Ref https://gitlab.alpinelinux.org/alpine/aports/-/issues/12091

Unai Martinez-Corral · Answer 5 · Tue Jan 12 2021 13:48:26 GMT+0800 (China Standard Time)

@HimbeersaftLP , @struanb , did you both use Raspbian or some other host OS?

Himbeer · Answer 6 · Tue Jan 12 2021 17:34:03 GMT+0800 (China Standard Time)

@umarcor Raspbian 32bit on a Pi 4

pi@rpi:~ $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

Marcin Zepp · Answer 7 · Thu Jan 14 2021 23:12:33 GMT+0800 (China Standard Time)

@umarcor

Background (maybe off-topic)

I'm trying to run the x86_64 deno docker container. On Raspbian I was compiling QEMU with -mcpu=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard CFLAGS. I got a working x86_64 VM with alpine. But it is too slow to run docker, and it ends with failed to start containerd: timeout waiting for containerd to start. In my opinion, it's because of 15 seconds timeout in this file. I will try to rebuild Docker with other timeouts and check if it's working. But my solution is unprofessional and inefficient.

Foreground

With the use of this repository, I end up with:

pi@raspberrypi:~ $ sudo docker run -it --rm hayd/deno
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', library/std/src/sys/unix/time.rs:361:62
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I have no idea what's going on, but maybe it will help someone:

pi@raspberrypi:~ $ lsb_release -a
No LSB modules are available.
Distributor ID:	Raspbian
Description:	Raspbian GNU/Linux 10 (buster)
Release:	10
Codename:	buster
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.4.83-v7+ #1379 SMP Mon Dec 14 13:08:57 GMT 2020 armv7l GNU/Linux
pi@raspberrypi:~ $ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 1
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 2
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 3
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

Hardware	: BCM2835
Revision	: a22082
Serial		: 000000008b1d9529
Model		: Raspberry Pi 3 Model B Rev 1.2
pi@raspberrypi:~ $ sudo apt-get remove docker docker-engine docker.io containerd runc
pi@raspberrypi:~ $ sudo apt-get update
pi@raspberrypi:~ $ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
pi@raspberrypi:~ $ echo "deb [arch=armhf] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list
pi@raspberrypi:~ $ sudo apt update
pi@raspberrypi:~ $ sudo apt install -y --no-install-recommends docker-ce cgroupfs-mount
pi@raspberrypi:~ $ sudo systemctl start docker
pi@raspberrypi:~ $ sudo docker run --rm --privileged aptman/qus -s -- -p x86_64
cat ./qemu-binfmt-conf.sh | sh -s -- --path=/qus/bin -p x86_64 --suffix -static
Setting /qus/bin/qemu-x86_64-static as binfmt interpreter for x86_64
pi@raspberrypi:~ $ sudo docker run --rm -it amd64/ubuntu bash
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
root@69f75e516a35:/# uname -a
Linux 69f75e516a35 5.4.83-v7+ #1379 SMP Mon Dec 14 13:08:57 GMT 2020 x86_64 x86_64 x86_64 GNU/Linux
root@69f75e516a35:/# date
Sun Jan 11 12:55:12 UTC 2004
root@69f75e516a35:/# date # after 10 seconds
Sun Jan 11 12:55:12 UTC 2004
root@69f75e516a35:/# date; sleep 10; date
Sun Jan 11 12:55:12 UTC 2004 <it was waiting 10 seconds>
Sun Jan 11 12:55:12 UTC 2004
root@69f75e516a35:/# ps -aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.6  1.3  63144 12280 pts/0    Ssl   2021   0:01 /qus/bin/qemu-x86_64-static /usr/bin/bash
root        42  0.0  1.0  64636  9196 ?        Rl+   2021   0:00 /usr/bin/ps -aux
root@69f75e516a35:/# date -s '21-01-14 14:48:00'
/usr/bin/date: cannot set date: Operation not permitted
Thu Jan 14 14:48:00 UTC 2021
root@69f75e516a35:/# hwclock --verbose
hwclock from util-linux 2.34
System Time: 0.000000
Trying to open: /dev/rtc0
Trying to open: /dev/rtc
Trying to open: /dev/misc/rtc
No usable clock interface found.
hwclock: Cannot access the Hardware Clock via any known method.
root@69f75e516a35:/# cat /sys/devices/system/clocksource/clocksource0/available_clocksource 
arch_sys_counter 
root@69f75e516a35:/# cat /sys/devices/system/clocksource/clocksource0/current_clocksource 
arch_sys_counter
root@69f75e516a35:/# exit
pi@raspberrypi:~ $ hwclock --verbose
hwclock from util-linux 2.33.1
System Time: 1610633977.502940
Trying to open: /dev/rtc0
Trying to open: /dev/rtc
Trying to open: /dev/misc/rtc
No usable clock interface found.
hwclock: Cannot access the Hardware Clock via any known method.
pi@raspberrypi:~ $ hwclock --verbose
hwclock from util-linux 2.33.1
System Time: 1610633981.035373
Trying to open: /dev/rtc0
Trying to open: /dev/rtc
Trying to open: /dev/misc/rtc
No usable clock interface found.
hwclock: Cannot access the Hardware Clock via any known method.
pi@raspberrypi:~ $ cat /sys/devices/system/clocksource/clocksource0/available_clocksource
arch_sys_counter 
pi@raspberrypi:~ $ sudo date -s "21-01-14 15:22:00"
czw, 14 sty 2021, 15:22:00 CET
pi@raspberrypi:~ $ sudo hwclock --verbose
hwclock from util-linux 2.33.1
System Time: 1610634106.140037
Trying to open: /dev/rtc0
Trying to open: /dev/rtc
Trying to open: /dev/misc/rtc
No usable clock interface found.
hwclock: Cannot access the Hardware Clock via any known method.
pi@raspberrypi:~ $ date
czw, 14 sty 2021, 15:22:01 CET
pi@raspberrypi:~ $ sudo date -s "21-01-14 15:24:00"
czw, 14 sty 2021, 15:24:00 CET
pi@raspberrypi:~ $ date
czw, 14 sty 2021, 15:22:21 CET

I was trying to do something with timedatectl, but in my view, it's impossible due to no systemd running as PID1.

Maybe the problem is with Raspberry Pi kernel? Or the raspi-config script can be useful?

Unai Martinez-Corral · Answer 8 · Fri Jan 15 2021 04:38:27 GMT+0800 (China Standard Time)

I'm trying to run the x86_64 deno docker container. On Raspbian I was compiling QEMU with -mcpu=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard CFLAGS. I got a working x86_64 VM with alpine. But it is too slow to run docker, and it ends with failed to start containerd: timeout waiting for containerd to start. In my opinion, it's because of 15 seconds timeout in this file. I will try to rebuild Docker with other timeouts and check if it's working. But my solution is unprofessional and inefficient.

I'm not sure to understand what are you trying to do here. Did you build QEMU, start a QEMU VM (qemu-system) and try building, installing and running Docker in there? Although that might theoretically work, that is very far from the expected use case in this repo/project. See https://dbhi.github.io/qus/#context-docker-and-qemu.

The concept of qus is that you a start with some host where you can run regular native containers without issues. Hence, on Rpi you would install docker through apt or through get.docker.com, and run either arm32v7 or arm64v8 containers. Then, should you want to run foreign containers too, you can use qus, which registers and loads qemu-user-static binaries into memory. Then, you will keep using the same kernel and the same docker engine (both armhf/aarch64), but foreign instructions will be passed by the kernel to the qemu-user binary for translation.

Installing QEMU otherhow, and/or building/installing Docker with custom options is out of the scope of this project, because that is precisely what we are trying to avoid. Calling the aptman/qus container works on Windows or Linux, with Podman or Docker, "regardless" of the host architecture. Introducing custom build procedures would conflict with that approach. Naturally, constraining qus to that scope does involve limitations regarding usability in non-trivial applications.

Don't take me wrong, I'm ok with you sharing your experiments in this regard, as it is always interesting to be aware of which other approaches do work or which don't. However, I think it'd be better located in a separated issue.

With the use of this repository, I end up with:

pi@raspberrypi:~ $ sudo docker run -it --rm hayd/deno
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', library/std/src/sys/unix/time.rs:361:62
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I don't think this is the same issue. Other users could succesfully run foreign images and execute commands inside. The problem is that commands/tools relying on time had issues.

Honestly, I did not see that error before. Where is that Rust code from? AFAIK, neither Docker nor QEMU are written in Rust. Is it possible that the tool inside the container is written in Rust and that one is crashing instantly? If that is the case, I would suggest running the tool on the host first. That is, get an x86_61/amd64 build/binary/tarball and try running it on the RPi after executing aptman/qus. As a matter of fact, the interpreters registered by qus are useful for either containers or host applications.

I was trying to do something with timedatectl, but in my view, it's impossible due to no systemd running as PID1.

Maybe the problem is with Raspberry Pi kernel? Or the raspi-config script can be useful?

My perception is that Raspbian is partially broken in several ways. On the one hand, it's a 32 bit OS, so that it can be executed on any RPi (the earliest versions don't support 64 bit). However, arm64v8 containers can be successfully executed on Raspbian, using the default docker installation. I would expect that to have corner cases, and handling time might be one of those. On the other hand, AFAIK raspbian is not free; some software components are included as binary blobs. In these embedded devices, interacting with peripherals is typically non-standard and finding bugs in those blobs can be really hard. I guess that's why other distributions (Fedora, Ubuntu, Arch, etc.) did not (announce) support for RPi until several years later.

Nevertheless, other users have reported successfully running x86/amd64 containers on RPi: #2 (comment). @jrcichra, would you kindly let us know whether you used Raspbian or some other OS?

Marcin Zepp · Answer 9 · Fri Jan 15 2021 07:16:34 GMT+0800 (China Standard Time)

Although that might theoretically work, that is very far from the expected use case in this repo/project.

Yes, I know, I've written it here:

But my solution is unprofessional and inefficient.

Also, I was trying to do it with the qus project, and I failed. So I began to try it on my own with full QEMU emulation. But it is so slow that the Docker can't start. And it's all being my background. I wanted to say what I want to achieve and how I'm trying to do it now. It's all off-topic, but for now, I can tell that the problem with no-ticking time does not occur on RPi with full-emulated QEMU system. I don't know if it's useful, but I think it's worth mentioning that the problem is not in QEMU itself. But maybe it's in qemu-user. I don't know.

However, I think it'd be better located in a separated issue.

Do you really want me to open the issue with "the another approach"? 😄
I have started writing my process of gaining experience, but I was thinking only about one gist. But I can add it here as another issue also if you want.

Honestly, I did not see that error before. Where is that Rust code from?

The deno is written i. e. in Rust. And it's an issue with time, see message: "Operation not permitted" }', library/std/src/sys/unix/time.rs:361:62 like in

$ date -s '21-01-14 14:48:00'
/usr/bin/date: cannot set date: Operation not permitted

So it's container-dependent and will only occur when someone run Rust, but it's related to no-ticking time in a qus container on RPi.

I would suggest running the tool on the host first.

It is not possible due to denoland/deno#2295. There is no support for my host CPU, and I think there is no hope for it for the next two months. But on my PC and other servers running x86_64 natively, it's working fine.

I tried running different commands on RPi, and I thought it could help someone with no direct access to RPi when I share some of my guessed commands (and its outputs) here. I would be glad if I could help somehow with a running RPi.

Sorry if I misled some concepts with userspace emulation (it's out of my mind). And thanks for the outstanding answer, I wouldn't have expected that.

Unai Martinez-Corral · Answer 10 · Fri Jan 15 2021 12:34:48 GMT+0800 (China Standard Time)

I can tell that the problem with no-ticking time does not occur on RPi with full-emulated QEMU system. I don't know if it's useful, but I think it's worth mentioning that the problem is not in QEMU itself. But maybe it's in qemu-user. I don't know.

I would expect that because qemu-system and qemu-user are, AFAIAA, different codebases with different features. In a full-emulated system, the host kernel is not used. It's a complete VM, isolated from the host. However, with qemu-user, the kernel is the one on the host, only instructions are translated (dynamically modified). Therefore, issues/bugs on one system are rarely reproducible on the other.

However, I think it'd be better located in a separated issue.

Do you really want me to open the issue with "the another approach"? 😄
I have started writing my process of gaining experience, but I was thinking only about one gist. But I can add it here as another issue also if you want.

Honestly, there is no much activity in the issues of this repo. I think it is harmless if you want to use one issue for that or for discussing experiments related to Docker and QEMU. For instance, it should theoretically be possible to convert containers to qemu-system images and vice versa. Not straightforward, probably ugly, but possible. Naturally, if you want to use a gist, that's awesome too.

The deno is written i. e. in Rust. And it's an issue with time, see message: "Operation not permitted" }', library/std/src/sys/unix/time.rs:361:62 like in
$ date -s '21-01-14 14:48:00'
/usr/bin/date: cannot set date: Operation not permitted
So it's container-dependent and will only occur when someone run Rust, but it's related to no-ticking time in a qus container on RPi.

Now I got it, and it makes sense to be here. Thanks for clarifying.

I would suggest running the tool on the host first.

It is not possible due to denoland/deno#2295. There is no support for my host CPU, and I think there is no hope for it for the next two months. But on my PC and other servers running x86_64 natively, it's working fine.

That's why you are setting up a qemu-user interpreter with qus first 😉

When the kernel calls the qemu-user interpreter for translating (in this case) amd64 instructions to armhf/aarch64, it is independent from the instructions coming from inside a container or from the host. Therefore, you can:

docker run --rm --privileged aptman/qus -s -- -p amd64
curl -o app URL_TO_YOUR_AMD64_EXECUTABLE
./app

You don't get the isolation that containers provide, but you can run foreign tools. Trying this will let us know whether the problem has anything to do with docker, or it's an issue about qemu-user and raspbian only.

BTW, you could run the same test after installing qemu-user through apt, instead of using qus. Version might be different, but the result should be similar.

Sorry if I misled some concepts with userspace emulation (it's out of my mind). And thanks for the outstanding answer, I wouldn't have expected that.

No worries at all, and you are welcome!

Marcin Zepp · Answer 11 · Fri Jan 15 2021 18:34:45 GMT+0800 (China Standard Time)

docker run --rm --privileged aptman/qus -s -- -p amd64

It ends up with ERROR: unknown CPU "amd64" but I have done it before with x86_64. There's a mistake if someone want to reproduce it.

Testing

I tried to get deno executable:

# src: https://github.com/hayd/deno-docker/blob/master/debian.dockerfile
$ DENO_VERSION=1.6.2
$ curl -fsSL https://github.com/denoland/deno/releases/download/v${DENO_VERSION}/deno-x86_64-unknown-linux-gnu.zip --output deno.zip && unzip deno.zip && rm deno.zip && chmod 755 deno
$ file deno
deno: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=9c8aecae420d2e0bc406713fab460bb4d72779e9, with debug_info, not stripped
$ ./deno
qemu-x86_64-static: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory

When I provided the file, I kept getting errors like:

ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libarmmem-${PLATFORM}.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
./deno: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory

There are commands to provide all libs (which were mentioned in errors):

$ mkdir libs
# on x86_64 machine: scp /lib64/ld-linux-x86-64.so.2 pi:/home/pi/libs/ld-linux-x86-64.so.2
# on x86_64 machine: scp /lib/x86_64-linux-gnu/{libdl.so.2,libpthread.so.0,libm.so.6,libc.so.6,libgcc_s.so.1}  pi:/home/pi/libs/
$ sudo mkdir /lib64
$ sudo mv libs/ld-linux-x86-64.so.2 /lib64/
$ sudo mkdir /lib/x86_64-linux-gnu
$ sudo mv libs/lib* /lib/x86_64-linux-gnu
$ rm libs

and finally I got:

$ ./deno
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libarmmem-${PLATFORM}.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.


#
# Fatal error in , line 0
# Fatal process out of memory: Failed to reserve memory for new V8 Isolate
#
#
#
#FailureMessage Object: 0x43c2ae00
==== C stack trace ===============================

    ./deno(+0x1086313) [0x41086313]
    ./deno(+0xaad68b) [0x40aad68b]
    ./deno(+0x1082215) [0x41082215]
    ./deno(+0xab5558) [0x40ab5558]
    ./deno(+0xc50622) [0x40c50622]
    ./deno(+0xc501d8) [0x40c501d8]
    ./deno(+0xb4b450) [0x40b4b450]
    ./deno(+0xacd30f) [0x40acd30f]
    ./deno(+0x43a7e1) [0x4043a7e1]
    ./deno(+0x62b4f9) [0x4062b4f9]
    ./deno(+0x3ad8c9) [0x403ad8c9]
    ./deno(+0x31a417) [0x4031a417]
    ./deno(+0x577322) [0x40577322]
    ./deno(+0x3c02a7) [0x403c02a7]
    ./deno(+0x1e4840) [0x401e4840]
    ./deno(+0x3cb48e) [0x403cb48e]
    /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x43cc40b3]
    ./deno(+0x183b7a) [0x40183b7a]
qemu: uncaught target signal 4 (Illegal instruction) - core dumped
Illegal instruction

I tried installing libs from debian repos, but I couldn't:

$ sudo dpkg --add-architecture amd64
$ echo "deb http://deb.debian.org/debian/ stable main" | sudo tee /etc/apt/sources.list
$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC 648ACFD622F3D138 DCC9EFBF77E11517
$ sudo apt-get update
$ LC_ALL=C sudo apt-get install libc6:amd64 libgcc1:amd64 gcc-8-base:amd64
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  <96 packages>
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  libidn2-0:amd64 libunistring2:amd64
Suggested packages:
  glibc-doc:amd64 debconf:amd64 | debconf-2.0:amd64 locales:amd64
The following packages will be REMOVED:
  <possibly all my packages installed: 1335 packages>
The following NEW packages will be installed:
  gcc-8-base:amd64 libc6:amd64 libgcc1:amd64 libidn2-0:amd64 libunistring2:amd64
WARNING: The following essential packages will be removed.
This should NOT be done unless you know exactly what you are doing!
  apt adduser (due to apt) gpgv (due to apt) raspbian-archive-keyring (due to apt) libapt-pkg5.0 (due to apt)
  libc6 (due to apt) libgcc1 (due to apt) libgnutls30 (due to apt) libseccomp2 (due to apt) libstdc++6 (due to apt)
  base-files mawk (due to base-files) base-passwd libdebconfclient0 (due to base-passwd) bash libtinfo6 (due to bash)
  debianutils (due to bash) bsdutils libsystemd0 (due to bsdutils) coreutils libacl1 (due to coreutils)
  libattr1 (due to coreutils) libselinux1 (due to coreutils) dash dpkg (due to dash) debconf (due to dash) diffutils
  libbz2-1.0 (due to dpkg) liblzma5 (due to dpkg) zlib1g (due to dpkg) tar (due to dpkg) e2fsprogs
  libblkid1 (due to e2fsprogs) libcom-err2 (due to e2fsprogs) libext2fs2 (due to e2fsprogs) libss2 (due to e2fsprogs)
  libuuid1 (due to e2fsprogs) fdisk libfdisk1 (due to fdisk) libmount1 (due to fdisk) libncursesw6 (due to fdisk)
  libsmartcols1 (due to fdisk) findutils grep libpcre3 (due to grep) install-info (due to grep) gzip hostname init
  systemd-sysv (due to init) init-system-helpers (due to init) perl-base (due to init-system-helpers) libc-bin login
  libaudit1 (due to login) libpam0g (due to login) libpam-runtime (due to login) libpam-modules (due to login) mount
  util-linux (due to mount) ncurses-bin sed sysvinit-utils libcap-ng0 (due to util-linux) libudev1 (due to util-linux)
0 upgraded, 5 newly installed, 1335 to remove and 3 not upgraded.
Need to get 3556 kB of archives.
After this operation, 2851 MB disk space will be freed.
You are about to do something potentially harmful.
To continue type in the phrase 'Yes, do as I say!'

I also found something like dpkg-cross, but I don't know how to use it.
I tried looking for:

ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libarmmem-${PLATFORM}.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.

and I found:

$ dpkg -S libarmmem
raspi-copies-and-fills: /usr/lib/arm-linux-gnueabihf/libarmmem-v6l.so
raspi-copies-and-fills: /usr/lib/arm-linux-gnueabihf/libarmmem-aarch64.so
raspi-copies-and-fills: /usr/lib/arm-linux-gnueabihf/libarmmem-v7l.so
raspi-copies-and-fills: /usr/lib/arm-linux-gnueabihf/libarmmem-v8l.so

so it's specific for RPi. I have no chance to find it on amd64 platform, do I?

I also tried with more trivial program and I built it on another machine:

$ cat date.cpp
#include <assert.h>
#include <stdio.h>
#include <time.h>
// src: https://stackoverflow.com/a/30759067/6732111
int main(void) {
    time_t t = time(NULL);
    struct tm *tm = localtime(&t);
    char s[64];
    assert(strftime(s, sizeof(s), "%c", tm));
    printf("%s\n", s);
    return 0;
}
$ gcc date.cpp -o date
$ scp date pi:/home/pi/date

and on RPi:

$ ./date
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libarmmem-${PLATFORM}.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
Fri Jan 15 10:50:06 2021

So it works, but there is this strange error.

I don't know why I can't run Deno. Maybe I don't have one lib?

I've got the same result with qemu-user from apt. Is it worth building it from sources? (qemu-system from apts wasn't working 😉)

Unai Martinez-Corral · Answer 12 · Fri Jan 15 2021 22:48:14 GMT+0800 (China Standard Time)

docker run --rm --privileged aptman/qus -s -- -p amd64

It ends up with ERROR: unknown CPU "amd64" but I have done it before with x86_64. There's a mistake if someone want to reproduce it.

You are correct. I have "architecture nomalisation" functions in CI scripts, but the QEMU naming must be used when running the container. Sorry about that.

I tried to get deno executable:

When I provided the file, I kept getting errors like:

There are commands to provide all libs (which were mentioned in errors):

and finally I got:

I tried installing libs from debian repos, but I couldn't:

I also found something like dpkg-cross, but I don't know how to use it.

I tried looking for:

and I found:

so it's specific for RPi. I have no chance to find it on amd64 platform, do I?

I don't know why I can't run Deno. Maybe I don't have one lib?

Ok. This is partially good. I forgot to specify that the x86_64/amd64 binary must be statically compiled/linked. The errors you are getting are because the app is dynamically linked. Therefore, as soon as you run it, it tries to first load the shared libraries it depends on, but those don't exist because it is referring the locations/names on a different system. Therefore, you are potentially missing multiple *.so files, not one only.

You might try finding all the missing libs in the repos, or in your workstation and copying all of them manually. However, that's kind of stupid because chroot's and containers are precisely meant for that purpose. We do not want to reinvent that wheel!

Can you try building deno statically instead? I know that Golang projects allow doing it trivially. I'm not sure about Rust, tho.

I also tried with more trivial program and I built it on another machine:

and on RPi:

This is nice! Let's start with this example. I believe that just adding -s to the gcc call should produce an static binary. You should be able to execute that on RPi without it complaining about the missing linker lib.

I've got the same result with qemu-user from apt. Is it worth building it from sources? (qemu-system from apts wasn't working 😉)

This is good, as expected. I don't think it's worth building from sources, unless you find a bug report which is fixed on the development branch. At the moment, the context is that qus, qemu-user through apt or building from sources produces the same result with regard to this issue.

I think that the main path we should follow now is the minimal C program you wrote. If you are good, let's continue investigating that.

Marcin Zepp · Answer 13 · Sat Jan 16 2021 05:13:06 GMT+0800 (China Standard Time)

$ gcc --static -o date date.cpp # the same file as above
$ file date
date: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=306a4f8ea19946f019b2e31ebc28f53918b5dcac, for GNU/Linux 3.2.0, not stripped
$ ldd date
        not a dynamic executable
# on RPi:
$ ./date
Fri Jan 15 22:08:21 2021

You should be able to execute that on RPi without it complaining about the missing linker lib.

Positive, it just works.

Unai Martinez-Corral · Answer 14 · Sat Jan 16 2021 05:34:53 GMT+0800 (China Standard Time)

Hmmm... this gets interesting. It seems to be a problem with docker and qemu-user, indeed. Can you please try running that same binary inside a container?

docker run --rm -itv $(pwd):/src -w /src amd64/ubuntu:18 ./date

Let's see if we can get a MWE to fail, instead of "regular" tools with many lines of code.

Marcin Zepp · Answer 15 · Sat Jan 16 2021 05:53:03 GMT+0800 (China Standard Time)

$ sudo docker run --rm -itv $(pwd):/src -w /src amd64/ubuntu:18.04 ./date
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
Thu Jan  1 00:00:00 1970

Unai Martinez-Corral · Answer 16 · Sat Jan 16 2021 06:07:20 GMT+0800 (China Standard Time)

Awesome! Thank you very much for getting here! I would propose first confirming that the issue is independent from qus by checking it with qemu-user installed through apt. After doing so:

It might be worth checking on some other host OS on the RPi. I tried Fedora on RPi3 two years ago, and it was barely usable. Arch Linux worked well, but I don't think docker was supported. Maybe any of those work well on RPi4. Ubuntu might work too.
Meanwhile, or after doing so, I think it's time to guess where is the docker build for Raspbian coming from. Where is that packaged? Who is building that? This is likely to be reported on QEMU and somewhere else at the same time. But we need to find out that other place (bugtracker).

Marcin Zepp · Answer 17 · Sat Jan 16 2021 06:22:19 GMT+0800 (China Standard Time)

Oh, it's a surprise.

I ran docker run --rm --privileged aptman/qus -- -r and installed with sudo apt install qemu binfmt-support qemu-user-static:

http://deb.debian.org/debian stable/main armhf qemu-user-static armhf 1:3.1+dfsg-8+deb10u8 [16,0 MB]
http://deb.debian.org/debian stable/main armhf binfmt-support armhf 2.2.0-2 [59,3 kB]
http://deb.debian.org/debian stable/main armhf qemu armhf 1:3.1+dfsg-8+deb10u8 [71,2 kB]

and:

$ ./date
Fri Jan 15 23:16:48 2021
$ sudo docker run --rm -itv $(pwd):/src -w /src amd64/ubuntu:18.04 ./date
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
Fri Jan 15 22:17:16 2021

Probably I forgot to unregister qus when I was installing qemu-user-static through apt before.

Marcin Zepp · Answer 18 · Sat Jan 16 2021 06:40:52 GMT+0800 (China Standard Time)

Wait, where is my Raspbian sources... I just double-checked with original sources.list (and apt remove and apt update) that the output is the same. The qemu-user-static from official Raspbian apt sources just works inside Docker container. There is no problem with time. I will try tomorrow with Deno.

Marcin Zepp · Answer 19 · Sat Jan 16 2021 07:31:23 GMT+0800 (China Standard Time)

It's interesting. I tried with Deno, it ends up with mentioned Fatal process out of memory: Failed to reserve memory for new V8 Isolate. So I tried to walk through the installation process, and I couldn't install anything due to failing triggers for libc-bin.
However, with qus there is no such problem with libc-bin but there is the time issue constantly.

Logs (qemu from apt)

$ sudo docker run --rm -it amd64/ubuntu
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
root@d7e95767dd83:/# apt-get update
Get:1 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [109 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:9 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [22.0 kB]
Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [905 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [961 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [173 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [4301 B]
Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [650 kB]
Get:15 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [146 kB]
Get:16 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [560 kB]
Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [741 B]
Fetched 16.8 MB in 57s (297 kB/s)
Reading package lists... Done
root@d7e95767dd83:/# apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done

The following packages will be upgraded:
  apt libapt-pkg6.0 libp11-kit0 tar
4 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 2551 kB of archives.
After this operation, 3072 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 tar amd64 1.30+dfsg-7ubuntu0.20.04.1 [240 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.2ubuntu0.2 [832 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.2ubuntu0.2 [1290 kB]
Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libp11-kit0 amd64 0.23.20-1ubuntu0.1 [188 kB]
Fetched 2551 kB in 3s (863 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
(Reading database ... 4121 files and directories currently installed.)
Preparing to unpack .../tar_1.30+dfsg-7ubuntu0.20.04.1_amd64.deb ...
Unpacking tar (1.30+dfsg-7ubuntu0.20.04.1) over (1.30+dfsg-7) ...
Setting up tar (1.30+dfsg-7ubuntu0.20.04.1) ...
(Reading database ... 4121 files and directories currently installed.)
Preparing to unpack .../libapt-pkg6.0_2.0.2ubuntu0.2_amd64.deb ...
Unpacking libapt-pkg6.0:amd64 (2.0.2ubuntu0.2) over (2.0.2ubuntu0.1) ...
Setting up libapt-pkg6.0:amd64 (2.0.2ubuntu0.2) ...
(Reading database ... 4121 files and directories currently installed.)
Preparing to unpack .../apt_2.0.2ubuntu0.2_amd64.deb ...
Unpacking apt (2.0.2ubuntu0.2) over (2.0.2ubuntu0.1) ...
Setting up apt (2.0.2ubuntu0.2) ...
(Reading database ... 4121 files and directories currently installed.)
Preparing to unpack .../libp11-kit0_0.23.20-1ubuntu0.1_amd64.deb ...
Unpacking libp11-kit0:amd64 (0.23.20-1ubuntu0.1) over (0.23.20-1build1) ...
Setting up libp11-kit0:amd64 (0.23.20-1ubuntu0.1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.1) ...
Segmentation fault (core dumped)
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@d7e95767dd83:/# apt-get install ca-certificates curl unzip
[...]
1 not fully installed or removed.
[...]
debconf: delaying package configuration, since apt-utils is not installed
Setting up libc-bin (2.31-0ubuntu9.1) ...
Segmentation fault (core dumped)
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@d7e95767dd83:/# exit

isolated libc-bin:

pi@raspberrypi:~ $ wget http://cz.archive.ubuntu.com/ubuntu/pool/main/g/glibc/libc-bin_2.31-0ubuntu9_amd64.deb
pi@raspberrypi:~ $ sudo docker run --rm -itv $(pwd):/src -w /src amd64/ubuntu
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
root@43b23f0ca1d3:/src# dpkg -i libc-bin_2.31-0ubuntu9_amd64.deb
dpkg: warning: downgrading libc-bin from 2.31-0ubuntu9.1 to 2.31-0ubuntu9
(Reading database ... 4121 files and directories currently installed.)
Preparing to unpack libc-bin_2.31-0ubuntu9_amd64.deb ...
Unpacking libc-bin (2.31-0ubuntu9) over (2.31-0ubuntu9.1) ...
Setting up libc-bin (2.31-0ubuntu9) ...
Segmentation fault (core dumped)
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--install):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
root@43b23f0ca1d3:/src#

Logs (qus)

$ sudo apt purge qemu binfmt-support qemu-user-static
$ sudo docker run --rm --privileged aptman/qus -s -- -p x86_64
cat ./qemu-binfmt-conf.sh | sh -s -- --path=/qus/bin -p x86_64 --suffix -static
Setting /qus/bin/qemu-x86_64-static as binfmt interpreter for x86_64
 sudo docker run --rm -itv $(pwd):/src -w /src amd64/ubuntu
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
root@3332d77ee427:/src# date
Sun Jan 11 12:55:12 UTC 2004
root@3332d77ee427:/src# dpkg -i libc-bin_2.31-0ubuntu9_amd64.deb
dpkg: warning: downgrading libc-bin from 2.31-0ubuntu9.1 to 2.31-0ubuntu9
(Reading database ... 4121 files and directories currently installed.)
Preparing to unpack libc-bin_2.31-0ubuntu9_amd64.deb ...
Unpacking libc-bin (2.31-0ubuntu9) over (2.31-0ubuntu9.1) ...
Setting up libc-bin (2.31-0ubuntu9) ...
root@3332d77ee427:/src# exit

Unai Martinez-Corral · Answer 20 · Sat Jan 16 2021 07:41:09 GMT+0800 (China Standard Time)

Can you please confirm that you are using the latest version of qus?

# docker run --rm -it --entrypoint=sh aptman/qus -c '/qus/bin/qemu-x86_64-static --version'
qemu-x86_64 version 5.2.0 (Debian 1:5.2+dfsg-3)
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers

Marcin Zepp · Answer 21 · Sat Jan 16 2021 07:46:34 GMT+0800 (China Standard Time)

Positive, the same output.

Unai Martinez-Corral · Answer 22 · Sat Jan 16 2021 07:57:46 GMT+0800 (China Standard Time)

I guess some testing script needs to be setup for testing it several times with each approach. The only lead I have at the moment is that all failures seem to provide either 1970 or 2004. These are the wrong dates reported in this issue:

Mon Jan 19 12:19:25 UTC 2004
Thu Jan  1 00:00:00 UTC 1970
Sat Oct 30 08:53:36 UTC 2004
Thu Jan  1 00:00:00 1970
Sun Jan 11 12:55:12 UTC 2004

It's surprising both that the years are two only and that it's not one only. It's not different users reporting different years. You got both of those on the same device too!

Marcin Zepp · Answer 23 · Sat Jan 16 2021 08:48:19 GMT+0800 (China Standard Time)

I tried building qemu from sources (the same result like from apt)

$ sudo apt-get install git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev libsdl2-dev ninja-build
$ git clone git://git.qemu.org/qemu.git
$ cd qemu
$ git checkout v5.2.0
$ git submodule update --init --recursive
$ ./configure --static --disable-system --enable-linux-user --target-list="x86_64-linux-user" --extra-cflags="-mcpu=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard" --extra-cxxflags="-mcpu=arm1176jzf-s -mfpu=vfp -mfloat-abi=hard"
$ make -j4
$ sudo apt-get install debootstrap qemu qemu-user-static
$ sudo cp build/x86_64-linux-user/qemu-x86_64 /usr/bin/qemu-x86_64-static
$ sudo docker run --rm -it amd64/ubuntu
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
root@52fff2c754bb:/# apt-get update
<works>
root@52fff2c754bb:/# apt-get upgrade
[...]
Processing triggers for libc-bin (2.31-0ubuntu9.1) ...
Segmentation fault (core dumped)
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@52fff2c754bb:/# date
Sat Jan 16 00:45:45 UTC 2021
root@52fff2c754bb:/#

Unai Martinez-Corral · Answer 24 · Sat Jan 16 2021 10:43:19 GMT+0800 (China Standard Time)

The binaries that qus uses are the ones from http://ftp.debian.org/debian/pool/main/q/qemu/. Currently, these might be the ones being used:

According to your logs (linux/arm/v7) the aptman/qus image you are using (it's a manifest, per se) includes the content of qemu-user-static_5.2+dfsg-3_armhf.deb. That's what qus is about! That deb file is downloaded, it is extracted, and it is put into a container based on arm32v7/busybox along with a couple of shell scripts.

I would expect these binaries to be slightly different from the ones you got when building QEMU from sources, since you used -mcpu=arm1176jzf-s. Maybe that's the point. Shall we follow that guess?

Do you want to try extracting that deb file and using the qemu-user-static binary for x86_64 from there. All of the executables you'll find when extracting qemu-user-static_5.2+dfsg-3_armhf.deb are different targets for a single host.
Why did you decide to use that -mcpu option? Is it possible that the artifacts you get through apt were also built using that option?
Did you try using the aptman/qus image for aarch64 hosts explicitly, instead of letting it pick the one for 32 bits? docker run --rm --privileged aptman/qus:arm64v8-f5.2.0 -s -- -p x86_64

EDIT

The tarball you can download from https://github.com/dbhi/qus/releases/tag/v0.0.6-v5.2%2Bdfsg-3 column arm32v7, row x86_64, should be the same as the one aptman:qus is using. And the on in the same row but column arm64v8, should be the one used by aptman/qus:arm64v8-f5.2.0.

Marcin Zepp · Answer 25 · Sat Jan 16 2021 17:21:14 GMT+0800 (China Standard Time)

These are the wrong dates reported in this issue:

I have written the following script to check which dates I can get:

test-distro() {
  echo -en " |\n| $1 | ";
  sudo docker run --rm -it $1 /bin/sh -c "date | tr -d '\r' | tr -d '\n'; sleep 10; echo -n ' | '; date | tr -d '\r' | tr -d '\n';";
}
test() {
  sudo docker run --rm --privileged aptman/qus -s -- -p $1 > /dev/null
  for distro in ubuntu:20.04 ubuntu:18.04 ubuntu:14.04 alpine centos centos:6 debian debian:8 amazonlinux fedora cirros clearlinux; do
    test-distro $2/$distro
  done;
  # sudo docker system prune -af; # CAUTION! it will delete all docker-related stuff
}
( echo -en "| distro | date | date after 10s |\n| - | - | -";
test x86_64 amd64;
test i386 i386;
test aarch64 arm64v8;
echo " |"; ) | tee out; echo -e '\n\n\n'; cat out;

and I left RPi working through the night, but today I can't connect to it, although it seems to be a connection issue (I don't have it physically, I have borrowed it from @Marwyk2003, thanks).

I would expect these binaries to be slightly different from the ones you got when building QEMU from sources, since you used -mcpu=arm1176jzf-s. Maybe that's the point. Shall we follow that guess?

If I have done everything correctly, the result of using my built QEMU is the same as the one of qemu-x86_64-static from http://ftp.debian.org/debian/pool/main/q/qemu/qemu-user-static_3.1+dfsg-8+deb10u8_armhf.deb. So the -mcpu=arm1176jzf-s did nothing special.

Do you want to try extracting that deb file and using the qemu-user-static binary for x86_64 from there. All of the executables you'll find when extracting qemu-user-static_5.2+dfsg-3_armhf.deb are different targets for a single host.

It's a nice idea! I will try when I have access.

Why did you decide to use that -mcpu option? Is it possible that the artifacts you get through apt were also built using that option?

Why? I don't know, I just found it here. I used it when was compiling QEMU for full system emulation, and it worked. But I don't know if this would work without this option.

Did you try using the aptman/qus image for aarch64 hosts explicitly, instead of letting it pick the one for 32 bits? docker run --rm --privileged aptman/qus:arm64v8-f5.2.0 -s -- -p x86_64

As far as I know, I can't. Raspberry Pi 3 Model B Rev 1.2 provides me ARMv7 Processor rev 4 (v7l) (got from /cpu/info), and I think it can't run 64-bits. But for sure I have (or I had if I've broken something) armv7l (uname) kernel which shouldn't support aarch64, does it?

Marcin Zepp · Answer 26 · Sat Jan 16 2021 17:34:06 GMT+0800 (China Standard Time)

distro	date	date after 10s
amd64/ubuntu:20.04	Sun Jan 11 12:55:12 UTC 2004	Sun Jan 11 12:55:12 UTC 2004
amd64/ubuntu:18.04	Thu Jan 1 00:00:00 UTC 1970	Thu Jan 1 00:00:00 UTC 1970
amd64/ubuntu:14.04	Thu Jan 1 00:00:00 UTC 1970	Thu Jan 1 00:00:00 UTC 1970
amd64/alpine	Thu Jan 1 00:00:05 UTC 1970	Thu Jan 1 00:00:05 UTC 1970
amd64/centos	Thu Jan 22 01:17:04 UTC 2004	Thu Jan 22 01:17:04 UTC 2004
amd64/centos:6	Thu Jan 1 00:00:00 UTC 1970	Thu Jan 1 00:00:00 UTC 1970
amd64/debian	Thu Jan 1 00:00:00 UTC 1970	Thu Jan 1 00:00:00 UTC 1970
amd64/debian:8	Fri Jan 2 12:26:20 UTC 1970	Fri Jan 2 12:26:20 UTC 1970
amd64/amazonlinux	Thu Jan 1 00:00:00 UTC 1970	Thu Jan 1 00:00:00 UTC 1970
amd64/fedora	Thu Jan 1 00:00:00 UTC 1970	Thu Jan 1 00:00:00 UTC 1970
amd64/cirros	Thu Apr 30 17:58:44 UTC 2037	Fri Jun 5 14:08:20 UTC 2037
i386/ubuntu:20.04	Tue Nov 23 22:38:24 UTC 2004	Tue Nov 23 22:38:24 UTC 2004
i386/ubuntu:18.04	Mon Nov 22 21:38:08 UTC 2004	Mon Nov 22 21:38:08 UTC 2004
i386/ubuntu:14.04	Thu Jan 1 00:00:00 UTC 1970	Thu Jan 1 00:00:00 UTC 1970
i386/alpine	- - 1100327680 00:32:1073741876 1900	- - 1100327680 00:32:1073741876 1900
i386/centos	Mon May 10 07:01:20 UTC 2004	Mon May 10 07:01:20 UTC 2004
i386/centos:6	Tue Apr 20 14:46:02 UTC 2004	Tue Apr 20 14:46:02 UTC 2004
i386/debian	Sat Oct 30 08:53:36 UTC 2004	Sat Oct 30 08:53:36 UTC 2004
i386/debian:8	Thu May 6 01:44:32 UTC 2004	Thu May 6 01:44:32 UTC 2004
i386/cirros	Thu Jan 1 00:00:36 UTC 1970	Thu Jan 1 00:00:36 UTC 1970

Unai Martinez-Corral · Answer 27 · Sat Jan 16 2021 17:46:34 GMT+0800 (China Standard Time)

If I have done everything correctly, the result of using my built QEMU is the same as the one of qemu-x86_64-static from http://ftp.debian.org/debian/pool/main/q/qemu/qemu-user-static_3.1+dfsg-8+deb10u8_armhf.deb. So the -mcpu=arm1176jzf-s did nothing special.

Unless you actually downloaded the DEB package from that URL, I wouldn't be sure about what you get through apt being that package. If that is the case, it should be exactly the same binary as the one used by qus.

Why? I don't know, I just found it here. I used it when was compiling QEMU for full system emulation, and it worked. But I don't know if this would work without this option.

It seems that you copied the options corresponding to ARMv6/ARM1176JZF-S. For ARMv8-A/BCM2837, those should be -march=armv7-a or -march=armv8-a+crc. Hence, I guess that option was/is not relevant.

As far as I know, I can't. Raspberry Pi 3 Model B Rev 1.2 provides me ARMv7 Processor rev 4 (v7l) (got from /cpu/info), and I think it can't run 64-bits. But for sure I have (or I had if I've broken something) armv7l (uname) kernel which shouldn't support aarch64, does it?

The RPi3 has a 64 bit CPU, which can handle armv7 and armv8 architectures (as shown in the gentoo wiki). Therefore, it can run 64 bits. You get ARMv7 from the system because Raspbian is built for 32 bits only, so that the same SD works on any RPi. However, I could successfully run arm64v8 containers on Raspbian. Just try docker run --rm -it arm64v8/ubuntu:18 bash. No need to setup QEMU before. It's the same as running i386 containers on wour amd64 PC/laptop.

Marcin Zepp · Answer 28 · Sun Jan 17 2021 01:53:59 GMT+0800 (China Standard Time)

Nope.

$ update-binfmts --display
python2.7 (enabled):
     package = python2.7
        type = magic
      offset = 0
       magic = \x03\xf3\x0d\x0a
        mask =
 interpreter = /usr/bin/python2.7
    detector =
python3.7 (enabled):
     package = python3.7
        type = magic
      offset = 0
       magic = \x42\x0d\x0d\x0a
        mask =
 interpreter = /usr/bin/python3.7
    detector =
$ sudo docker run --rm -it arm64v8/ubuntu bash
WARNING: The requested image's platform (linux/arm64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested
standard_init_linux.go:219: exec user process caused: exec format error

My lsb_release -a, uname -a and /proc/cpuinfo are in #6 (comment).

Marcin Zepp · Answer 29 · Sun Jan 17 2021 02:17:43 GMT+0800 (China Standard Time)

Okay...

$ wget http://ftp.debian.org/debian/pool/main/q/qemu/qemu-user-static_5.2+dfsg-3_armhf.deb
$ sudo dpkg -i qemu-user-static_5.2+dfsg-3_armhf.deb
$ ./date && sudo docker run --rm -it amd64/ubuntu date
Sat Jan 16 19:32:11 2021
Sun Jan 11 12:55:12 UTC 2004

and

$ sudo dpkg -i qemu-user-static_5.2+dfsg-3~bpo10+1_armhf.deb # buster backports
$ ./date && sudo docker run --rm -it amd64/ubuntu date
Sat Jan 16 19:23:01 2021
Sat Jan 16 18:23:14 UTC 2021

and

$ sudo dpkg -i qemu-user-static_3.1+dfsg-8+deb10u8_armhf.deb # buster
$ ./date && sudo docker run --rm -it amd64/ubuntu date
Sat Jan 16 19:25:20 2021
Sat Jan 16 18:25:43 UTC 2021

and

$ sudo dpkg -i qemu-user-static_2.8+dfsg-6+deb9u12_armhf.deb # stretch
$ ./date && sudo docker run --rm -it amd64/ubuntu date
Sat Jan 16 19:26:59 2021
Sat Jan 16 18:27:19 UTC 2021

If someone wants to reproduce it, I sometimes have standard_init_linux.go:219: exec user process caused: no such file or directory error, but reinstalling resolves it.

Marcin Zepp · Answer 30 · Sun Jan 17 2021 04:51:06 GMT+0800 (China Standard Time)

I have observed that all versions mentioned above fail with Fatal process out of memory: Failed to reserve memory for new V8 Isolate when I try to run Deno. The only exception is with the version 5.2+dfsg-3_armhf in docker container which fails even faster with Rust complaining about time. So even fixing a date issue will not help me in running Deno. But in the context of this issue, I think it's a problem with the specific version of qemu.

Marcin Zepp · Answer 31 · Sun Jan 17 2021 06:29:58 GMT+0800 (China Standard Time)

It seems that you copied the options corresponding to ARMv6/ARM1176JZF-S. For ARMv8-A/BCM2837, those should be -march=armv7-a or -march=armv8-a+crc. Hence, I guess that option was/is not relevant.

I tried building QEMU with -march=armv8-a+crc and -mcpu=cortex-a53 -mfloat-abi=hard -mfpu=neon-fp-armv8 -mneon-for-64bits -mtune=cortex-a53 found here. Both work (tested with sudo docker run --rm -it amd64/ubuntu bash -c "apt-get update && apt-get upgrade") and both crash due to the date issue.
So the problem persists in v5.2.0 in qemu repo. But how 5.2+dfsg-3_armhf and 5.2+dfsg-3~bpo10+1_armhf debian packages differ? How can only one of them not produce the date issue if they are both built from 5.2 qemu release or didn't they?

FailedShack · Answer 32 · Thu Apr 29 2021 00:44:06 GMT+0800 (China Standard Time)

Experiencing the exact same issue on another Raspberry Pi 4, date stuck in 2004 and containers crashing

Edit: I am also able to confirm that installing the qemu-user-static from buster backports somehow sidesteps this issue