Giters

ceph / ceph-container

Docker files and images to run Ceph in containers

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

arm64 image missing ganesha.nfsd binary

tibbe opened this issue · comments

commented

Is this a bug report or feature request?

  • Bug Report

Bug Report

What happened:
The ceph-v15.2.10 (and others) arm64 image doesn't contain the ganesha.nfsd binary, causing the NFS container to fail to start. See rook/rook#7671 for the full bug report against rook (where I first experienced this).

What you expected to happen:
arm64 image to contain ganesha.nfsd binary.

How to reproduce it (minimal and precise):

Please see rook/rook#7671.

Environment:
Please see data in rook/rook#7671.

  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Docker version (e.g. docker version):
  • Ceph version (e.g. ceph -v):
commented

That's because nfs-ganesha packages aren't available on arm64 at the moment.

https://download.ceph.com/nfs-ganesha/rpm-V3.3-stable/octopus/el8/aarch64/

commented

Filed an issue against ceph: https://tracker.ceph.com/issues/50437

commented

I'm confused: https://hub.docker.com/layers/rook/nfs/v1.6.0/images/sha256-d83c8fef4ecaf16e18de65ff4c6a767aaad9a84eb648c2ea057f4dd45fac8c23?context=explore is an arm64 ganesha and is used by rook/nfs so there's at least a ganesha build available somewhere that is used by rook, just not through their rook/ceph images.

commented

I'm not a rook expert but I'm pretty sure rook/nfs is not the same than rook/ceph via nfs-ganesha.

The one you linked seems like a regular nfs service as a rook storage provider. But nothing related to nfs-ganesha.

The rook/nfs container images aren't based on this project (as opposed to rook/ceph images)

https://github.com/rook/rook#project-status
https://github.com/rook/rook/blob/master/Documentation/nfs.md

commented

I thought so as well but I took a peak at the rook/nfs image and it's launching the NFS service like so:

nfs server --ganeshaConfigPath=/nfs-ganesha/config/rook-nfs

I realize that these are different containers. I just wanted to point out that there are arm64 packages out there already used by the rook project (which I realize is different from this project).

It's one of these things that seems to fall between the cracks when lots of different projects are involved ending up with a bit of a fragmented support from a (rook/ceph) user's perspective.

commented

nfs server --ganeshaConfigPath=/nfs-ganesha/config/rook-nfs

That's not how ganesha should be started so it's maybe a different thing. The binary for ganesha is ganesha.nfsd, like:

https://github.com/ceph/ceph-container/blob/master/src/daemon/demo.sh#L303
https://github.com/ceph/ceph-container/blob/master/src/daemon/start_nfs.sh#L38

I just wanted to point out that there are arm64 packages out there already used by the rook project (which I realize is different from this project).

I'm not saying there's no nfs-ganesha packages for arm64 at all. I'm just saying that the ceph infrastructure (download.ceph.com) doesn't offer nfs-ganesha arm64 builds for both rpm and deb. And the ceph container image only use the ceph infrastructure as source of ceph/ganesha packages.

I know that there are existing arm64 elsewhere like

And probably in other locations.

The rook/nfs container image is based on Ubuntu (whereas the ceph one is based on CentOS) and uses a gluster ppa repository http://ppa.launchpad.net/gluster/nfs-ganesha-2.7/ubuntu/pool/main/n/nfs-ganesha/.

Don't get me wrong, I would love to add nfs-ganesha to the ceph arm64 container image if we have builds on the ceph infra in the future (like we did for ceph-iscsi recently).