Build issue with eigen (MappedSparseMatrix)

[meerfrau]

What happened?

MappedSparseMatrix was removed from eigen 3.4.x

Therefore any build against it fails:

./eckit/linalg/sparse/LinearAlgebraEigen.cc:29:22: error: 'MappedSparseMatrix' in namespace 'Eigen' does not name a template type; did you mean 'SparseMatrix'?
   29 | using spm_t = Eigen::MappedSparseMatrix<Scalar, Eigen::RowMajor, Index>;
      |                      ^~~~~~~~~~~~~~~~~~
      |                      SparseMatrix
./eckit/linalg/sparse/LinearAlgebraEigen.cc: In member function 'virtual void eckit::linalg::sparse::LinearAlgebraEigen::spmv(const eckit::linalg::SparseMatrix&, const eckit::linalg::Vector&, eckit::linalg::Vector&) const':
./eckit/linalg/sparse/LinearAlgebraEigen.cc:45:5: error: 'spm_t' was not declared in this scope
   45 |     spm_t Ai(A.rows(), A.cols(), A.nonZeros(), const_cast<Index*>(A.outer()), const_cast<Index*>(A.inner()), const_cast<Scalar*>(A.data()));
      |     ^~~~~
./eckit/linalg/sparse/LinearAlgebraEigen.cc:49:10: error: 'Ai' was not declared in this scope; did you mean 'yi'?
   49 |     yi = Ai * xi;
      |          ^~
      |          yi
./eckit/linalg/sparse/LinearAlgebraEigen.cc: In member function 'virtual void eckit::linalg::sparse::LinearAlgebraEigen::spmm(const eckit::linalg::SparseMatrix&, const eckit::linalg::Matrix&, eckit::linalg::Matrix&) const':
./eckit/linalg/sparse/LinearAlgebraEigen.cc:62:5: error: 'spm_t' was not declared in this scope
   62 |     spm_t Ai(A.rows(), A.cols(), A.nonZeros(), const_cast<Index*>(A.outer()), const_cast<Index*>(A.inner()), const_cast<Scalar*>(A.data()));
      |     ^~~~~
./eckit/linalg/sparse/LinearAlgebraEigen.cc:66:10: error: 'Ai' was not declared in this scope; did you mean 'Ci'?
   66 |     Ci = Ai * Bi;
      |          ^~
      |          Ci

When I however build against a locally cloned eigen (3.2.1 or patched 3.3.9) at eckit/maths/eigen3 the build works, but unluckily it segfaults(?) SIGABRTs when running:

./eckit-info

Exception: Bad operator: ELF (String) method 'contains' not implemented  (./eckit/value/Content.cc +124 contains)
terminate called after throwing an instance of 'eckit::BadOperator'
  what():  Bad operator: ELF (String) method 'contains' not implemented

What are the steps to reproduce the bug?

  cmake \
    -DCMAKE_INSTALL_PREFIX=/usr \
    -DCMAKE_BUILD_TYPE=production \
..
make

Version

git master

Platform (OS and architecture)

ArchLinux

Relevant log output

No response

Accompanying data

No response

Organisation

earth friends

[Pedro Maciel]

Can you write how it segfaults when running? So I can reproduce.

I routinely build and test with eigen 3.4.0 (on both macOS/ARM64 and RHEL/amd64) and I don't find the same issue(s).

[meerfrau]

Here is an strace of eckit-info

I routinely build and test with eigen 3.4.0 (on both macOS/ARM64 and RHEL/amd64) and I don't find the same issue(s).

@pmaciel May you please link me an RPM of the eigen 3.4.0 which you use?

I've invested quite an effort to get it compiling. I could be mad at myself ;)

[Pedro Maciel]

You are right - we do use eigen 3.4.0 but not for the production software stack I'm responsible for (we currently have 3.3.9 already for some time). On my local machine I confirm that i'm not using Eigen but a generic matrix implementation (of our own design) instead, thus bypassing 3.4.0. I'm sorry for my wrong assumption.

I'll look into eigen 3.4.0 support, thanks for pointing us to this.

[Pedro Maciel]

@meerfrau you're right that the Eigen::MappedSparseMatrix has been deprecated and removed from Eigen's master branch. But, note that the master branch is currently ahead of the latest tag (we, in principle and as you're assuming, always tag the latest commit to master/main)

Note that eckit's interfacing to Eigen complies with the latest tag (3.4.0) -- inclusively, I actually was using 3.4.0 (without paying much attention) for integration with the Research Department which are always ahead of our production stack.

Obviously, a notice of deprecation is very important and will be heeded. At the moment however, without a formal release, there isn't an urgency.

[meerfrau]

@pmaciel You maybe haven't noticed it, but I quickly changed to eigen 3.3.9 to compile successfully. I was mostly concerned about the abort signal directly after starting eckit:

Exception: Bad operator: ELF (String) method 'contains' not implemented  (./eckit/value/Content.cc +124 contains)
terminate called after throwing an instance of 'eckit::BadOperator'
  what():  Bad operator: ELF (String) method 'contains' not implemented

I've created a zipped tarball, I hope that works for you. May you please check whether you get the same error on running?
eckit_bin.tar.zip

[Pedro Maciel]

Unfortunately I don't have a setup able to run your build as-is. I don't know you you mean by eigen "patched" 3.3.9, and one the "What are the steps to reproduce the bug" there's the cmake variable CMAKE_BUILD_TYPE=production which actually fails configuration in eigen 3.3.9 (they have something special going on there that I don't think is correct).

The failure you mention is not really symptomatic of eigen but instead of something else (?). I suspect your build directory is out of sync with the sources, and try first to do a clean build disabling Eigen (passing -DENABLE_EIGEN=OFF).

Berfore proceeding, can you try the above please?

[Oliver Lomax]

In case it's relevant, I've been using Eigen::Map<SparseMatrix> successfully with eckit SparseMatrix objects. Appears to be supported by Eigen 3.4.x https://www.eigen.tuxfamily.org/dox/classEigen_1_1Map_3_01SparseMatrixType_01_4.html

[xiangyi wang]

I encountered the same problem and 3.3.9 can succeed