Segfault while calculating emds after import pytorch

[Raghav Kansal]

I'm getting an unusual segfault when using the emd.emds() function on Mac OS 11.4, which seems to only occur if importing pytorch before energyflow.

Script to reproduce segfault:

import torch
import energyflow.emd as emd
import numpy as np

x = np.random.rand(100, 30, 4)
y = np.random.rand(100, 30, 4)

emd.emds(x, y)  # will segfault

using:

macOS: 11.4
python: 3.9.6
numpy: 1.21.2
energyflow: 1.3.2
wasserstein: 1.0.1
torch: 1.9.0

Importing torch after energyflow works fine:

import energyflow.emd as emd
import torch
import numpy as np

x = np.random.rand(100, 30, 4)
y = np.random.rand(100, 30, 4)

emd.emds(x, y)  # no segfault

[Patrick T. Komiske III]

Thanks for pointing this out. I'm 95% sure that what's going on is a library version incompatibility. The EnergyFlow dependency Wasserstein is currently compiled against libomp 11.0 on Mac, whereas torch probably ships with an older version of the library (probably 10.something). The way the library symbols are dynamically resolved on Mac means that the first time the library is loaded, those symbols are used. So loading torch first will resolve the symbols needed by Wasserstein with an incompatible version, causing a mysterious seg fault.

So for now, it should be harmless to use the imports in the order that works. Wasserstein should be compiled with an older version of libomp on Mac, which is in the works (I'm also the maintainer of that package). If you ask my opinion, the way that dynamic library loading in Python defaults to global symbol resolution on Mac is problematic in that it allows exactly this problem to occur.