JNI implementation
GuilhemSempere opened this issue · comments
Hi
We have several web applications submitting jobs to SGE using the Java-based Opal Toolkit (https://sourceforge.net/projects/opaltoolkit/) via drmaa.jar
In the process of migrating from SGE to Slurm I realized that the JNI methods do not exist in the slurm-drmaa library. Do you know of any out-of-the-box solution for submitting jobs via drmaa.jar to Slurm?
Best regards,
Guilhem
Hi, I'm trying to do the same using drmaa.jar from SGE but after compiling libdrmaa.so for SLURM I'm trying to point to the compiled library in:
export DRMAA_LIBRARY_PATH=/home/vargasfr/slurm_drmaa/slurm-drmaa-1.1.2/install/lib/libdrmaa.so
export LD_LIBRARY_PATH=/home/vargasfr/slurm_drmaa/slurm-drmaa-1.1.2/install/lib/
But I'm getting an error in the Java code:
Exception in thread "main" java.lang.UnsatisfiedLinkError: com.sun.grid.drmaa.SessionImpl.nativeInit(Ljava/lang/String;)V
at com.sun.grid.drmaa.SessionImpl.nativeInit(Native Method)
at com.sun.grid.drmaa.SessionImpl.init(SessionImpl.java:291)
at Test2.main(Test2.java:16)
Did you find a solution?
Thanks!
Hi
Well the solution I went for was to write my own JobManager (which doesn't use drmaa) for Opal Toolkit.
If your goal is also to make Opal work with Slurm you will probably be able to use this out of the box: https://gitlab.southgreen.fr/AGAP_ID_Java/Opal-SlurmCompatible
Otherwise maybe you can use it as an example and adapt it to your own purpose?
G.
This jar is the result of running build.xml, it contains just the 2 classes that needed to be added to Opal Toolkit for it to be compatible with Slurm. So this is indeed where you have to look:
I must specify that if your current app entirely relies on drmaa, I think you'll have much more to implement. In my case Opal was already able to talk to several non-drmaa schedulers so it was quite easy to take one of them as example and adapt the code to Slurm syntax.
Good luck ;-)
I'm not really familiar enough with Java to understand the details here but it seems that Java bindings for DRMAA like the one provided with SGE are not simply wrappers around the C libraries like the bindings for other languages (e.g. Python). Here is some discussion of it from a long while back for someone looking to do the same with Torque:
https://stackoverflow.com/questions/2715243/is-there-a-drmaa-java-library-that-works-with-torque-pbs
Creating a full Java DRMAA binding that is Slurm-specific is well outside the scope of this project, unless I am misunderstanding what is necessary to support a Java binding that links against slurm-drmaa.
Closing for out of scope, but please let me know if I've misunderstood.