Build cogutils under cygwin
marknuzz opened this issue · comments
This is the first step to porting the project to cygwin. I will leave notes and comments here
TLS (Thread-Local Storage) is not supported with PE Executables (win32/cygwin) under clang. This is stated on the Cygwin mailing list posting here: https://cygwin.com/ml/cygwin/2014-09/msg00414.html
This causes compilation to fail under clang on cygwin when using #include <mutex>.
I looked into using Boost for this.. Boost provides a cross-platform implementation of the Mutex class, as suggested in this SO Q&A as well as other pieces of documentation here and here
Problem is, it seems that there are some issues with Boost and that the project is not prioritizing Cygwin compatibility, but rather Mingw64... See https://svn.boost.org/trac/boost/ticket/10942 for background information.
TLS seems to work fine under gcc, so it looks like for now that Cygwin compilation will only be supported with gcc until the Mutex issues are fixed.
Spent a good two days getting ubuntu to connect to my shared cygwin-hosted drive, due to various technical issues. cmake would fail when using gvfsd due to the comma and equal sign in the folder name. After switching to smbnetfs, cmake would hang when starting a job when the target folder was hosted on cygwin.
I was able to get cmake to write to my windows drive from ubuntu, by specifying an option within smbnetfs (max_retry_count=0). Wasn't strictly necessary to have cmake write to cygwin from ubuntu, but the bug indicated an underlying problem with the file system mount, which I didn't feel good about not resolving.
Filed a bug report with the dev here, and the hang issue is resolved on my end for now: https://sourceforge.net/p/smbnetfs/bugs/34/
Next on the list is getting cmake to locate the boost library within my ubuntu installation.
Then the goal is to have side by side terminals (one cygwin, one ubuntu) where I can run cmake jobs on each, using the same source folder. That way, I can determine if problems I am seeing on cygwin are specific to cygwin or if they are caused by something else. Very good tool to have.
It's OK if clang isn't supported if gcc is. I'm not sure anyone has compiled OpenCog with clang recently anyway.
I was able to get cogutils to build on cygwin with only minor changes, and will do a pull request after the unit tests pass. I just need to find an equivalent for cxxtest which runs on cygwin, since cxxtest doesn't seem to be available. (Or I will just try to compile from source first)
Added cxxtest to /usr/local/include in cygwin, added cxxtest's bin folder to PATH, and cxxtest now works. Current status is that compilation is failing here:
cogutils/tests/util/iostreamContainerUTest.cxxtest:34:22: error: expected unqualified-id before numeric constant
Will review to see what changes to preprocessor directives would be needed for compilation under cygwin to continue
Compilation has succeeded and unit tests pass when the DLL is in the correct location. I am having trouble setting the output of the cygcogutils.dll to /usr/local/bin, when it is in /usr/local/lib right now. Or to be more specific, the /usr/local/lib folder is not searched for DLL files.
The fix would be either to
A) Set the lib output location to /usr/local/bin (may require a later version of CMake or a lengthy hack as I was unable to figure out how to change the target location for individual types of files)
B) Make sure that cygwin recognizes /usr/local/lib as a dll path.
The main problem I see with A is that it may be a bad practice to keep dll files in the bin folder if they are supposed to be in the lib folder. If the standard way of doing things is to keep dlls in /usr/local/lib, then deviating from this is a form of technical debt, leading to compatibility issues, and developers/users scratching their heads in frustration.
The main problem I see with B is that it may require changing global config files, which I try to avoid if possible since you end up with a higher risk for compatibility issues. Changing an environment variable that was in the form of a list of paths would be less of an issue but I had no luck when using LD_LIBRARY_PATH or PATH. There's also the possibility that the "way to do things" is in fact to use /usr/local/bin on Cygwin, but I will have to do some more research.
It seems that A is the lesser evil here since it seems more likely to be compatible with the most amount of users. But if I can find an environment variable (and not a global config file) that fixes B, then that would be ideal.
Cygwin doesn't support RPATH, because it's part of the ELF executable format (Linux). The linker functions within CMake simply won't do anything in Cygwin.
The best way to solve this is to ensure that the DLL files are located in the same folder as whatever executable runs it. This is mostly an issue for the unit tests since they need to be in an arbitrary folder.
In the context of make install, I don't see any issue with moving the dll to /usr/local/bin (for cygwin builds) since it is clear that /usr/local/lib is NOT the way to go with cygwin, citing the lack of RPATH.
I tried to run a copy command in cmake to copy the .dll file to the tests folder, but the problem is that this command only runs with cmake and not with make itself. I'll try to set up a duplicate binary path for that dll, if possible.
To get cogutils to build from on cygwin. note: these directions do not include installing dependencies other than cxxtest
- clone repository
- for tests, download cxxtest and extract anywhere. go into that folder and copy cxxtest to /usr/local/include. 'make test' should succeed after that.
- go through make process as normal
- during installation('make install') cygcogutil.dll and libcogutil.dll.a are installed under usr\local\bin\opencog . move them to usr/local/include/opencog. (this step needs to remedied via an update cmake file)
all tests succeeded.
Better support for OpenCog on windows will arrive over the next couple of years due to better OS virtualization technology that is rapidly becoming available. Microsoft already has a virtualized linux kernel in beta, which it is supporting officially. The performance is likely to be on the same order of magnitude, as long as there is enough RAM for both the host and guest environments.
This would IMO provide the best of both worlds, removing the need to port OpenCog (which is an expensive thing to do in terms of man hours), and it provides a path to use OpenCog on Windows without needing to maintain your own Linux installation.
Nitpick: DLL should never be installed in either /usr/local/include more /usr/local/bin/ They should always go into /usr/local/lib -- lib is for libraries (dlls), include is for include files (header files) and bin is for binaries 9executables)
Yes, I know that was the wrong folder to place it in. However, that's where atomspace is looking.
Yes, Windows 10 anniversary will bring a fully functional ubuntu virtualization. I have been using it in beta. ubuntu and windows share the same system resources. i'm currently running an opencog server using that right now. So yes, opencog can be ran on windows 10 anniversary using this technology but it can not be ran on any other version of windows.
Open-source virtualization tools of similar nature are in the works too, but older versions of Windows might not have the sophistication necessary for it anyway.