Compiling for Windows
hwjeremy opened this issue · comments
Hi @migueldeicaza, your work is amazing, and I'm very much enjoying exploring SwiftGodot/SwiftGodotKit.
I've had some success using SwiftGodotKit to start a new project on MacOS. Driving Godot from Swift is fantastic. However, I need the project to run on Windows as well.
It seems that the core issue preventing use of SwiftGodotKit on Windows right now is the lack of a Windows version of libgodot. Might the solution be as simple as compiling libgodot on Windows as a DLL, and using it in place of the MacOS DyLib, retaining the same umbrella header? If so, I'm game to have a go - But I'm guessing there are some gotchas and nuances I should be aware of before I go down that rabbit hole.
Let me know how I can help.
There are no Mac dependencies on libgodot, it is a pretty straightforward patch.
Would love for someone to look into compiling it, and helping me sort out the CI so we can publish binaries for Windows people. I am happily doing the Mac version, but I don't have a Windows machine around.
@migueldeicaza I have a Windows machine available - I'd be happy to help, I just need some tips to get started.
I've cloned your libgodot repo and compiled it on Windows via scons. I get the standard Godot editor executable output in /bin. I'm guessing your libgodot fork of Godot is modified to produce a DLL somehow, equivalent to the DyLib you generate for Mac. Do you provide some extra command line parameter to scons perhaps? If I know how you do it for Mac, I might be able to work backwards and do the equivalent on Windows.
I know from your perspective this must sound very silly - I'm sure there's something obvious I'm missing there.
Good question!
I use the script in SwiftGodotKit/Scripts/make-libgodot.xcframework to make the MacOS binary, but the core component is this command line invocation:
scons platform=macos target=template_debug library_type=shared_library debug_symbols=yes
The script does thing like putting it in a format that is convenient for Xcode, and builds for two platforms - so you can really ignore all of that, but it is a good reference to keep in mind.
Let me know if that unblocks you.
Thanks Miguel that's definitely set me on a productive path. I'll report back here with findings over the coming days. Very determined to get this working: I'd love to be able to do cross-platform game development in Swift.
Progress update: What I'm about to share will probably sound a bit obvious to those more well versed in Windows library development than myself.
Swift's C++ interoperability depends on Clang C++ Modules (not to be confused with C++20 modules). If you observe the structure of the libgodot.xcframework produced by Miguel's make-libgodot.xcframework tool, you'll notice it has a module.modulemap file among other attributes alongside the .dylib itself.
The structure of the .xcframework, conducive to C++ interoperability, is created by xcodebuild -create-xcframework -library libgodot.dylib -headers $tmp -output libgodot.xcframework inside the make-libgodot.xcframework tool.
It's not obvious to me how a similar structure could be created on Windows. There's no equivalent of xcodebuild -create-xcframework on Windows. When compiling on Windows, MSVC is used, and it's also not clear to me whether the output DLL can be used in a manner consistent with Clang's "Modules" for Swift C++ interoperability.
In summary, it seems I've walked onto the bleeding edge of Swift-on-Windows. The crucial question seems to be - How do we create a Clang-module compatible DLL on Windows, and then package that DLL in an equivalent manner to a .xcframework, such that it can be made available to Swift's C++ interoperability features.
I'll continue my research.
Hello
Just a note that LibGodot does not use c++ interoperability at all. What it uses is the old C interoperability.
That one should be working, but I don't know how one would use that
Thanks for nudging me back in the right direction Miguel.
I've opened a discussion on the Swift Forums and @compnerd has kindly been replying with more information with respect to making the DLL available to Swift on Windows.
With yours and Saleem's help, and some serious learning on my part, I bet this challenge will be solved.
Oh that discussion looks like it is heading in the right direction!
Keep me posted, this is exciting
Another progress update, the following is effectively a mirror of the latest post I've made on the Swift Forums, but I thought it good to keep this thread updated as well.
Miguel, I thought you might be interested in the references to "plugins" in the compilation failure, as I believe you've performed some plugin wizardry to get SwiftGodot working (though, like so many things on this adventure, I might have a misunderstanding about the meaning of the word "plugin" in this context!).
I've managed to prepare a libgodot.dll and associated .lib, .pdb. I'll write up notes on that preparation another time, but for now, I'll focus on the results. The library structure on Windows (a rough analogue of the .xcframework containing a dylib) is as follows:
libgodot/
├── bin/
│ └── libgodot.dll
├── lib/
│ └── libgodot.lib
├── pdb/
│ └── libgodot.pdb
└── include/
└── libgot/
├── gdextension_interface.h
└── libgodot.h
└── module.modulemap
... with modulemap:
module libgodot {
header "libgodot/libgodot.h"
export *
}
I provided the libgodot library to SwiftGodotKit package.swift as a .systemLibrary target:
.systemLibrary(
name: "libgodot",
path: "libgodot/include
)
And then invoked swift with arguments:
swift build -Xlinker -I"libgodot/include" -Xlinker -L"libgodot/lib"
Compilation proceeded as follows:
Building for debugging...
Build complete! (0.76s)
warning: unable to create symbolic link at C:\Users\Hugh\Desktop\SwiftGodotKit-main\.build\plugins\debug: Error Domain=NSCocoaErrorDomain Code=256 "(null)"
Building for debugging...
error: compile command failed due to exception 3 (use -v to see invocation)
With the following error output, the "output" body of which I've snipped due to its verbosity:
error: failed parsing the Swift compiler output: unexpected JSON message: {
"exception" : 3,
"kind" : "abnormal-exit",
"name" : "compile",
"output" : "\u001b[1m<unknown>:0: \u001b[0m\u001b[0;1;31merror: \u001b[0m\u001b[1merror opening 'C:\\Users\\Users\\Hugh\\Desktop\\SwiftGodotKit-main\\.build\\plugins\\outputs\\swiftgodot\\SwiftGodot\\CodeGeneratorPlugin\\GeneratedSources\\generated.swiftdeps' for output: no such file or directory\r\n\u001b[0mAssertion failed: loadedGraph.has_value() && \"Should be able to read the exported graph.\", file C:\\Users\\swift-ci\\jenkins\\workspace\\oss-swift-windows-toolchain\\swift\\lib\\AST\\FineGrainedDependencies.cpp, line 237\r\nPlease submit a bug report (https:\/\/swift.org\/contributing\/#reporting-bugs) and include the crash backtrace [ **snip** ],
"pid" : 32024,
"process" : {
"real_pid" : 32024
}
}: dataCorrupted(Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "kind", intValue: nil)], debugDescription: "invalid kind", underlyingError: nil))
For now, I'm going to focus on warning: unable to create symbolic link at C:\Users\Hugh\Desktop\SwiftGodotKit-main\.build\plugins\debug: Error Domain=NSCocoaErrorDomain Code=256 "(null)" - I imagine I might be missing something very simple here. Any thoughts, leads, ideas, input, or guidance is most welcome.
Can you try the “subst” trick?
Your path might be too long.
The pdb should be collapsed into bin as that makes debugging easier. The default behaviour for windbg/x is to look next to the dll for the PDB.
The modulemap should be sunk a layer lower into libgodot or the headers hoisted a layer into include to best match expectations, though it does not matter much. This is even more true on Windows where there is no unified include search path and therefore the module.modulemap cannot collide with co-installed libraries (consider Linux where you may have GTK+ and GLib headers both installed into /usr/include, which would prevent you from having individual module maps for each residing in /usr/include even though the headers for GTK+ would go to something like /usr/include/gtk and GLib would install to /usr/include/glib-2.0).
The symbolic link warning is a red herring, it is not fatal. However, that indicates that you do not the SeCreateSymbolicLinkPrivilege privilege. You can either assign that to yourself via a GPO if you are running Windows 10/11 Pro (unless you want to do some other trickery to inject the MUMs into the Windows image to gain gpedit.msc) or you can enable Developer Mode (reducing security) on any of the SKUs.
@migueldeicaza that is an interesting thought, I'm not sure that it is the cause of this particular failure, though I agree with you that subst'ing the drive is generally recommended practice.
$ echo 'C:\\Users\\Users\\Hugh\\Desktop\\SwiftGodotKit-main\\.build\\plugins\\outputs\\swiftgodot\\SwiftGodot\\CodeGeneratorPlugin\\GeneratedSources\\generated.swiftdeps' | wc -c
149149 << 261
error: error opening 'C:\Users\Users\Hugh\Desktop\SwiftGodotKit-main\.build\plugins\outputs\swiftgodot\SwiftGodot\CodeGeneratorPlugin\GeneratedSources\generated.swiftdeps' for output: no such file or directory
Assertion failed: loadedGraph.has_value() && "Should be able to read the exported graph.", file C:\Users\swift-ci\jenkins\workspace\oss-swift-windows-toolchain\swift\lib\AST\FineGrainedDependencies.cpp, line 237
Seems like the swiftdeps didn't get written? It is unclear what the invocation was, but that should be written by the driver 🤔 . Perhaps building with -v -j 1 would illuminate what was executed and what might be going wrong?
Miguel, Saleem, Thank you both for your input - It's tremendously motivating to have you both throwing ideas into the ring. We continue to advance - here's a progress update.
First, I granted the SeCreateSymbolicLinkPrivilege privilege to my Windows user, using secpol.msc (Settings -> Local Policies -> User Rights Assignment).
Next, I used subst to reduce the lengths of paths. For those watching, subst allows you to map a path to a virtual drive letter in Windows. So, I executed the following...
subst S: C:\Users\Hugh\Desktop\SwiftGodotKit-main ... to map my working SwiftGodotKit directory to a temporary S: drive. I then performed:
swift package clean
swift build -Xlinker -I"libgodot\include" -Xlinker -L"libgodot\lib"
... and, voila! Build complete! (452.70s) 🎉 (more on that build time in a little bit). That seems to me to be a huge win.
During that build process, the terminal was spammed with LNK4217 ("locally defined symbol imported") warnings. It seems there is quite a bit of discussion on this phenomenon already, e.g.:
Swift on windows linking msvc dll errors
Linker warnings on Windows
Enabling Static Linking on Windows
... and a few more, with great input from Saleem. My initial reading indicates that this phenomenon explains the long build times. I will read these threads in depth and work to resolve these warnings.
Regardless, I decided to proceed to attempt to build my "minimum example" project, which is extremely similar to Miguels's "trivial example" - It simply attempts to draw a cube.
The result is, as expected, the same LNK4217 spam as when building SwiftGodotKit individually. And at the end, the following failure:
lld-link: error: undefined symbol: libgodot_gdextension_bind
>>> referenced by M:\.build\x86_64-unknown-windows-msvc\debug\SwiftGodotKit.build\SwiftGodotKit.swift.o:($s13SwiftGodotKit03runB04args8initHook9loadScene0H15ProjectSettings7verboseySaySSG_y0aB011GDExtensionC19InitializationLevelOcyAI0I4TreeCcyAI0jK0CcSbtF)
lld-link: error: undefined symbol: godot_main
>>> referenced by M:\.build\x86_64-unknown-windows-msvc\debug\SwiftGodotKit.build\SwiftGodotKit.swift.o:($s13SwiftGodotKit03runB04args8initHook9loadScene0H15ProjectSettings7verboseySaySSG_y0aB011GDExtensionC19InitializationLevelOcyAI0I4TreeCcyAI0jK0CcSbtFySpySpys4Int8VGSgGSgXEfU1_)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
As discussed earlier, the whole linking process is my greatest knowledge deficiency in working this problem. So any ideas or comments anyone has are most welcome and would be very helpful. My next steps are to thoroughly read the forum threads referenced above, and learn more about LNK4217 and its manifestation in the Swift on Windows context, as I suspect it is related to the lld-link: error that presents when attempting to build an executable.
I'm very excited at this progress, and I imagine that with a but more research into the linking process, I will be able to clear what feels like the final hurdle.
Note that the LNK4217 warnings are just that - warnings. The issue is in the serialization or deserialization of the missile (I'm uncertain which). However, in your case, you are building with SPM which does not correctly build Swift code and the warnings are expected. If you wish to minimize (Abdulrasool hopefully soon erase) them, you worked need to use CMake for the time being.
those symbols that are missing are part of libgodot.
What Godot are you using to build?
you must use the libgodot fork from my repositories (it is just plain Godot with the PR for Libgodot merged into it)
Oh I realize something: on windows you might need to declare the public entry points in some kind of public manifest. So those two symbols need to be listed somewhere. I believe windows uses DEF files for that, but that’s the extent of my Windows linker knowledge.
Thanks for that tip off Miguel, I'll have an opportunity to try adding a DEF file later this week.
I can confirm am using your libgodot fork to build the Windows DLL.
This is a bug in the implementation of godot.
The function is properly attributed with GODOT_API which is defined at:
This needs to be conditional on an additional macro so that it is defined to __declspec(dllimport) when it is consumed by the Swift code. CMake would have defined core_EXPORTS to configure the behaviour as:
#if defined(_WINDLL)
#if defined(core_EXPORTS)
#define GODOT_API __declspec(dllexport)
#else
#define GODOT_API __declspec(dllimport)
#endif
#else
#define GODOT_API
#endifUpdate: I don't have much time to look at this during the holiday period, but I'm excited to continue this thread in the new year.
I've attempted to quickly add a DEF file, but have been running in to some obscure behaviour when compiling. I'm still gathering notes before I post back here.
compnerd, I'll also try to grok your contribution and make the suggested changes.