Failure to produce offline documentation if you consume C++ libraries via Swift C++/Interop

[Miguel de Icaza]

If you create a project that references a C++ library with the new swiftSettings: [.interoperabilityMode(.Cxx)]
mode in Package.swift, the system fails to generate symbols, which causes it to silently fail and produce
an incomplete documentation package.

Additionally, there is no warning issued, so you only find out about these problems much later on, from the
Web interface that merely produces an "Unknown error" message on the web.

This is new ground, I am using Xcode 15, beta 7, but the problem also happens with Xcode 15, beta 5.

Checklist

• If possible, I've reproduced the issue using the main branch of this package.
• This issue hasn't been addressed in an existing GitHub issue.

Expected behavior

I have linked a simple repro here

Using this command should produce working documentation, in particular, you can tell if it worked because the directory /tmp/docs/documentation is present:

Clone the repro:

$ git clone https://github.com/migueldeicaza/SwiftDocCPlusPlusBug.git

When you run generate-documentation targeting /tmp/docs as the output directory:

$ swift package --allow-writing-to-directory /tmp/docs generate-documentation --target demo --disable-indexing --transform-for-static-hosting --hosting-base-path /SwiftNavigation --output-path /tmp/docs --diagnostic-level information

This should produce a directory in /tmp/docs/documentation and the symbols in .build/arm64-apple-macosx/extracted-symbols/demo/demo/demo.symbols.json

Actual behavior

Notice that /tmp/docs/documentation does not exist, and there is a warning:

/private/tmp/demo/Sources/demo/demo.docc/Documentation.md:1:3: warning: No symbol matched 'demo'. Can't resolve 'demo'.

Additionally the .build/arm64-apple-macosx/extracted-symbols/demo/demo directory is empty and lacks the demo.symbols.json

Steps to Reproduce

Small test case is here:

https://github.com/migueldeicaza/SwiftDocCPlusPlusBug

Instructions to reproduce the issue once cloned:

swift package --allow-writing-to-directory /tmp/docs generate-documentation --target demo --disable-indexing --transform-for-static-hosting --hosting-base-path /SwiftNavigation --output-path /tmp/docs --diagnostic-level information

Swift-DocC Plugin Version Information

Swift-DocC plugin version: 26ac575 (1.3.0)
Swift Compiler version: swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.106 clang-1500.0.40.1)

[Joseph Heck]

Jumping in sideways a bit here - I haven't done the internal analysis, but I suspect this is related to swiftlang/swift-package-manager#6162 (and in turn #50), where Docc is relying in swift package manager to "provide all the relevant symbols" - but there's some edge cases where its unable to provide them.

The gist in my debugging found that there we assumptions and potentially multiple paths in SwiftPM for getting the symbols, and the paths diverged when compiling swift vs. using libraries. There were some comments in SwiftPM source that talked to "combining paths down the road", which I suspect is fundamentally part of this issue. If I read the code right (no promises there), the "quick path" for libraries expected relevant module caches to have been populated already from all the various libraries, and process which didn't hold up when using the static library from a binaryTarget. I suspect C++ interop is in the same relative boat, although there may be more blocks - if they're C++ symbols, I've no idea if those have representations yet in the symbol graphs.

[Miguel de Icaza]

While running swift package dump-symbol-graph works with the attached sample, which does not do much of any C++ at all, the same command breaks with my actual codebase that uses C++, like this:

Building for debugging...
[2/2] Compiling plugin Swift-DocC Preview
Build complete! (6.61s)
-- Emitting symbol graph for SwiftNavigation
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "/Users/miguel/cvs/SwiftNavigation/Sources/CRecast/include/Bridging.h"
        ^

[Rauhul Varma]

You may want to give this another try with the Swift nightly toolchain. I was running into similar bugs in Swift-MMIO and hopefully fixed the issues in SwiftPM / swift-symbol-graph extract which prevented generating docs with cxx in the module graph