v4.0.1 breaks LLDB in Xcode

Question

v4.0.1 breaks LLDB in Xcode

andyyhope opened this issue a year ago · comments

We're using the latest version (4.0.1) via SPM and it has broken our ability to use LLDB. Upon further inspection, it appears that the included binary twilio-conversations-ios-4.0.1.xcframework is exporting absolute search paths for the SDKStatCaches.noindex directory. For example, when we try to po a variable, the following error is shown:

(lldb) po myString
error: expression failed to parse:
error: stat cache file '/Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/SDKStatCaches.noindex/iphonesimulator16.4-20E238-.sdkstatcache' not found
error: stat cache file '/Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/SDKStatCaches.noindex/iphonesimulator16.4-20E238-.sdkstatcache' not found

error: couldn't IRGen expression. Please check the above error messages for possible root causes.

Possible solutions:

Update the project to include the following flag so absolute search paths aren't used:

OTHER_SWIFT_FLAGS = "-Xfrontend -no-serialize-debugging-options"

mthormann-dhi · Answer 1 · Thu Jul 20 2023 03:29:00 GMT+0800 (China Standard Time)

Saw this as well. #36 also but don't think anything was ever done on it. @andyyhope Did that workaround work for you w/o side effects?

Andyy Hope · Answer 2 · Thu Jul 20 2023 09:21:46 GMT+0800 (China Standard Time)

I've actually tried a bunch of things to get a workaround on this but without any success:

LLDB

Added an LLDBinit file to redirect the distiller path to a local one:

settings set target.source-map /Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/SDKStatCaches.noindex/iphonesimulator16.4-20E238-.sdkstatcache /Users/andyyhope/Library/Developer/Xcode/DerivedData/SDKStatCaches.noindex/iphonesimulator16.4-20E238-.sdkstatcache /Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/Build/Intermediates.noindex/ArchiveIntermediates/TwilioConversationsClient/IntermediateBuildFilesPath/TwilioConversationsClient.build/Release-iphonesimulator/TwilioConversationsClient.build/all-product-headers.yaml /Users/andyyhope/Downloads/TestTwilioLLDBWorkspace/TestTwilioLLDB/all-product-headers.yaml

VFS Overlays

Added a VFS overlay to the swift compile flags

-Xcc -ivfsoverlay -Xcc vfsoverlay.yaml

{
  'version': 0,
  'roots': [
    { 
      'name': '/Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/SDKStatCaches.noindex', 
      'type': 'directory',
      'contents': [
        { 
          'name': 'iphonesimulator16.4-20E238-.sdkstatcache', 
          'type': 'file',
          'external-contents': 'SDKStatCaches.noindex/iphonesimulator16.4-20E238-.sdkstatcache'
        },
      ]
    },
    { 
      'name': '/Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/Build/Intermediates.noindex/ArchiveIntermediates/TwilioConversationsClient/IntermediateBuildFilesPath/TwilioConversationsClient.build/Release-iphonesimulator/TwilioConversationsClient.build', 
      'type': 'directory',
      'contents': [
        { 
          'name': 'all-product-headers.yaml', 
          'type': 'file',
          'external-contents': 'all-product-headers.yaml'
        },
      ]
    },
  ]
}

Added compiler flags

To redirect distiller paths to my own

-fdebug-prefix-map=/Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/SDKStatCaches.noindex=/Users/andyyhope/Downloads/TestTwilioLLDBWorkspace/TestTwilioLLDB/SDKStatCaches.noindex

Modifying the `twilio-conversations-ios-4.0.1` binary

Removed the references to dSYMs and dSYM files altogether
Modified the actual binary files to swap paths to distiller with my own

Ryan Waggoner · Answer 3 · Wed Jul 26 2023 02:05:05 GMT+0800 (China Standard Time)

I'm also experiencing this issue and would very much appreciate a fix.

Berkus Decker · Answer 4 · Mon Jul 31 2023 21:35:54 GMT+0800 (China Standard Time)

4.0.2 is cooking folks.

Berkus Decker · Answer 5 · Thu Aug 03 2023 20:56:22 GMT+0800 (China Standard Time)

Fixed by 4.0.2.

Andyy Hope · Answer 6 · Fri Aug 04 2023 08:07:30 GMT+0800 (China Standard Time)

@berkus Sadly, I'm still seeing the issue. What changes did you make to the binary?

Berkus Decker · Answer 7 · Fri Aug 04 2023 19:39:01 GMT+0800 (China Standard Time)

@berkus Sadly, I'm still seeing the issue. What changes did you make to the binary?

Interesting, I've tested both 4.0.1 and 4.0.2 in xcode; with 4.0.1 the lldb debugging was failing with the error you've posted above. With 4.0.2 it actually worked.

Changes to the binary include setting SWIFT_SERIALIZE_DEBUGGING_OPTIONS = NO in user build flags. I've confirmed that build then uses -Xfrontend -no-serialize-debugging-options when building.

Do you have more details? Can you confirm that you're actually using 4.0.2? Xcode seems very stubborn on updating the deps properly.

mthormann-dhi · Answer 8 · Fri Aug 04 2023 20:13:25 GMT+0800 (China Standard Time)

Do we have to clear SPM cache locally to see the change possibly?

Andyy Hope · Answer 9 · Fri Aug 04 2023 21:17:41 GMT+0800 (China Standard Time)

@berkus Sadly, I'm still seeing the issue. What changes did you make to the binary?

Interesting, I've tested both 4.0.1 and 4.0.2 in xcode; with 4.0.1 the lldb debugging was failing with the error you've posted above. With 4.0.2 it actually worked.

Changes to the binary include setting SWIFT_SERIALIZE_DEBUGGING_OPTIONS = NO in user build flags. I've confirmed that build then uses -Xfrontend -no-serialize-debugging-options when building.

Do you have more details? Can you confirm that you're actually using 4.0.2? Xcode seems very stubborn on updating the deps properly.

We have a test project that is able to reproduce the issue. I'll send it to you first thing Monday (Australia time)

Andyy Hope · Answer 10 · Fri Aug 04 2023 21:18:12 GMT+0800 (China Standard Time)

@berkus We have a test project that is able to reproduce the issue. I'll send it to you first thing Monday (Australia time)

casr-anz · Answer 11 · Mon Aug 07 2023 09:38:49 GMT+0800 (China Standard Time)

I'm also still seeing the problem. I've removed DerivedData, ~/Library/Caches/org.swift.swiftpm and ~/Library/Developer/swift-sh.cache and also confirmed the TwilioConversationsClient binary being used is the 4.0.2 one.

(Aside the Info.plist for the updated framework still has CFBundleShortVersionString set to 2.1.0 rather than 4.0.2...)

Here's the test project @andyyhope referred to TestTwilioLLDBWorkspace.zip - it's also in Support Ticket 12867609.

Update the TwilioDependencies/Package.swift to use 4.0.2, then build and run TestTwilioLLDB.xcworkspace's TestTwilioLLDB. Set a breakpoint in AppDelegate's didFinishLaunchingWithOptions and try to po application.

Andyy Hope · Answer 12 · Mon Aug 07 2023 09:51:27 GMT+0800 (China Standard Time)

OK. So I'm able to repro this reliably with a test project on Xcode 14.3.1 but also noticed that Xcode 15b5 is able to resolve this issue (albeit with a bit of lag) but we can't really rely on that because there are a few things holding us back from adopting Xcode 15 at launch.

Check out @casr-anz's project above for repro or mine:
https://github.com/andyyhope/twilio-conversations-lldb

Berkus Decker · Answer 13 · Mon Aug 07 2023 16:38:06 GMT+0800 (China Standard Time)

Aside the Info.plist for the updated framework still has CFBundleShortVersionString set to 2.1.0 rather than 4.0.2...)

ouch

Berkus Decker · Answer 14 · Mon Aug 07 2023 16:39:05 GMT+0800 (China Standard Time)

@andyyhope @casr-anz thanks folks, I am checking it out now.

Berkus Decker · Answer 15 · Mon Aug 07 2023 19:59:26 GMT+0800 (China Standard Time)

Hm. I've changed TwilioDependencies/Package.switft to depend on 4.0.2 before opening it with Xcode.
Opened in Xcode.
Built - po application failed with the issue we know.
THEN i've changed the dependency back to 4.0.1, rebuilt and this is the result:

Andyy Hope · Answer 16 · Tue Aug 08 2023 08:06:28 GMT+0800 (China Standard Time)

quick sanity check. Does /Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/SDKStatCaches.noindex/iphonesimulator16.4-20E238-.sdkstatcache exist on your local machine?

Berkus Decker · Answer 17 · Tue Aug 08 2023 17:54:34 GMT+0800 (China Standard Time)

ls /Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/SDKStatCaches.noindex/iphonesimulator16.4-20E238-.sdkstatcache
ls: /Users/distiller/rtd-sdk-monorepo/sdk/ios/Local/temp/DerivedData/SDKStatCaches.noindex/iphonesimulator16.4-20E238-.sdkstatcache: No such file or directory

This user exists only on CI boxes.

Berkus Decker · Answer 18 · Tue Aug 08 2023 17:55:44 GMT+0800 (China Standard Time)

Additional info: if I open project as-is, with 4.0.1 as dependency, then debug fails the same way after build.

Berkus Decker · Answer 19 · Fri Aug 11 2023 18:46:00 GMT+0800 (China Standard Time)

After further investigation:

You need to delete the dSYMs from the framework bundle for it to work
Until a patched xcode is released, -no-serialize-debugging-options option doesn't seem to have any effect.

casr-anz · Answer 20 · Mon Aug 21 2023 09:26:54 GMT+0800 (China Standard Time)

Thanks @berkus
Could you clarify exactly which dSYMs you needed to delete for it to work? (Or were you saying that without an updated Xcode even deleting the dSYMs doesn't work?)

With our project I've just tried removing all the Twilio dSYMs to no avail:

Quit Xcode
Remove DerivedData
Clear out ~/Library/Caches/org.swift.swiftpm and ~/Library/Developer/swift-sh.cache
Configure our Package.swift to use 4.0.2
Resolve packages from the command-line
Remove all Twilio dSYMs find . -name "Twilio*.framework.dSYM" -exec rm -r \{\} \; from inside DerivedData
Open Xcode, build and run, po still fails
(Confirm there are no dSYMs still in DerivedData find . -name "Twilio*.framework.dSYM")

There are still lots of these kinds of references to the CI user's directory in these files in the Twilio framework bundles, which may be the culprits though:
"filePath": "\/Users\/distiller\/rtd-sdk-monorepo\/sdk\/ios\/convo\/TwilioConversationsClient\/Sources\/TCHParticipant+Extension.swift"
in files like TwilioConversationsClient.xcframework/ios-arm64_x86_64-simulator/TwilioConversationsClient.framework/Modules/TwilioConversationsClient.swiftmodule/x86_64-apple-ios-simulator.abi.json

Jesus (iOS) · Answer 21 · Tue Aug 22 2023 03:53:22 GMT+0800 (China Standard Time)

@andyyhope which was the change to fix it?

This? OTHER_SWIFT_FLAGS = "-Xfrontend -no-serialize-debugging-options" or this SWIFT_SERIALIZE_DEBUGGING_OPTIONS = NO

Berkus Decker · Answer 22 · Tue Aug 22 2023 16:21:55 GMT+0800 (China Standard Time)

@jesus-mg-ios if you just override using OTHER_SWIFT_FLAGS then the old setting of "-Xfrontend -serialize-debugging-options" remains active in the command line in addition to the other flags, so that doesn't work well. Setting SWIFT_SERIALIZE_DEBUGGING_OPTIONS = NO on the other hand cleans up the command line nicely.

Jesus (iOS) · Answer 23 · Wed Aug 23 2023 14:56:42 GMT+0800 (China Standard Time)

So @berkus, with SWIFT_SERIALIZE_DEBUGGING_OPTIONS = NO the issue is solved, doesn't it?

mthormann-dhi · Answer 24 · Wed Aug 23 2023 20:10:28 GMT+0800 (China Standard Time)

@jesus-mg-ios Unless I'm missing something, still feels like this issue needs to be resolved inside Twilio with a new release of the SDK? This really just boils down to an incorrectly packaged/configured SPM package. Workarounds like this are appreciated but shouldn't be expected for a third party SDK.

Andyy Hope · Answer 25 · Fri Aug 25 2023 09:51:12 GMT+0800 (China Standard Time)

@berkus are you sure that the -no-serialize-debugging-options setting you're inputting isn't being overwritten later in the stack or ? Have you tried compiling the Xcframework from the command line?

Try doing a grep search for:

grep -F -r . -e "OTHER_SWIFT_FLAGS"

and check to see it's being set everywhere

Berkus Decker · Answer 26 · Mon Sep 18 2023 20:41:44 GMT+0800 (China Standard Time)

So @berkus, with SWIFT_SERIALIZE_DEBUGGING_OPTIONS = NO the issue is solved, doesn't it?

Not really. Even when sdk is compiled with the correct settings xcode manages to go crazy.

Berkus Decker · Answer 27 · Mon Sep 18 2023 20:42:39 GMT+0800 (China Standard Time)

@berkus are you sure that the -no-serialize-debugging-options setting you're inputting isn't being overwritten later in the stack or ? Have you tried compiling the Xcframework from the command line?

Try doing a grep search for:
grep -F -r . -e "OTHER_SWIFT_FLAGS"
and check to see it's being set everywhere

Yeah i believe I’ve checked everything. I was grepping the logs from the CI cli build and it seems all options were propagated fully.

Berkus Decker · Answer 28 · Tue Sep 26 2023 20:31:42 GMT+0800 (China Standard Time)

@andyyhope @mthormann-dhi @ryanwaggoner @casr-anz @jesus-mg-ios now that Xcode 15 is out, does it work for you with the regular conversations package to debug?

Berkus Decker · Answer 29 · Tue Sep 26 2023 22:36:32 GMT+0800 (China Standard Time)

Tested in Xcode 15, seems to work fine.

casr-anz · Answer 30 · Wed Sep 27 2023 08:16:01 GMT+0800 (China Standard Time)

Hi @berkus - yes, it does seem to work fine now. Both 4.0.1/4.0.2 with my test project and our production app seem to like LLDB again with Xcode 15.

Thanks for the reminder to re-check. (Can now get rid of various /Users/distiller sym-links that were working around the problem).

Berkus Decker · Answer 31 · Wed Sep 27 2023 20:04:50 GMT+0800 (China Standard Time)

I am going to close this as fixed, since it clearly was a bug in Xcode 14.3.1.

v4.0.1 breaks LLDB in Xcode

Possible solutions:

LLDB

VFS Overlays

Added compiler flags

Modifying the twilio-conversations-ios-4.0.1 binary

Modifying the `twilio-conversations-ios-4.0.1` binary