Corrupted data with Swift 2.0 / Xcode 7 beta 3

[Canis Lupus]

So, I'm not sure how much this is a FastCoding issue vs a Swift 2.0 issue, but I figured you'd want to know about it either way:

I've been migrating a project to Swift 2.0, I skipped the first couple of betas but dived into Xcode 7 beta 3, and I've run into an issue where data is being corrupted when passed from Swift into FastCoding.

What appears to be happening is that the caching system is incorrectly identifying cache hits, and writing aliases to the previous entries, when in fact the data is different.

I'm not familiar enough with the internals of FastCoding or Swift to be sure, but I think this may be because Swift is wrapping some of its data structures in temporary objects when they get passed to Obj-C, and those might get recycled within one archiving operation?

This gist shows a minimal repro for the issue (tested with FastCoding version 3.2.1, Swift version 2.0 swiftlang-700.0.45 clang-700.0.57.2)

https://gist.github.com/Canis-UK/2409f367bbef978b43c3

If you create a new iOS Single View Application project in Xcode 7b3, and paste the gist code into its ViewController.swift, it'll attempt to encode and then decode a test dictionary (containing a couple of TestObject instances) when the app is launched, and compare what it decoded against the original.

If you flip the #define to use NSKeyed(Un)Archiver, it's fine -- but with FastCoder the two separate object instances end up with identical values.

The encoder for TestObject writes an array, which gets passed into FastCoder's encodeObject:forKey: as type "Swift._SwiftDeferredNSArray", which I'm guessing is a handle that points to the original Swift array and creates an NSArray on-demand with a reference to the same data; this deferred object then turns up in the cache -- a breakpoint in FCWriteObjectAlias shows it getting hit on the second instance.

(Irritatingly, certain debugging explorations cause the code to work correctly, making this tricky to discover. I'm guessing that poking at the objects in the debugger can cause the deferred objects to be instantiated or something like that, but I'm kinda shooting in the dark on that one. Just giving you a heads up that its a bit of a Heisenbug)

If it turns out to be strictly a Swift bug, I'll cheerfully file a radar :) but other folks using FastCoding should probably be warned!

[Canis Lupus]

Some additional notes on reproing this:

• To repro, you also need to (obviously) add FastCoder.m/.h to the project along with a Swift bridging header that includes FastCoder.h
• As recommended, I set it to compile with ARC disabled
• I was testing on the "iPhone 6" simulator, not that it likely makes any difference (but it was a 64-bit Intel build, anyway).

Having poked around in FastCoder some more, I think I see what's happening. The CFDictionary in which objects are cached is initialised with null keyCallbacks, so the objects used as keys are just treated as bare pointers. If the objects get freed after being cached, of course those pointer addresses may be reused by subsequent allocations. (Turning on the Xcode 7 Address Sanitiser made the problem go away, suggesting this is indeed the issue: Mike Ash writes that amongst other things "Address Sanitizer defends against [use-after-free errors] by placing newly freed memory into a recycling queue that keeps it unallocated for a while before it can be reused.".

My guess is that previously, the issue wasn't arising because the objects were autoreleased, and thus kept alive until the autoreleasepool terminated, which would be beyond the scope of the coding operation (either the pool in -dataWithRootObject: or one outside it depending on when the objects were created), but I'm guessing that now Swift 2 is more aggressive about optimising object lifetimes, doing an immediate release on them in more circumstances.

As a temporary hack, I've modified FCCacheWrittenObject():

static inline NSUInteger FCCacheWrittenObject(__unsafe_unretained id object, __unsafe_unretained NSMutableDictionary *cache)
{
    NSUInteger count = (NSUInteger)CFDictionaryGetCount((CFMutableDictionaryRef)cache);
    CFDictionarySetValue((CFMutableDictionaryRef)cache, (const void *)(object), (const void *)(count + 1));
#if FC_RETAIN_RELEASE_HACK_ENABLED
    [[object retain] autorelease];
#endif
    return count;
}

...where FC_RETAIN_RELEASE_HACK_ENABLED is defined based on ARC status. Obviously, this is kind of kludgy, and has performance implications on both speed and working set size, but it was effective in solving the problem. An alternative might be to set appropriate keyCallbacks on the cache dictionary?

Anyway, hope that helps.

[Garvan Keeley]

Nice find, I confirmed your test case on Xcode 7.2, swift 2, with ARC enabled on fastcoder.m. It fails as you described.

[Fabian Zwick]

This is indeed an actual issue that should be resolved. It's not related to Swift, I'm having the same issue with a pure Objective-C app. When serializing a dictionary two objects ended up with the same content for one key, resulting in a corrupt state.

[Nick Lockwood]

Can anyone confirm if this is still happening in the latest releases? A test case would be extremely helpful.

[Fabian Zwick]

Unfortunately, I was never able to reproduce the issue in a stand-alone test case and we now moved to a different structure where the issue never occurred. It happened with a dictionary where the keys were custom objects that implemented NSCoding and stored some small (only a couple of bytes) NSData instances.

The dictionary only contained objects of type ComplexObject for its values and keys (!). Maybe this is the reason. NSDictionary copies its keys, whereas CFDictionarySetValue() in FCCacheWrittenObject() doesn't (I believe).

• NSDictionary
  -- ComplexObject
  -- NSUInteger
  -- NSData
  -- NSArray

If I ever run into the issue again, I'll let you know.

[Nick Lockwood]

@fabiankr ah, good point about me not copying the keys. That's a bug on its own regardless of any other side-effects.