`close` does not work. WebSockets stay open indefinitely

[Lee M]

Running socket.close(code: .normalClosure) should trigger a close of the socket on the server end.

Attaching .wait() resolves immediately on the client end. But whatever I do, the socket is still marked as open by the server. This was working at some point and now it's broken.

[Lee M]

Just doing some tracing, everything seems to be in working order, except self.channel.isActive still returning true. This value seemingly never gets set to false.

[Lee M]

Problem identified - mode should be all rather than output. Connections were staying open indefinitely, verified with lsof. I do worry that this code doesn't seem to have been tested before being put out for release 🤔

[EthanLozano]

Is the client supposed to be configured a certain way to get close to work? My only workaround is to call eventLoop.shutdownGracefully() { _ in } on the client, which is a hack. I can't use this workaround on the server since I don't manage the event loop's lifecycle. @lmcd how are you getting around this issue? Without a fix, it seems like downgrading (or maybe even forking) is the only option?

[Lee M]

@EthanLozano - yeah there's an actual bug in websocket-kit, I just haven't gotten around to submitting a PR and have just patched it myself in my local install. See my previous message for the source of the issue.

It's annoying how both WebSockets and HTTP/2 are simply just broken in Vapor - major bugs basically, and they're not getting patched 🤷‍♂️. My recommendation to anyone would be to not use Vapor in production.

[Lee M]

FYI WebSocket.swift, in handle, change mode to .all instead of .output in both instances.

[EthanLozano]

@lmcd thanks for the response! I was worried that forking/patching was the approach you chose. Yeah, Vapor has disappointed me: I reported issues on multiple Vapor repos months ago and nobody has even commented on my issues. I'm a little confused... are people not using Vapor for WebSockets? Or, is Vapor just used for prototyping?

[Tim Condon]

@lmcd @EthanLozano Thanks for raising these and other issues! Turns out I wasn't subscribed to this repo so I've missed everything! I'll take a look at your PR tomorrow and then start looking in to the other issues

[Lee M]

Thanks @0xTim

Note: I haven't studied the SwiftNIO source in detail, so am not 100% sure the above fix is correct, I just know that it fully resolves the issue for me and have noticed no unintended side-effects.

[EthanLozano]

Thanks @0xTim! I can only confirm that @lmcd's patch solved the issues for me. I also don't know if this is the correct fix. I used Apple's Editing a Package Dependency as a Local Package to get the patch to work locally.

I briefly tried to reproduce the issue within the tests, and I was unsuccessful. Specifically, I tweaked testServerClose, testClientClose, testWebSocketWithTLSEcho to include an additional ws.onClose.cascade(secondClose), but the tests still pass successfully on my machine. So, it appears that the tests don't capture a realistic environment. Perhaps the bug is exposed with SSL, real devices, or real networks?

The bug appears while using a vapor server (v4.45.2) with a basic tlsConfiguration running on MacOS (v11.2.1) running either in XCode (12.4) or with swift (v5.3.2) and with an iOS (v14.4) client using websocket-kit (v2.1.2).

[EthanLozano]

I confirmed that the issue is exposed when ssl is enabled.

First, I created a fresh project:

vapor new websocket-kit-bug-repro

Second, I added the following server-side code to configure.swift:

    app.http.server.configuration.hostname = Environment.get("HOSTNAME")!
    if Environment.get("SSL_ENABLED") != nil {
        try app.http.server.configuration.tlsConfiguration = .forServer(
            certificateChain: [
                .certificate(.init(file: Environment.get("SSL_CERT_FILE")!, format: .pem))
            ],
            privateKey: .file(Environment.get("SSL_KEY_FILE")!))
    }
    app.webSocket("echo") { req, ws in
        ws.onText { ws, text in
            if text == "close" {
                print("about to close")
                _ = ws.close()
            }
        }
        ws.onClose.whenComplete { _ in
            print("server closed")
        }
    }

Third, I added the following client-side code to main.swift

struct CloseMe: Command {
    struct Signature: CommandSignature { }

    var help: String {
        "tests websocket close"
    }

    func run(using context: CommandContext, signature: Signature) throws {
        let elg = context.application.eventLoopGroup
        let closePromise = elg.next().makePromise(of: Void.self)
        let hostname = Environment.get("HOSTNAME")!
        let ssl = Environment.get("SSL_ENABLED") != nil
        let web = ssl ? "wss" : "ws"
        let connect = WebSocket.connect(to: "\(web)://\(hostname):8080/echo", on: elg) { ws in
            print("telling to close")
            ws.send("close")
            ws.onClose.whenComplete { _ in
                print("client closed")
            }
            ws.onClose.cascade(to: closePromise)
        }
        connect.whenSuccess {
            print("successful connection")
        }
        connect.whenFailure { error in
            print("unsucessful connection: \(error)")
        }
        try closePromise.futureResult.wait()
    }
}
app.commands.use(CloseMe(), as: "closeme")

I generated a root certificate, created a certificate for my local machine, added the root certificate to my Keychain, and then set the environment variables to point to the appropriate cert files.

Output without SSL

server:

$ unset SSL_ENABLED && vapor run serve
[ NOTICE ] Server starting on http://[redacted]:8080
[ INFO ] GET /echo [request-id: 9BB86C27-D7EB-4E2E-9E00-25068D6A0C6D]
about to close
server closed

client:

$ unset SSL_ENABLED && vapor run closeme
telling to close
successful connection
client closed

Output with SSL

server:

$ export SSL_ENABLED=true && vapor run serve
[ NOTICE ] Server starting on https://[redacted].home:8080
[ INFO ] GET /echo [request-id: 3F6B1C22-7895-4633-83D9-EDDB3C6FF415]
about to close

client:

$ export SSL_ENABLED=true && vapor run closeme
telling to close
successful connection
[hanging indefinitely]

[Tim Condon]

@lmcd was this happening only on TLS connections as well or both? Essentially I'm trying to write a unit test to replicate this bug and for me, with no TLS, channel.isActive is marked as false after the close

[Lee M]

I'm using Vapor only in a TLS environment. I'm also using websocket-kit on both ends, so client and server.

[Tim Condon]

Ok writing a unit test that involves terminating TLS in web-socket Kit is going to hard so I'll merge the PR

[EthanLozano]

Thanks @0xTim! I created another issue to track writing a unit test for this issue