net.schmizz.sshj.sftp.SFTPException: "Handle not Found" when Attempting to Upload a File

[Jonathan]

When attempting to upload a file to an SFTP server, I get a "Handle Not Found" error. The stack trace looks something like this:

net.schmizz.sshj.sftp.SFTPException: Handle not found.
    at net.schmizz.sshj.sftp.Response.error(Response.java:113)
    at net.schmizz.sshj.sftp.Response.ensureStatusIs(Response.java:106)
    at net.schmizz.sshj.sftp.Response.ensureStatusPacketIsOK(Response.java:99)
    at net.schmizz.sshj.sftp.RemoteResource.close(RemoteResource.java:55)
    at net.schmizz.sshj.sftp.SFTPFileTransfer$Uploader.uploadFile(SFTPFileTransfer.java:223)
    at net.schmizz.sshj.sftp.SFTPFileTransfer$Uploader.upload(SFTPFileTransfer.java:184)
    at net.schmizz.sshj.sftp.SFTPFileTransfer$Uploader.access$100(SFTPFileTransfer.java:174)
    at net.schmizz.sshj.sftp.SFTPFileTransfer.upload(SFTPFileTransfer.java:70)
    at net.schmizz.sshj.sftp.SFTPClient.put(SFTPClient.java:248)

A little bit of research turned up this Google Groups post that claims that the target server returned an 8-byte file handle to the client, which in turn sent a 12-byte string to back the server when trying to write the file contents.

I wrote a simple unit test to prove this theory, and found that it made some sense:

@Test(groups = "unittests")
private void bufferTest() throws BufferException {
    // create a buffer containing an SSH_FXP_HANDLE message that I got from my debugger
    byte[] sourceBytes = new byte[] {102, 0, 0, 0, 2, 0, 0, 0, 8, 8, 28, -84, -64, 0, 0, 0, 4};
    Buffer b = new Buffer(sourceBytes);

    // set rpos to 5 to simulate having read the id field - I also got this value from the debugger
    b.rpos(5);

    // call readString on the buffer, and then change this string into a byte array, maintaining the UTF-8 encoding
    byte[] stringBytes = b.readString().getBytes(StandardCharsets.UTF_8);

    // according to the google groups post, this should be 8 bytes long, not 12
    Assert.assertEquals(stringBytes.length, 12);

    // this assert will fail, because stringBytes is no longer equal to the last 8 bytes of the original message
    Assert.assertEquals(stringBytes, Arrays.copyOfRange(sourceBytes, sourceBytes.length - 8, sourceBytes.length));
}

In this case, the sourceBytes array and the rpos=5 variables came from my debugger. To get them, I set a breakpoint in the open method in SFTPEngine (line 137) and stepped into the readString() method of the Buffer class (line 336) to see how the server's response was being parsed.

According to the SFTP RFC, the response to an SSH_FXP_OPEN message is an SSH_FXP_HANDLE message that contains an integer id field and a string handle field. The RFC describes this message thusly:

The SSH_FXP_HANDLE response has the following format:
uint32 id
string handle
where id' is the request identifier, andhandle' is an arbitrary string that identifies an open file or directory on the server. The handle is opaque to the client; the client MUST NOT attempt to interpret or modify it in any way. The length of the handle string MUST NOT exceed 256 data bytes.

The thing is, I think that SSHJ is modifying the handle - The readString() method in the Buffer class attempts to convert the handle into a UTF-8 string. This is fine if each of the bytes in the handle can be represented in this encoding, but the javadoc for the String() constructor that is used by readString specifically states that "the behavior of this constructor when the given bytes are not valid in the given charset is unspecified".

Later, in the newRequest method of the RemoteResource class (line 46), the handle is turned back into a byte array by the putString method in the Buffer class (line 401). My unit test shows that extracting the bytes from the string handle results in the 12-byte array that the original Google Groups post was complaining about.

In short, I think that the way to fix this problem is to always treat the handle that is returned in the SSH_FXP_HANDLE message as a byte array, rather than bashing it into a UTF-8 string, as I suspect that some SFTP servers return a handle that cannot be encoded in this manner.

[David Kocher]

This is a duplicate of #119 fixed in #150.

[Jonathan]

@dkocher My mistake, sorry! Has the version with the fix been released? I'm currently referencing version 0.10.0

[Jeroen van Erp]

Thx @dkocher, indeed exactly the same.

[Jeroen van Erp]

@MusikPolice There is no newer version yet than 0.10.0. Currently working on that.

[Jeroen van Erp]

@MusikPolice Version 0.11.0 is now released, which should fix this. Please look at the README, or the announcement here: https://groups.google.com/forum/#!topic/sshj-users/ZvFB6gZCoyI

Closing this issue.