Offset mismatch when reading bytes in a decoder

[Steven Vandevelde]

The issue occurs when …

1. You use a decoder that uses Decode.bytes and the bytes you want don't have an offset of 0.
   (eg. in our example, we discard the first byte)
2. You use another decoder that uses Decode.bytes.

Note: With "using a decoder" I mean, using the decoder with the decode function.

Why does the issue occur?

• When you use the decode function, the offset is "reset" to 0.
• The bytes's ArrayBuffer is still the original slice of bytes, not the slice we want.

Could you show an example?

let
  decoders =
    { skipOne       = Decode.andThen (\_ -> Decode.bytes 1) (Decode.bytes 1)
    , maybeOneUint8 = Decode.map (Decode.decode Decode.unsignedInt8) (Decode.bytes 1)
    }
in
[ Encode.unsignedInt8 1
, Encode.unsignedInt8 2
]
  |> Encode.sequence
  |> Encode.encode
  |> Decode.decode decoders.skipOne
  |> Maybe.andThen (Decode.decode decoders.maybeOneUint8)
  |> Debug.log "should be `Just (Just 2)`"

SSCCE

Which use-cases does this have?

I stumbled onto this issue when writing a SHA-2 implementation. I needed to divide my bytes into chunks of 64 bytes and then divide those chunks into smaller chunks of 4 bytes. I used multiple decode loops to do this.

How can we solve this?

Myself, and @eriktim, found two ways to solve this:

1. Track the previous offset, which means: Use bytes.byteOffset as done in #10
2. Make a new buffer (see below)

var _Bytes_read_bytes = F3(function(len, bytes, offset)
{
	return __Utils_Tuple2(
		offset + len,
		new DataView(bytes.buffer.slice(bytes.byteOffset), offset, len)
	);
});

Important difference between (1) and (2):

1. Keeps reference to old buffer, no bytes are discarded when using decode.
2. Makes new instance of buffer each time, no references, bytes are discarded when using decode.

Final notes

I'm not sure what the correct solution is here.
Let me know if I need to make a PR for the second case presented ☝️

Thanks!

[Evan Czaplicki]

Thank you! Helped me understand and test things out. I'm going to go through other issues before releasing the patch change. Should be sometime today though.