boundary_test failures under Go 1.12.1
jhillyerd opened this issue · comments
What I did:
Run unit tests with go version go1.12.1 linux/amd64
What I expected:
Success!
What I got:
--- FAIL: TestBoundaryReader (0.00s)
boundary_test.go:95: Rest of reader:
got: "\n--STOPHERE\r\nafter", want: "\r\n--STOPHERE\r\nafter"
boundary_test.go:95: Rest of reader:
got: "\n--STOPHERE\t\r\nafter", want: "\r\n--STOPHERE\t\r\nafter"
boundary_test.go:95: Rest of reader:
got: "\n--STOPHERE--\r\nafter", want: "\r\n--STOPHERE--\r\nafter"
boundary_test.go:95: Rest of reader:
got: "\n--STOPHERE \t\r\nafter", want: "\r\n--STOPHERE \t\r\nafter"
boundary_test.go:95: Rest of reader:
got: "\n--STOPHERE--\t \r\nafter", want: "\r\n--STOPHERE--\t \r\nafter"
boundary_test.go:95: Rest of reader:
got: "\n--STOPHERE--\r\nafter", want: "\r\n--STOPHERE--\r\nafter"
boundary_test.go:95: Rest of reader:
got: "\n--STOPHERE--\r\nafter", want: "\r\n--STOPHERE--\r\nafter"
FAIL
FAIL github.com/jhillyerd/enmime 1.556s
Release or branch I am using: develop
Also fails with go version go1.12 linux/amd64
Hey @requaos, any guesses on this one? At first I thought it was a line ending conversion issue, but now I see the \r preserved later in the got string.
I don't have a Windows dev environment setup right now, so I'm not sure if it happens there also.
Still failing with go version go1.12.5 linux/amd64 -- may have to break out the debugger
I have a windows machine at home, I'll run the tests tonight and see what I come up with. I will also run the tests in a linux docker container
Could you assign the issue to me?
Seems like they've changed the way issue assignment works. I've sent you a collaborator invite for the repo.
I wrote this multi-stage Dockerfile to run the tests in Alpine and Debian:
#####################################
# ALPINE TEST
#####################################
FROM golang:1.12-alpine as Alpine
RUN apk add --no-cache build-base git
WORKDIR /go/src/github.com/jhillyerd/enmime
COPY . ./
# not using `-race` for this musl-based os image:
# https://github.com/golang/go/issues/14481
RUN make deps && go test ./...
#####################################
# LINUX TEST
#####################################
FROM golang:1.12 as Linux
RUN apt-get -y update --no-install-recommends && apt-get -y install --no-install-recommends build-essential git
WORKDIR /go/src/github.com/jhillyerd/enmime
COPY . ./
RUN make deps && make test
# More linux varients?
Execution results:
$ docker build -t test-enmime-linux .
Sending build context to Docker daemon 2.618MB
Step 1/10 : FROM golang:1.12-alpine as Alpine
---> c7330979841b
Step 2/10 : RUN apk add --no-cache build-base git
---> Using cache
---> 7671a7103904
Step 3/10 : WORKDIR /go/src/github.com/jhillyerd/enmime
---> Using cache
---> 4c72fb9cc184
Step 4/10 : COPY . ./
---> e2683661122a
Step 5/10 : RUN make deps && go test ./...
---> Running in f726f8593359
go get -t ./...
ok github.com/jhillyerd/enmime 0.090s
ok github.com/jhillyerd/enmime/cmd 0.015s
? github.com/jhillyerd/enmime/cmd/mime-dump [no test files]
? github.com/jhillyerd/enmime/cmd/mime-extractor [no test files]
ok github.com/jhillyerd/enmime/internal/coding 0.007s
ok github.com/jhillyerd/enmime/internal/stringutil 0.016s
ok github.com/jhillyerd/enmime/internal/test 0.006s
Removing intermediate container f726f8593359
---> acfa01fcf2e1
Step 6/10 : FROM golang:1.12 as Linux
---> 7ced090ee82e
Step 7/10 : RUN apt-get -y update --no-install-recommends && apt-get -y install --no-install-recommends build-essential git
---> Using cache
---> 3cb3393ecb77
Step 8/10 : WORKDIR /go/src/github.com/jhillyerd/enmime
---> Using cache
---> e13de1395378
Step 9/10 : COPY . ./
---> e1db6e833a3c
Step 10/10 : RUN make deps && make test
---> Running in c14fd9c27ab2
go get -t ./...
go test -race ./...
ok github.com/jhillyerd/enmime 1.168s
ok github.com/jhillyerd/enmime/cmd 1.039s
? github.com/jhillyerd/enmime/cmd/mime-dump [no test files]
? github.com/jhillyerd/enmime/cmd/mime-extractor [no test files]
ok github.com/jhillyerd/enmime/internal/coding 1.030s
ok github.com/jhillyerd/enmime/internal/stringutil 1.038s
ok github.com/jhillyerd/enmime/internal/test 1.025s
Removing intermediate container c14fd9c27ab2
---> 73d8e21f1efa
Successfully built 73d8e21f1efa
Successfully tagged test-enmime-linux:latest
Rebased on feature/go112 and added ENV GO111MODULE=on to Dockerfile, now I can reproduce the failure...
Tracked the issue down to UnreadByte() call in the boundary reader, since go1.12 this now consistently results in bufio.ErrInvalidUnreadByte
It may be the call to Peek(1) that occurs between ReadByte() and UnreadByte() that results in a change to the bufio.Reader.lastbyte value. Pure speculation.
In the mean-time, I will refactor this to avoid calling UnreadByte()
That sucks, nice find though!
I opened the PR to feature/go112 with the assumption that, once passing, it would be merged into develop
Please let me know if you want it done differently
+// Calling Peek prevents a UnreadByte or UnreadRune call from succeeding
+// until the next read operation.
Commit in go1.12 where Peek()/UnreadByte() behavior changed
Turns out that this change was documented in the go1.12 release notes:
bufio changes