cmd/go: build_trimpath test doesn't correctly test executable reproducibility

[jayconrod]

What version of Go are you using (go version)?

$ go version
go version devel +689f6f77f0 Thu Nov 7 20:14:02 2019 +0000 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jayconrod/Library/Caches/go-build"
GOENV="/Users/jayconrod/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/jayconrod/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/jayconrod/Code/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/jayconrod/Code/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jayconrod/Code/go/src/cmd/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/rq/x0692kqj6ml8cvrhcqh5bswc008xj1/T/go-build459473337=/tmp/go-build -gno-record-gcc-switches -fno-common"

Observed this on both darwin and windows.

What did you do?

I saw this when expanding src/cmd/go/testdata/script/build_trimpath.txt. In short, this test builds a binary in one directory, copies the source and go.mod file to another directory, builds the binary again there, and compares the two executable files. When -trimpath is used, they should be identical.

Here's the modified version of that test I was using.

[short] skip

# A binary built without -trimpath should contain the current workspace
# and GOROOT for debugging and stack traces.
cd a
go build -o $WORK/paths.exe paths.go
exec $WORK/paths.exe $WORK/paths.exe
stdout 'binary contains GOPATH: true'
stdout 'binary contains GOROOT: true'

# A binary built with -trimpath should not contain the current workspace
# or GOROOT.
go build -trimpath -o $WORK/paths.exe paths.go
exec $WORK/paths.exe $WORK/paths.exe
stdout 'binary contains GOPATH: false'
stdout 'binary contains GOROOT: false'

# A binary from an external module built with -trimpath should not contain
# the current workspace or GOROOT.
cd $WORK
env GO111MODULE=on
go get -trimpath rsc.io/fortune
exec $WORK/paths.exe $GOPATH/bin/fortune$GOEXE
stdout 'binary contains GOPATH: false'
stdout 'binary contains GOROOT: false'

# Two binaries built from identical packages in different directories
# should be identical.
mkdir $WORK/_alt/src/a
cp $GOPATH/src/a/go.mod $WORK/_alt/src/a/go.mod
cp $GOPATH/src/a/paths.go $WORK/_alt/src/a/paths.go
cd $WORK/_alt/src/a
go build -trimpath -o paths-alt.exe .
cmp -q $WORK/paths.exe paths-alt.exe

[!exec:gccgo] stop

# Binaries built using gccgo should also be identical to each other.
env GO111MODULE=off # The current released gccgo does not support builds in module mode.
cd $GOPATH/src/a
go build -compiler=gccgo -trimpath -o $WORK/gccgo-GOPATH.exe .

env old_gopath=$GOPATH
env GOPATH=$WORK/_alt
cd $WORK/_alt/src/a
go build -compiler=gccgo -trimpath -o $WORK/gccgo-alt.exe .
cd $WORK
exec $WORK/gccgo-alt.exe $WORK/gccgo-alt.exe
stdout 'binary contains GOPATH: false'
stdout 'binary contains GOROOT: false'
cmp -q gccgo-GOPATH.exe gccgo-alt.exe

-- $GOPATH/src/a/paths.go --
package main

import (
	"bytes"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"path/filepath"
)

func main() {
	exe := os.Args[1]
	data, err := ioutil.ReadFile(exe)
	if err != nil {
		log.Fatal(err)
	}

	gopath := []byte(filepath.ToSlash(os.Getenv("GOPATH")))
	if len(gopath) == 0 {
		log.Fatal("GOPATH not set")
	}
	fmt.Printf("binary contains GOPATH: %v\n", bytes.Contains(data, gopath))

	goroot := []byte(filepath.ToSlash(os.Getenv("GOROOT")))
	if len(goroot) == 0 {
		log.Fatal("GOROOT not set")
	}
	fmt.Printf("binary contains GOROOT: %v\n", bytes.Contains(data, goroot))
}
-- $GOPATH/src/a/go.mod --
module example.com/a

What did you expect to see?

Test should pass.

What did you see instead?

The executable files were not identical. They are the same size and the differences don't seem to be in file paths or build ids. So this doesn't seem to be an exact duplicate of #33772.

[gopherbot]

Change https://golang.org/cl/208234 mentions this issue: cmd/go: fix and re-enabled build_trimpath test

[jayconrod]

For anyone tracking this issue, this was a bug in the test, not in the go command. The test was comparing a binary built from a list of source files with a binary built from a named package. Those aren't necessary (or actually) identical.

[mvdan]

@jayconrod perhaps retitle the issue to clarify that the test was buggy? With the current title, I just had a slight scare thinking that Go 1.13.x's -trimpath is broken :)