go_path embedding still failing on go 1.18

[mancusi]

What version of rules_go are you using?

git_repository(
    name = "io_bazel_rules_go",
    commit = "3f84d8cd64702be3d08c67c47069b3f85d5a69a7",
    remote = "https://github.com/bazelbuild/rules_go",
)

What version of gazelle are you using?

0.25

What version of Bazel are you using?

5.1.1

Does this issue reproduce with the latest releases of all the above?

Yes

What operating system and processor architecture are you using?

macOS x86

Any other potentially useful information about your toolchain?

n/a

What did you do?

We have a custom rule which runs easyjson in a custom gopath that fails even while including the fix to #3080.

The main command is:

    ctx.actions.run_shell(
        inputs = inputs,
        outputs = [ctx.outputs.out],
        tools = [ctx.executable._easyjson_tool],
        command = """
           export GOPATH=$PWD/{gopath} &&
           $PWD/{godir}/go env >> tmp_go_env.txt &&
           source tmp_go_env.txt &&
           rm tmp_go_env.txt &&
           export PATH=$GOROOT/bin:$PWD/{godir}:$PATH &&
           export GOCACHE=$GOPATH/pkg &&
           export GO111MODULE=off &&
           export SRCDIR=$GOPATH/src/{importpath} &&
           {easyjson} {args} $SRCDIR/{sourcebase} &&
           mv $SRCDIR/{outputbase} {output}
        """.format(
            godir = go_ctx.go.path[:-1 - len(go_ctx.go.basename)],
            gopath = ctx.attr.gopath_dep[GoPath].gopath_file.path,
            easyjson = ctx.executable._easyjson_tool.path,
            args = ctx.attr.args,
            source = ctx.file.source.path,
            importpath = importpath,
            sourcebase = ctx.file.source.basename,
            outputbase = ctx.outputs.out.basename,
            output = ctx.outputs.out.path,
        ),
        mnemonic = "GoEasyJSON",
    )

What did you expect to see?

I'd expect that this passes after the fix here.

What did you see instead?

Files in external/go_sdk are being symlinked which is causing go:embed to fail with ../../../../../../../../../../../external/go_sdk/src/crypto/elliptic/p256_asm.go:24:12: pattern p256_asm_table.bin: cannot embed irregular file p256_asm_table.bin

[mancusi]

The line: {easyjson} {args} $SRCDIR/{sourcebase} && in particular is the issue.
For one example, this command resolves to:

           $PWD/external/go_sdk/bin/go env >> tmp_go_env.txt &&
           source tmp_go_env.txt &&
           rm tmp_go_env.txt &&
           export PATH=$GOROOT/bin:$PWD/external/go_sdk/bin:$PATH &&
           export GOCACHE=$GOPATH/pkg &&
           export GO111MODULE=off &&
           export SRCDIR=$GOPATH/src/yext/livedata2 &&
           bazel-out/host/bin/external/com_github_mailru_easyjson/easyjson/easyjson_/easyjson  $SRCDIR/response.go

[fmeum]

@mancusi Does this also fail with the other modes ("archive", "link") set on go_path? If Go itself uses embeds but can't embed symlinks, then some modes of go_path will probably remain broken going forward and there is little we could do about it.

[mancusi]

Unfortunately those options don't seem to work for me.

I attempted to include the bin file ("@go_sdk//:src/crypto/elliptic/p256_asm_table.bin") as src in embedsrcs of the go_library the go_path includes as a dependency but I'm getting

Traceback (most recent call last):
	File "/private/var/tmp/_bazel_jmancusi/cc6e0bfe61b0904eacf3d4a5bdc454f9/external/io_bazel_rules_go/go/private/tools/path.bzl", line 86, column 51, in _go_path_impl
		dst = pkg.dir + "/" + paths.relativize(f.path, src_dir)
	File "/private/var/tmp/_bazel_jmancusi/cc6e0bfe61b0904eacf3d4a5bdc454f9/external/bazel_skylib/lib/paths.bzl", line 186, column 17, in _relativize
		fail("Path '%s' is not beneath '%s'" % (path, start))
Error in fail: Path 'external/go_sdk/src/crypto/elliptic/p256_asm_table.bin' is not beneath 'gocode/src/yext/livedata2'

[tokongs]

I'm struggling with the same problem. Did you find a solution to this @mancusi ?

[mr-real]

For some reason that library only builds with the local build strategy. So you can either set the repo-wide strategy to local in .bazelrc or ideally (and esp. if local strategy breaks something else) use build --strategy_regexp=...=local in .bazelrc to only build certain rules locally in order to fix this issue.

[jieyu]

I hit this issue as well.

I think this is due to this https://go-review.googlesource.com/c/go/+/380475

So go does not like symlink for go:embed files. However, in bazel, the sandbox uses the symlinks to the actual files, thus cause golang tooling to complain. Using local=True works because it's using execroot and does not symlink each individual files.