nice-hydrogen-2868902/20/2022, 5:43 AM
generated a bunch of build files, but when I to run
I get an error complaining that one of our dependencies does not exist — it’s something that we’ve vendored and linked through go.mod, and when I looked in the chroot for that step I found that only my mod and sum files were there. No sources or anything.
./pants list ::
has tried to generate a build file for every directory, specifying individual `go_package`s and `go_binary`s. I don’t understand why it does this: from my perspective, building go binaries is just
, so I assumed that the build step for a Go project would just be • specify input files • specify output file(s) • write a command that runs
to install deps, then
, and writes to the output location(s) And in theory, that should give you a hermetic build, as long as the Go toolchain is included as part of the sandbox. I don’t see why having a build-system-level abstraction for each package and for each third-party dependency is necessary — but when I tried Bazel I noticed that it had a very similar model, so I’m guessing there’s a reason for it I just haven’t grokked yet.
happy-kitchen-8948202/20/2022, 6:50 AM
nice-hydrogen-2868902/20/2022, 9:11 AM
hundreds-father-40402/21/2022, 2:30 AM
I don’t see why having a build-system-level abstraction for each package and for each third-party dependency is necessarySo with both Pants and Bazel, a "target" is how you give decentralized metadata for parts of your code. That metadata includes things like which packages/dirs depend on which others, and also optional metadata like the timeout for tests. A key point of build systems is to keep up as your codebase scales, and to do that, we generally believe metadata should live near the code it describes, rather than a centralized file. Hence, BUILD files. But Pants tries hard to make that boilerplate more minimal, such as inferring your dependencies for you. It also generates the
targets for you, rather than having to write them to a BUILD file or use a BUILD file generator We actually used to generate the
targets for you in Pants 2.8, rather than having one BUILD file per directory. But after a lot of discussion, we changed it in Pants 2.9 to be explicit. https://blog.pantsbuild.org/pants-2-9/ Does that make sense?
fast-nail-5540002/21/2022, 8:22 PM
I get an error complaining that one of our dependencies does not exist — it’s something that we’ve vendored and linked through go.mod, and when I looked in the chroot for that step I found that only my mod and sum files were there.vendoring isn’t supported yet. https://github.com/pantsbuild/pants/issues/12768
, it is because
is designed as a CLI tool for end users and makes many assumptions consistent with that design but that conflict with the needs of Pants. For example,
does not provide any insight into the separate build cache that it maintains. Thus, Pants uses
go tool compile
to invoke the Go compiler and linker directly. This allows Pants to specify where to output the “package archive” for a compiled package and then store that package archive in the Pants cache (which includes any configured remote cache which is something
go tool link
does not support. Bazel rules_go had to make similar choices. Note that Bazel vendors or reimplements some of the
source. See https://github.com/bazelbuild/rules_go/tree/master/go/tools/builders.)
were fully available as a library, but that is not the case. For example, Pants originally used
to analyze Go packages but that caused performance problems for Pants due to
assuming that transitive dependencies would always be available. Pants ended up incorporating an adapted version the package analysis code from
to side step that issue. (Note that Bazel also made a similar choice, for what I will presume are similar reasons.)
nice-hydrogen-2868902/22/2022, 12:59 AM
itself is smart enough to recognize the presence of already-compiled copies of matching packages in that cache and not recompile them at all I knew Go did build caching at the package level and that you can save and restore that cache from the filesystem (we do this in our current CI), but it had never occurred to me to populate the cache selectively. which it sounds like is what you’re saying Pants does.
fast-nail-5540002/22/2022, 1:00 AM
with all of the package archives that are already stored in the Pants cache.
go tool link
nice-hydrogen-2868902/22/2022, 1:01 AM
was a thing
go tool link
even with a hot Go build cache, because it doesn’t have to do any checking that the cache is up to date?
fast-nail-5540002/22/2022, 1:22 AM
is very optimized for its use case. But then again we haven’t micro-optimized the Go backend in Pants at all. The main reason for not using
is really around the fact that Pants would not be able to leverage its own cache at all, which is part of how Pants tracks dependencies and is able to invalidate build artifacts only when they change.
), Pants would have no way to provide the developer experience that users have to come to expect from other Pants language backends. And if the Pants Go backend cannot provide that experience, no particular reason to have a Go backend any more.
nice-hydrogen-2868902/22/2022, 1:35 AM
is “sandboxed” by Pants, but access to the Go filesystem-based build cache is considered an allowed exception, like (I think?) the toolchain itself
fast-nail-5540002/22/2022, 2:26 AM
Or is that what you mean by “no need for a backend” — at that point, the use case is well served by a simple genrule?I meant that no real need to implement the Go backend at all if Pants can’t provide the expected developer experience. The user is free to use
happy-kitchen-8948202/22/2022, 5:31 AM
tooling can provide, but Pants can.
nice-hydrogen-2868902/22/2022, 6:01 AM
happy-kitchen-8948202/22/2022, 5:04 PM