https://pantsbuild.org/ logo
Powered by
b

bitter-ability-32190

05/10/2022, 4:30 PM
Our Bazel build has a 
genrule
which generates a file which dumps some git info to be included in packaged things. Trying to use 
experimental_shell_command
to do the same thing, I unfortunately noticed the command runs in the sandbox and not in repo root. Alternatively I could use 
experimental_run_shell_command
but then I can't capture outputs 😞
✅ 1
w

witty-crayon-22786

05/10/2022, 4:47 PM
@happy-kitchen-89482 landed https://github.com/pantsbuild/pants/pull/15374 yesterday to do a flavor of this… but yea, scripted direct access to git isn’t currently possible. a rule could do it in a similar fashion to what he did here
b

bitter-ability-32190

05/10/2022, 4:49 PM
Ahh bummer. I'll just make a dirty in-repo plugin which dumps the file. It's the only place we do this kinda thing
Or I could just...
Copy code
def build_file_aliases():
    def repo_root_path(parse_context):
        return lambda: PurePath(parse_context.build_root)
    return BuildFileAliases(context_aware_object_factories={"repo_root_path": repo_root_path})
🙈
h

hundreds-father-404

05/10/2022, 5:11 PM
sounds like it would be useful for Pants to surface that in BUILD files? That doesn't seem dirty to me
oh well I guess, actually, you need the absolute path?
b

bitter-ability-32190

05/10/2022, 5:16 PM
It is the absolute path 😂 Otherwise I tried 
build_file_dir().parent
which is 
'.'
I can make a PR later
w

witty-crayon-22786

05/10/2022, 5:34 PM
how was the 
genrule
invalidated in bazel? i’m guessing it wasn’t, and you needed to restart the server?
b

bitter-ability-32190

05/10/2022, 5:36 PM
I know little of Bazel's behavior. i suspect one of these is responsible:
Copy code
local = True,
    stamp = 1,
    tags = ["no-cache"],
👍 1
Apparently this is still cached in Bazel. Also my hack doesn't invalidate the cache 😞
w

witty-crayon-22786

05/10/2022, 6:51 PM
mm, yea. my expectation of 
no-cache
would be to skip writing it to the persistent cache, rather than to necessarily not memoize it in memory. would be the same here probably… interaction with 
git
needs the pants equivalent of 
cache_scope=PER_SESSION
i have a vague feeling that there might be some cases where exposing PER_SESSION to 
experimental_shell_command
could be useful… but i’m not sure what the API would look like, since when you set it, you’d still also need to escape the sandbox like you did with the absolute path.
💯 1
cc @curved-television-6568, in case you have thoughts
b

bitter-ability-32190

05/10/2022, 6:54 PM
Yeah. I admit what I'm doing is dirty. Really I need a hack for this to work today (ideally in a way that doesn't shut me out from using the equivalent Bazel thing while I'm straddleing). And then also whatever the right way of doing this is, which I can upgrade to when off bazel 🙂
w

witty-crayon-22786

05/10/2022, 6:56 PM
@bitter-ability-32190: doing a much lighter form of what Benjy did (codegen rule attached to a target) would do it. depending on the 
MaybeGitWorktree
gives you PER_SESSION behavior.
b

bitter-ability-32190

05/10/2022, 6:58 PM
Yeah that's what I started with. The reason I short-circuited to the caof is I'd like to still use the same sh script we're using in Bazel. And easily 😛
Aww RIP 2.11.x doesn't use 
GitWorkTree
as an 
EngineAwareReturnType
Awww darn: 
Exception: Error lifting Process: Absolute paths are not allowed: "/home/joshuacannon/work/techlabs"
Copy code
result = await Get(
        ProcessResult,
        Process(
            ["git", "rev-parse", "HEAD"],
            working_directory=build_root.path,
            description="Getting Git SHA.",
            cache_scope=ProcessCacheScope.PER_SESSION,
        ),
    )
Was trying to whip up a way to have the rule fire per-state change and this was my current path
Ahh ignore me. I just found it this file is deeply rooted in our deps, so trying to make it work right would invalidate most of our graph lol
👍 1
c

curved-television-6568

05/10/2022, 9:01 PM
When workingwith git you dont need pwd set, you can pass the —git-root (IIRC the option name, or as env var) for your .git dir to it. https://git-scm.com/docs/git#Documentation/git.txt---git-dirltpathgt
👍 1
b

bitter-ability-32190

05/10/2022, 9:19 PM
So, for the curious, here's how I'm solving it: • Expose the file just like we do in bazel. Bazel has the cache bug (as I found out) so keeping that in theme with Pants • Make another plugin which exposes a dummy file codegen which only gets cached per git SHA, and a target for it • Make another 
experimental_shell_command
which generates the file but depends on the address of the target which depends on git state ◦ This means the file correctly gets re-generated every time SHA changes • Use the latter dependency when building docker images
Once everything is migrated to Pants, I can untangle the first generated file, since it will no longer be needed.
Make another plugin which exposes a dummy file codegen which only gets cached per git SHA
I feel so evil, yet so good
😈 1
2 Views
Powered by