For most Pants goals i e `test` if file changes are detected Pants #plugins

For most Pants goals (i.e. `test`) if file changes...

dry-orange-3870

03/14/2024, 4:18 PM

For most Pants goals (i.e.

test

) if file changes are detected, it automatically cancels and restarts the rules or goal_rule. We have a repo with some custom plugins and an assortment of

goal_rule

and

rule

functions. Some of these involve running a bunch of operations using Pants, and then making an API call to an external service. We commonly run into situations where these external API calls / side-effecting operations happen twice. Based on what I see in the pants logs, it looks like

notify invalidation

is just slow to catch up with the filesystem; this is common when merging with

main

and then immediately running one of our custom pants goals. How can I work around this? Is there some way I can tag the `rule`s themselves to tell Pants that it should fail them instead of re-running them when a file system change is detected? EDIT: we're currently on

2.18.0

, but could likely upgrade without too much hassle

curved-television-6568

03/14/2024, 4:23 PM

besides either sleeping a bit after updating the worktree, or restarting pantsd, another option could be to disable file watching when running the custom goal (but this would also result in restarting pantsd..)

dry-orange-3870

03/14/2024, 4:36 PM

sleeping a bit after updating the worktree

I'm not sure I follow this. Are you suggesting I just count 10 ten after merging with main?

dry-orange-3870

03/14/2024, 4:38 PM

another option could be to disable file watching when running the custom goal (but this would also result in restarting pantsd..)

Would I just put a

cli.alias

in my

pants.toml

that calls the goal but with

--no-pantsd

on it to avoid this filesystem inotify thing?

dry-orange-3870

03/14/2024, 4:52 PM

On closer inspection, it seems my root cause is not necessarily correct. This time, I merged with main and waited for the

notify invalidation

logs to stop populating. I then ran my custom goal rule. Zero

notify invalidation

logs happened while it ran, but for some reason, some of the underlying rules were run twice

curved-television-6568

03/15/2024, 7:39 AM

> sleeping a bit after updating the worktree

I'm not sure I follow this. Are you suggesting I just count 10 ten after merging with main?

yea, but it's a horrible idea I hope you can avoid 😉

curved-television-6568

03/15/2024, 7:41 AM

On closer inspection, it seems my root cause is not necessarily correct.

is it possible there are multiple paths leading to your rule? my hypothesis here being that it may be called once, it starts processing but before it finishes it is called again, starting the second iteration rather than using the cached result of a previous call as it wasn't completed yet.

happy-kitchen-89482

03/15/2024, 11:02 PM

This would be hard to debug without seeing the source code of your plugins, but are they being run twice on identical inputs?

happy-kitchen-89482

03/15/2024, 11:02 PM

Do those runs appear to be concurrent, or sequential?

4 Views

Open in Slack

Previous Next