https://pantsbuild.org/ logo
Join Slack
Powered by
Hello dear Pants developers and happy New Year for...
# general
l
Hello dear Pants developers and happy New Year for those I did not interact with (yet) this year! šŸ„‚ We upgraded from Pants 2.14 to Pants 2.15 at my client and a custom 
bump
plugin we have written is now having hiccups with messages of the form:
Filesystem changed during run: retrying 
Bump
in 500ms...
Because the 
bump
task bumps version numbers in 
pyproject.toml
files, the retry is causing multiple bumps to happen with one command. As in, we request bumping the patch number once, and 
0.0.1
gets bumped to 
0.0.3
or 
0.0.4
. It's not deterministic and I did not yet understand the trigger šŸ¤· My question is: would 
--no-watch-filesystem --no-pantsd
be a quick workaround until I figure things correctly?
w
You have a plugin that re-writes the watched pyproject.tomls? I'm surprised that ever worked.
Or do they bump once, re-run, and then are fine?
l
I'm not sure what you mean by watch here. I mean the modified 
pyproject.toml
are part of the 
BUILD
files infrastructure.
So I think that's what you mean by watched šŸ™‚
Honestly I add in mind to do this with a simple sed initially, but some Pants-enthusiast introduced a custom Pants plugin to do this, and it worked; so it stayed like this šŸ™‚ We had no trouble with it so far.
We use it like this:
Copy code
ā†’ tree -L 1 libs/wsi | head -n 8
libs/wsi
ā”œā”€ā”€ BUILD
ā”œā”€ā”€ docker
ā”œā”€ā”€ kaiko
ā”œā”€ā”€ pyproject.toml
ā”œā”€ā”€ README.md
ā”œā”€ā”€ requirements.txt
ā”œā”€ā”€ scripts
ā†’ ./pants bump libs/wsi
20:58:47.27 [INFO] Initializing scheduler...
20:58:47.28 [WARN] DEPRECATED: Setting `Goal.environment_behavior=EnvironmentBehavior.UNMIGRATED` for `Goal` `bump` is scheduled to be removed in version 2.17.0.dev0.

custombump - custombump succeeded.
 - (python) libs/wsi: 0.0.23 ā†’ 0.0.24
w
Is this just an informational message? Or is it stuck in a loop? Is it breaking anything?
l
Well the output above is the expected one : it does exactly one bump. But since we upgraded to 2.15, we also have runs like this:
Copy code
ā†’ ./pants bump libs/wsi
20:58:47.27 [INFO] Initializing scheduler...
20:58:47.28 [WARN] DEPRECATED: Setting `Goal.environment_behavior=EnvironmentBehavior.UNMIGRATED` for `Goal` `bump` is scheduled to be removed in version 2.17.0.dev0.

See <https://www.pantsbuild.org/v2.15/docs/plugin-upgrade-guide>

20:58:48.05 [INFO] Scheduler initialized.
20:58:48.15 [WARN] {}
20:58:48.15 [INFO] Completed: Bump pyproject.toml version - custombump - custombump succeeded.
 - (python) libs/wsi: 0.0.23 ā†’ 0.0.24

āœ“ custombump succeeded.
20:58:48.15 [INFO] Filesystem changed during run: retrying `Bump` in 500ms...
20:58:48.65 [WARN] {}
20:58:48.65 [INFO] Completed: Bump pyproject.toml version - custombump - custombump succeeded.
 - (python) libs/wsi: 0.0.24 ā†’ 0.0.25

āœ“ custombump succeeded.
20:58:48.66 [INFO] Filesystem changed during run: retrying `Bump` in 500ms...
20:58:49.16 [WARN] {}
20:58:49.16 [INFO] Completed: Bump pyproject.toml version - custombump - custombump succeeded.
 - (python) libs/wsi: 0.0.25 ā†’ 0.0.26

āœ“ custombump succeeded.
w
Does bump only update one version at a time?
l
But possibly the answer is that we shouldn't do this at all, you tell me šŸ˜„
w
Im not sure yet, just learning what's happening first
l
It can take multiple directories at once as in 
./pants bump libs/wsi libs/other
w
right, but the version - it's the same lib being upgraded, each by 1 patch version
Ah, interesting, 
bump
is a goal? Or an alias?
l
bump
is a goal
w
Okay, so - without really knowing everything, what I'm seeing seems to make sense. I would probably treat 
bump
as a fixer myself, and then just run 
pants fix ::
or maybe use it as an alias, but I get not wanting to run it ALL the time. Anyways, if that's the case, I would think this is expected. When you run a formatter or fixer, and the code it's monitoring changes (intentionally), I think there is always a re-run of that goal (don't quote me on that, I'd have to run something and fmt or fix on it - as I'm immune to a lot of these messages). The weird part is that the bump goal seems to only update one patch version at a time, instead of to the latest
The other weird part is that you're saying these are new messages? I would have thought they'd always be there
l
I mean that before the update to Pants 2.15 we never observed the messages, but we also had to modify our 
bump
plugin when upgrading from 2.14 to 2.15.
w
Oh. Interesting.
l
So it may not be a Pants change we are observing, but only that our change to accomodate 2.15 was wrong.
Interestingly it's not deterministic, most of the time the thing works as expected šŸ™‚
w
I just ran a format, and that message didn't appear. I would have put a ton of money on having seen that message a lot more.
l
For the record, our diff of the 
bump
plugin update is like this:
w
Aha, there we go - an "unexpected" file change caused this.
Copy code
15:13:45.57 [INFO] Initializing scheduler...
15:13:47.83 [INFO] Scheduler initialized.
15:13:48.86 [INFO] Filesystem changed during run: retrying `Fmt` in 500ms...
15:13:51.27 [INFO] Completed: Format with Black - black made no changes.
15:13:51.41 [INFO] Completed: Format with docformatter - docformatter made no changes.
15:13:51.60 [WARN] Completed: Format with isort - isort made changes.
  src/python/pants/backend/cc/util_rules/compile.py
l
How did you trigger the change in the other file? I've been trying to 
touch libs/wsi/various_files
in a concurrent 
while sleep 0.01s
loop, but I can't seem to trigger the retrial consistently šŸ˜„ (but my 
bump
task is quite fast, maybe too fast most of the time)
w
I saved one of my python files while the engine was initialized, but not yet formatting - took a couple of tries. I think the message you're seeing here makes sense. However, it's a common github mention https://github.com/pantsbuild/pants/issues?q=Filesystem+changed+during+run https://github.com/pantsbuild/pants/issues/14466
šŸ‘ 1
@bitter-ability-32190 Why don't FmtRequest or FixRequest trigger the "Filesystem changed during run" message? Are they suppressed, or does the engine know to not care for those goals?
l
A mechanism for requests to state that they have side effects?
w
Essentially yeah, because fix/fmt also change your files - as intended (which is why I was asking if 
bump
being a "fix" goal plugin makes sense for your team?)
b
Yeah, they know they changed the files
šŸ‘ 1
l
But fixing goals are supposed to be idempotent right? If they run twice, the second run doesn't change anything. That's not the case with my 
bump
goal. So could I keep my 
bump
goal by making it a "fix" one and still adhere to the kind of side effects that Pants goals are supposed to do?
(I'm asking to see whether I can keep 
bump
a Pants plugin or just do a plain old vanilla python script to do it)
w
Is it by design that 
bump
needs to run multiple times on the same data?
l
No it shouldn't run on the same data multiple times. The specification is as follows:
Copy code
./pants bump dir1 dir2 dir3
should bump the patch number of the 
pyproject.toml
files in 
dir1
, 
dir2
, and 
dir3
. Of course you could pass 
dir1
multiple times, but that's not the intent.
w
Ah okay, i only ask because that's what it looks like it's doing from your example above. It runs, bumps 
wsi
by 0.0.1, filesystem change causes a re-run, then bumps by another 0.0.1, and again.
Copy code
20:58:48.15 [INFO] Completed: Bump pyproject.toml version - custombump - custombump succeeded.
 - (python) libs/wsi: 0.0.23 ā†’ 0.0.24

āœ“ custombump succeeded.
20:58:48.15 [INFO] Filesystem changed during run: retrying `Bump` in 500ms...
20:58:48.65 [WARN] {}
20:58:48.65 [INFO] Completed: Bump pyproject.toml version - custombump - custombump succeeded.
 - (python) libs/wsi: 0.0.24 ā†’ 0.0.25

āœ“ custombump succeeded.
20:58:48.66 [INFO] Filesystem changed during run: retrying `Bump` in 500ms...
20:58:49.16 [WARN] {}
20:58:49.16 [INFO] Completed: Bump pyproject.toml version - custombump - custombump succeeded.
 - (python) libs/wsi: 0.0.25 ā†’ 0.0.26
l
Yeah and the problem is the repetition of runs, we want exactly one bump. Using 
--no-watch-filesystem --no-pantsd
did the trick though. In the long run, we won't use a Pants target for this anymore.
5 Views
Open in Slack
PreviousNext
Powered by