In V2 land, what is the unit of work for code qual...
# development
h
In V2 land, what is the unit of work for code quality tools like isort and Pylint? Would isort get a new invocation for every single Python Target, or like in V1 would it run over the whole set of targets in one invocation?
a
that should be dependent on what’s faster, not the engine executing it. in v1 junit tests we have
--fast
which makes that choice (all at once or per target) configurable
h
You mean that both are potentially valid and that we could decide which execution model to use? We would test out both models (all targets per invoke vs. one target per invoke) and determine which is faster?
a
In general per-target is good because it means that we get caching with fine granularity
But if there’s an order of magnitude difference in the perf between running something one time on 100 files, or 100 times on one file, it’s worth thinking about
w
definitely want to bias toward per-target, yea. it's also generally more correct
👌 1
👍 1
and if per-target is too slow, diving in to decompose the work that the tool is doing so that it can be split out into more processes...? (more hypothetical.)
h
And then who in V2 land determines whether the target should be ran against again? E.g. if we just ran isort on
foo.py
, then we could avoid running it again. Would this be local caching?
w
caching in general
pantsd caches things in memory, remoting caching caches things remotely, local process caching (would, after that ticket) cache things on local disk
👍 1
(pantsd caches at the
@rule
level, and local and remote caching cache only process invokes, which have stronger cache keys)
👍 1
a
As a general rule, also, all `@rule`s should be 100% cacheable in all situations, but `@console_rule`s will be unconditionally run, so if you’re planning to e.g. use
isort
to change file contents, rather than just check whether something is correct, it should be the
@console_rule
which does that, and we have not entirely worked out what API it should use to actually manipulate the filesystem
👍 1
w
unless you were to import and use
subprocess
in a
@(console_)rule
, or use
open
, it shouldn't be able to do that damage
But if you did, you might also invalidate yourself and thus re-run the next time...?