https://pantsbuild.org/ logo
#general
Title
# general
c

cuddly-window-48195

10/09/2019, 6:55 PM
Anyone know if
pants test
can skip reruns if dependencies weren't changed?
a

aloof-angle-91616

10/09/2019, 6:55 PM
hm, it should. which type of testing are you doing?
c

cuddly-window-48195

10/09/2019, 6:57 PM
python_tests
a

aloof-angle-91616

10/09/2019, 6:58 PM
hm, interesting. are you making any changes at all in between runs?
c

cuddly-window-48195

10/09/2019, 6:58 PM
I'm seeing it in CI; we are making changes, but as far as I can tell they aren't affecting our dependency tree?
a

aloof-angle-91616

10/09/2019, 6:59 PM
could you describe the type of changes you're making? there may be some unexpected invalidation we can remove
h

hundreds-father-404

10/09/2019, 7:00 PM
If this is in CI, what does your caching look like? Danny would they need to cache
.pants.d/
?
c

cuddly-window-48195

10/09/2019, 7:01 PM
Our CI runs with the same pants cache (CircleCI manages sharing our pants cache across jobs).
a

aloof-angle-91616

10/09/2019, 7:01 PM
yes, also just having a local/remote artifact cache should also work. there's a specific set of choices made in the pytest runner on how caching works when you pass some tests and not others which may be relevant
c

cuddly-window-48195

10/09/2019, 7:02 PM
In this case, the change was a single line to a test target we'll call
A
which caused a completely unrelated test target
B
to run. Both tests are tagged with the
functional
tag, and we're running via
./pants tag=functional test.pytest ...
a

aloof-angle-91616

10/09/2019, 7:03 PM
ah
this may be exactly what the v2 test runner could solve
a lot of our python pipeline right now resolves and invalidates using all the targets in play as the cache key
the newer python test task does not do this, and we have been using it in our CI for a bit thanks to @hundreds-father-404
💯 2
i would need to check whether my suspicion is correct (that it is caching using all the targets as a key)
c

cuddly-window-48195

10/09/2019, 7:06 PM
Ah, so the old version sees that both
A
and
B
are in play, so it hashes the union of both, while the new version hashes them independently?
a

aloof-angle-91616

10/09/2019, 7:06 PM
yes -- but! i also think this can be done in v1
i think that
--fast
is enabled by default
if you use
--no-fast
then you get per-target caching
(verifying this)
c

cuddly-window-48195

10/09/2019, 7:07 PM
ah, cool. I'll give it a try.
a

aloof-angle-91616

10/09/2019, 7:08 PM
yes, ok.
--fast
is on by default (that's in the
test.pytest
scope), and if you disable it on the command line or (probably) in
pants.ini
, things should work
you'll need to run it once with
--no-fast
first to populate the cache with per-target keys of course and then it should be cached
if not, let us know so we can fix it!
note that
--no-fast
works by doing everything in its own process. the v2 engine allows us to do this much more efficiently and there are efforts to do the same for the whole python pipeline
c

cuddly-window-48195

10/09/2019, 7:10 PM
Is v2 production ready? Or where can I learn more about it?
h

hundreds-father-404

10/09/2019, 7:11 PM
The UX isn’t totally there, like not supporting passthru args, but it otherwise works great! You run it like
./pants —no-v1–v2 test
(double dashes, am on phone and it autocorrected)
💯 1
Another big benefit if you still have any python 2 targets is that it resolves the interpreter per target. So you can have some tests that use Py2 and some that use Py3 and call them all in the same Pants run
💯 1
a

aloof-angle-91616

10/09/2019, 7:40 PM
@cuddly-window-48195 https://github.com/pantsbuild/pants/blob/master/src/rust/engine/README.md we are constantly iterating on this document
👍 1
the status of the v2 python pipeline specifically (as opposed to the engine itself) might be in an issue somewhere?