https://pantsbuild.org/ logo
r

rough-vase-83553

10/20/2022, 9:35 PM
Does
[python].default_run_goal_use_sandbox = false
allow you to write non-hermetic test cases, e.g. tests on code that reads files directly from the source tree without specifying them as dependencies? It seems like the answer is no but I'm curious how that's the case without sandboxing / hoping there is a way to (temporarily) write non-hermetic tests.
b

bitter-ability-32190

10/20/2022, 9:38 PM
It's only for the
run
goal. If you have the test runner in a file that you're
run
-nin then yes ๐Ÿ˜Š
๐Ÿฅน 1
h

happy-kitchen-89482

10/20/2022, 9:49 PM
But that would not be a good way to run tests
E.g., you would get no caching
b

bitter-ability-32190

10/20/2022, 9:50 PM
I suspect this might be for VCR recording or similar snapshot testing where you wanna write the snapshot
h

happy-kitchen-89482

10/20/2022, 9:50 PM
The non-hermeticity is for loading code that you donโ€™t have a dep on?
r

rough-vase-83553

10/20/2022, 9:51 PM
^Yeah. We just have a lot of these use cases where code searches the repo sources. It would be nice if there was a way to discard hermeticity temporarily for onboarding.
But at some point the right solution is explicit deps either in test cases or in the underlying code
h

happy-kitchen-89482

10/20/2022, 9:52 PM
lemme think about that one
๐Ÿ™ 1
is it the test that searches the repo, or the code being tested?
r

rough-vase-83553

10/20/2022, 9:53 PM
From what I can tell it's usually the code itself. But the code is often a wrapper for searching the repo
b

bitter-ability-32190

10/20/2022, 9:58 PM
Hmm this makes me think we should make it "execution use sandbox" which could be used for test as well
h

happy-kitchen-89482

10/20/2022, 10:24 PM
Iโ€™m trying to figure out what invalidation should be like in this case
It sounds like any change in any file in the repo must invalidate the tests that do this searching?
b

bitter-ability-32190

10/20/2022, 10:25 PM
Much like run, the final process isnt cached
It can't be. For something like VCR it makes network requests, which can't be cached ๐Ÿ˜…
r

rough-vase-83553

10/20/2022, 10:26 PM
I imagine any non-hermetic solution will fail to ensure that the test cache entry is still valid. It's as if you add HTTP calls to your tests
Which is not nice but... depending on how much you value purity...
b

bitter-ability-32190

10/20/2022, 10:35 PM
I wouldn't call it impure, you're just forcibly rerunning it in a way it can write to disk meaningfully
If you file an issue I'll look at it
r

rough-vase-83553

10/20/2022, 10:43 PM
Oh I see, so for these non-hermetic tests you just wouldn't cache. That works. Let me file
b

bitter-ability-32190

10/20/2022, 10:47 PM
This actually scratches an itch I've been having ๐Ÿ˜Š
r

rough-vase-83553

10/20/2022, 10:54 PM
5 Views