Does

[python].default_run_goal_use_sandbox = false

allow you to write non-hermetic test cases, e.g. tests on code that reads files directly from the source tree without specifying them as dependencies? It seems like the answer is no but I'm curious how that's the case without sandboxing / hoping there is a way to (temporarily) write non-hermetic tests.

It's only for the

run

goal. If you have the test runner in a file that you're

run

-nin then yes 😊

But that would not be a good way to run tests

I suspect this might be for VCR recording or similar snapshot testing where you wanna write the snapshot

The non-hermeticity is for loading code that you don’t have a dep on?

^Yeah. We just have a lot of these use cases where code searches the repo sources. It would be nice if there was a way to discard hermeticity temporarily for onboarding.

lemme think about that one

From what I can tell it's usually the code itself. But the code is often a wrapper for searching the repo

Hmm this makes me think we should make it "execution use sandbox" which could be used for test as well

I’m trying to figure out what invalidation should be like in this case

Much like run, the final process isnt cached

I imagine any non-hermetic solution will fail to ensure that the test cache entry is still valid. It's as if you add HTTP calls to your tests

I wouldn't call it impure, you're just forcibly rerunning it in a way it can write to disk meaningfully

Oh I see, so for these non-hermetic tests you just wouldn't cache. That works. Let me file

This actually scratches an itch I've been having 😊