Is there a way to get pants to work with

pytest-recording

(https://pypi.org/project/pytest-recording/)? It records network calls and saves them to a file. This way we can get deterministic tests with LLMs - which is super useful :) I searched around and I found this issue from 3 years go: https://pantsbuild.slack.com/archives/C046T6T9U/p1652455675685289 And I'm curious if someone found a way to get it to work

https://github.com/pantsbuild/pants/issues/11622 is the relevant ticket, and yeah, the sandboxing means that any snapshots/recordings get "safely" saved into the transient sandbox, and never written back to the main repository. We have teed up a wrapper script to make

syrupy

(same vein of library) work for us: https://github.com/pantsbuild/pants/issues/11622#issuecomment-1308009408. I'm not familiar with

pytest-recording

but from a quick skim of the docs, maybe one can control the VCR config

cassette_library_dir

to make that sort of hack work.

Interesting solution @broad-processor-92400! I didn't think of that. I eventually was successful using VCRpy directly. Leaving my solution below in case its useful to others: 1. Change the rule from

python_test

to

python_source

. This is mildly inconvenient, as we can't just do

pants test ::

, but we don't have that many recorded tests so it's not too much of a problem. 2. Create my own fixture like this:

import os

import pytest
import vcr

VCR_RECORD_MODE = os.environ.get("VCR_RECORD_MODE", "none")


@pytest.fixture
def vcr_session(request):
    test_name = os.path.splitext(os.path.basename(__file__))[0]
    cassette_name = f"cassettes/{test_name}/{request.function.__name__}.yaml"
    with vcr.use_cassette(
        cassette_name, filter_headers=["authorization"], record_mode=VCR_RECORD_MODE
    ) as cassette:
        yield cassette


def test_openai(vcr_session):
    make_network_call(...)
    ...


if __name__ == "__main__":
    pytest.main(["-v", __file__])

To record the tests, I do:

VCR_RECORD_MODE=once pants run ....

and to run them I do

pants run ...

Ah neat. I have a vague recollection that a python_test target is runnable too, so you might be able to get away with both (

run

for recording and

test

for “normal”)

I tried that too, but the test would fail with pants test (and not with pants run)