Hi, I am having trouble running tests that use loc...
# general
c
Hi, I am having trouble running tests that use local files. What is the correct way to include files as resources for testing? My directory layout is like this • pants.toml • pants • service ◦ BUILD ◦ examples ▪︎ data • data.csv ◦ sub_dir_1 ▪︎ BUILD ▪︎ <other files> ◦ test ▪︎ sub_dir_2 • BUILD • test_service.py service/BUILD has this:
Copy code
resources (
    name="test-data",
    sources=["./examples/data/*"],
)

pex_binary(
    name="service",
    entry_point="service.py",
    platforms=[
        "current",
        "manylinux2014-x86_64-cp-39-cp39",
    ],
    dependencies=[":test-data"],
)
test/sub_dir_2/BUILD has this:
Copy code
pex_binary(
    name="test_service",
    entry_point="test_service.py",
    dependencies=["service:test-data"],
)
I am trying to package everything as a pex file. A test in test_service.py uses the files as such:
Copy code
path = str(pathlib.Path(__file__, "../../../examples/data/data.csv").resolve())
do_something_with(path)
Checking the sandbox, indeed, the files don't exist(actually no resources exist). However, I do see the files in the packaged .pex archives. I am trying to run the test as such
Copy code
./pants --keep-sandboxes=on_failure test service/test/sub_dir_2/test_service.py -- -k test_csv
I am using pants 2.13.0. I would really appreciate any pointers on how to go about including files in tests. TIA, CSN
h
Try loading the file using a resource loading API, such as https://docs.python.org/3/library/importlib.html#module-importlib.resources
🙏 1
But also: try adding the explicit dep on the resources to the python_sources target that owns
service.py
, rather than to the pex_binary
🙏 1
Also, using
pex_binary
to represent tests is unusual. Any reason you’re not using
python_tests
?
c
Hi @happy-kitchen-89482, Thank you very much for the quick response! Adding explicit dep (on the resources target) to the python_sources and in python_tests worked. ref. why pex_binary: I am a pants noob (and a big fan after using it!). When I ran tailor on the directory, it created both a pex_binary and a target for python_tests. Since I was able to list the tests, I naively assumed things were as they must be. This is what my BUILD in test/sub_dir_2 looks like:
Copy code
python_tests(
    name="tests",
    sources=["test_service.py"],
    dependencies=["service:test-data"],
)

python_sources()

pex_binary(
    name="test_service",
    entry_point="test_service.py",
)
Thank you so much for the help! qq: Is there a way to force pants to rerun the tests? I wanna experiment a bit with my setup and pants returns the memoized results.
h
Ah! I’m guessing your
test_service.py
has an
if __name__ == '__main__'
stanza in it? That will cause
tailor
to guess that it’s a binary and generate a
pex_binary
target for it.
Do you normally run that test via
pytest
, or as a binary?
c
I run it through pytest and indeed there is that stanza! Is it safe to delete the binary target?
h
It is, unless you plan to run this file like a binary, but generally you can just run it as a test, it sounds like?
c
Yes sir, that is pretty spot on. Again, thank you so much for the help!
h
You’re welcome!
🙏 1
Out of curiosity, why does the test have that stanza?
c
I must admit I don’t know the answer to this question. I picked up this repo from a coworker to test out build systems. I have only ever run this with pytest and now that you mention it, does seem curious. Let me check and revert back.
Sorry took a while, for closure: • The test was being run in an ipython workflow as an executable hence the stanza. • ref resource loader: Apparently importlib.resources will not import static files if they are namespace packages prior to v 3.10. Seems like a gotcha to me, sharing here should someone else be in the same boat.
h
Thanks for closing the loop!
👍 1