https://pantsbuild.org/ logo
#general
Title
# general
r

ripe-cpu-85141

07/29/2022, 3:11 AM
Hello. I have a question as I'm unsure of the right way here. I have a python package with several levels of sub packages (basically a hierarchy of directories, each with their own
__init__.py
). In the end, though, everything is packaged in a single wheel, and the whole hierarchy forms a coherent app.
./pants tailor
creates a `BUILD`file per package/directory. Is that necessary or should I have a single `BUILD`file on top with a target like
python_sources(sources=["**/*.py"])
instead?
1
h

hundreds-father-404

07/29/2022, 3:19 AM
Hi there! The number of
python_sources
does not need to correspond to the number of "apps" you have etc. What Pants is actually doing with
python_sources
is generating one
python_source
target per source file.
python_source
without an s is what Pants actually cares about. For example, dependencies are between individual files. You can run Black on an individual file etc
python_sources
and target generators solely exist for boilerplate reduction -- Some folks disagree w/ this, although the general best practice thus far has been to have one BUILD file per directory because we've found it's useful for metadata to be closer to its source. Over time, it's common to add more metadata than you started w/, e.g. test timeouts -- https://www.pantsbuild.org/docs/targets#target-generation in the docs discusses this a little
r

ripe-cpu-85141

07/29/2022, 3:54 AM
One problem I have is with the tests. I have a `tests`directory outside the packages and sometime, pants can't infer the dependencies (like when I
mocker.patch
for example). I could put dependencies manually for each test file requiring it, but I'm worried it would become cumbersome. So I wonder if I could have a big python_sources target to refer to (and if it is appropriate to do so),
c

cool-yacht-37128

07/29/2022, 5:35 AM
having a big python_sources target results in all of those being dependancies for all your tests. This defeats the purpose of using pants to a degree, since it will cause goals to be run against files that haven’t changed. Best to work out why it can’t infer the dependencies.
1
I haven’t had any issues with regular package dependancies myself: https://docs.python.org/3/reference/import.html#regular-packages can you give a simplified example of where it’s not working?
👍 1
h

hundreds-father-404

07/29/2022, 11:51 AM
pants can't infer the dependencies
This indeed might be an issue with dependency inference, which would be best to figure out. Generally, manually declared dependencies should only be needed for non-inferrable things like that you need
pyscopg2
in the binary, but you never import it Have you seen this guide? https://www.pantsbuild.org/docs/troubleshooting#import-errors-and-missing-dependencies Also this option could be useful for you https://www.pantsbuild.org/docs/reference-python-infer#section-string-imports
r

ripe-cpu-85141

07/29/2022, 4:12 PM
Ha, the second link is interesting for the mocker.patch issue, I'll test it. The other issue I have with the inference is that I'm using
importlib.import_module
to force the import of a subtree for factory registration purposes. But I think I just need to add dependencies for the file containing the factory. I'm still getting used to pants 🙂 Thanks for you help!
👍 1
h

hundreds-father-404

07/29/2022, 4:16 PM
Cool, makes sense. Pants can't always infer dynamically loaded things like
importlib.import_module
-- so there, explicit dependencies make sense We generally recommend using
overrides
when feasible so that your
dependencies
are as precise as possible, which gets you a) finer-grained caching, and b) more informative project introspection, like the
dependees
goal. The target generation docs mention this: https://www.pantsbuild.org/docs/targets#target-generation
I'm still getting used to pants 🙂
Welcome 🙂 We actually really cherish the perspective of newcomers, and welcome you to continue asking questions and sharing feedback. We try to think about "the curse of knowledge" https://en.wikipedia.org/wiki/Curse_of_knowledge, which we talked about on a podcast recently https://www.pantsbuild.org/docs/media#humans-of-devops
r

ripe-cpu-85141

07/29/2022, 4:22 PM
ah that's nice 🙂 I have another issue now but I think I'll create a separate message for it.
❤️ 1
4 Views