Hi everyone. I have a monorepo where each project ...
# general
q
Hi everyone. I have a monorepo where each project has a
pyproject.toml
file and a
setup.cfg
file. The
setup.cfg
file contains information such as the version, third-party dependencies, and so on. I read the documentation (https://www.pantsbuild.org/2.21/reference/targets/python_distribution#generate_setup) and it says the
generate_setup
attribute in the
python_distribution
(if set to false) will use this existing setup information. I don't think it is able to detect the third-party dependencies since I still get warning for unowned dependencies (do I need to state them explicity? I thought pants would automatically infer them from this file). Also, how to I retrieve some information explicitly from the
setup.cfg
file? For example, I want to use the version specified in that file to add to the version of the wheel distribution in the
BUILD
file of that project (and so on).
h
There are two different things in play: A) Inferred dependencies and B) distribution requirements metadata. Pants infers source-level dependencies based on
import
statements. This is for things that happen at the source level without ever building the distribution, such as running tests. Separately, when you build wheels for a distribution, Pants has to embed
Requires-Dist
metadata in that wheel. If you let Pants generate the setup.py, it will use the inferred dependencies to populate this metadata. If not, you have to populate it yourself. So, the question here is why is A) not working. And I suspect the answer is that you have no target to represent your "universe" of third-party dependencies. For example, if they were in a
requirements.txt
, you would have a
python_requirements()
stanza in a BUILD file with that
requirements.txt
as a source, which would generate a target for each requirement listed. There are similar target generators for requirements in pyproject.toml. But in your case, it's in
setup.cfg
, and we don't have a
python_setup_cfg()
target generator. So your options are: break the requirements out into a requirements.txt and use
file:path/to/requirements.txt
in setup.cfg. Or switch to pyproject.toml. Or add a
python_setup_cfg()
target generator to Pants. It should be very easy.
q
> break the requirements out into a requirements.txt and use
file:path/to/requirements.txt
in setup.cfg Could you elaborate this a bit more? I need to put
file:path/to/requirements.txt
in
setup.cfg
? Also, I also want to get access to other information (metadata) present in the
setup.cfg
file such as the name, and version. If I use a
pyproject.toml
file, how would I be able to get this information?
h
I believe setup.cfg supports
file:...
as a requirement, but that is not a Pants question
You can read more about that in setup.cfg documentation, I assume
q
Oh I see. I thought it was a pants feature. I will read more about that, thank you!
h
setup.py, setup.cfg and pyproject.toml are all competing ways to specify metadata for building distributions. This is confusing, for sure.
q
yes haha. The projects in my monorepo use setup.cfg. If I switch to using pyproject.toml, would I be able to get that information in BUILD files? For example, I want to name the wheel with the version that is specified in setup.cfg
If I switch to pyproject.toml, is there any way I can access this inside the BUILD file so I can use it in the name for example for the wheel
Does pants not parse all of pyproject.toml? Does it only parse
[build-system]
entries (https://www.pantsbuild.org/2.21/docs/python/overview/building-distributions#pep-517)?
h
Pants uses various bits of info from pants.toml, such as requirements (see https://www.pantsbuild.org/2.20/docs/python/overview/third-party-dependencies#pep-621-compliant-pyprojecttoml)
But what you're concerned about (versions and other distribution metadata) is unrelated to that
When pants is set to not generate setup.py, it doesn't parse any of that, it just runs the build backend on the input files you provide.
So yes, anything that works with setup.cfg (which Pants actually knows nothing about) should work if you switch to pyproject.toml
but maybe switching to a requirements.txt and referencing it in your setup.cfg is less disruptive?
q
I switched to using pyproject.toml and added the dependencies, name, version under the
[project]
table in pyproject.toml. I also created a requirements.txt file with the same dependecies and it is able to infer all the dependencies properly (I checked using
pants dependencies
). however, when I create the wheel using the
python_distribution
target, it only adds the name and version in the wheel METADATA file. it doesn't add the dependencies under
Requires-Dist
. Also, if I want to add custom METADATA to the wheel, I am not sure how to do that using pants?
h
You shouldn't need to duplicate the requirements in requirements.txt and pyproject.toml
you just need pyproject.toml and a
python_requirements(source="pyproject.toml")
target
And this is an example of how to tell Pants to build from a pyproject.toml that you provide. If that doesn't add requirements then possibly something is incorrect with your pyproject.toml
q
Yes I was looking at that in the documentation for reference. It says pants inspects the
[build-system]
table. Does pants also do that for
[project]
? Because I had written the dependencies under there. Also, If I want to add custom information into the METADATA of the wheel using pants, is there a way to do that?
h
Again, pants reads just the build system info out of pyproject.toml so it knows which build system to use, but then it just presents that build system with your complete pyproject.toml, so you can put anything in it, pants does not care
1