https://pantsbuild.org/ logo
g

gorgeous-winter-99296

10/20/2022, 7:47 PM
When packaging a python project; is the intention that passing a
pyproject.toml
to
python_distribution
the code isn't packaged if set via dependency. E.g.
Copy code
dependencies=[
        ":pyproject",
        "//pants-plugins/backends/kubernetes:kubernetes",
    ],
packages no code, while
Copy code
dependencies=[
        "//pants-plugins/backends/kubernetes:kubernetes",
    ],
properly includes all code. I believe from the website the first variant should work too. It does get traversed while finding deps, but not actually included.
I guess this is what I'm hitting https://github.com/pantsbuild/pants/issues/12487 and the docs are wrong? In the sense that I have to duplicate all the info that pants knows into my pyproject.toml 😢
h

happy-kitchen-89482

10/20/2022, 8:26 PM
Just double-checking: are you setting
generate_setup=False
on the target?
Are we in the context of wanting Pants to generate the setup.py, or not wanting it to? 🙂
🙏 1
g

gorgeous-winter-99296

10/21/2022, 8:04 AM
I tried a bunch of variants. I guess it makes sense it doesn't work; since "where does the code come from" depends on tool... I'm using
pdm
, and for that one I had to put the below block in my pyproject.toml. Maybe another one would work with pants directly?
h

happy-kitchen-89482

10/21/2022, 3:07 PM
I’m not familiar with
pdm
, but it looks like it’s a pep-517 build backend? So, yeah, today there are two options: “Pants runs your build backend, but you have to write its config”, in which case you follow https://www.pantsbuild.org/docs/python-distributions#pep-517, or “Pants generates a setup.py and runs setuptools for you”. It sounds like you want the former.
So you need the
[build-system]
table in
pyproject.toml
to specify your backend, in this case, I guess, pdm
And you need to explicitly depend on the entry points to be packaged into the distribution (their transitive deps will be inferred)
A great way to proceed with figuring out why this isn’t working is: A) Opening a Q&A discussion for this here, so this doesn’t get lost in the ephemerality of Slack: https://github.com/pantsbuild/pants/discussions B) Creating a simple repro repo on github that we can base the discussion on Because it seems like you’re doing the right thing, so I’d like to find out why it’s not quite right
g

gorgeous-winter-99296

10/21/2022, 3:14 PM
Yeah; I think my expectation was that I'd have the code also passed to the build backend in a way that it gets packaged - but since that isn't actually part of pep517; it seems quite hard for pants to solve. If I understand your correctly, it'd correctly pass dependencies since that's a standardized key? I don't think anything is actually wrong here; if I pass the information for
tool.pdm.build.includes
it does correctly bundle the code. I just didn't expect that to be required...
h

happy-kitchen-89482

10/21/2022, 3:16 PM
Oh, you mean pass it in to
pyproject.toml
??
So @future-oxygen-10553 is looking into having pants inject data it knows into a
pyproject.toml
template, similar to how Pants does this internally today for setup.py
but a lot better
That might sort this
So instead of “generate setup.py, yes or no?“, it’s “use this pyproject.toml/setup.cfg/setup.py/whatever, but replace any
{{}}
placeholders in it with the appropriate data”
g

gorgeous-winter-99296

10/21/2022, 3:19 PM
Yeah, that's his issue I linked at the top 😛 Maybe I wasn't clear then; but yeah. I don't think anything here is broken, just that the docs suggested that depending on code would make it get packaged - but it just gets put into the sandbox for packaging and I have to actually say in my pyproject.toml what files should be included. Which realistically should've been part of PEP517 or PEP621 but like anything in Python packaging it's 90% working. 😛
And yeah, placeholders in some way would work. But since different backends have different formats for passing that I'm not sure how realistic it is for Pants to generate it anyways. Maybe Pants should have/needs a blessed build backend that it knows how to deal with for that to work well.
(In practice; I tend to just choose the backend that's suggested by the frontend I'm using anyways - poetry-core for poetry, pdm517 for pdm, hatchling for hatch, etc.)
f

future-oxygen-10553

10/21/2022, 3:29 PM
but it just gets put into the sandbox for packaging and I have to actually say in my pyproject.toml what files should be included.
@happy-kitchen-89482 I’m actually concerned about this aspect as well for the PEP 517 generation. Each backend has its own configuration method to figure out what files should be packaged, especially if the package name is different from the import name.
So doing this in the general case actually turns out to be a little tricky. I think this was part of why I mentioned trying to parse the user-provided template
Because pants is going to need to add/modify fields, not just write them out wholesale, at least to my inexperienced eyes
h

happy-kitchen-89482

10/21/2022, 3:54 PM
Hmm, yeah, tricksy. Let’s follow up on the issue?
f

future-oxygen-10553

10/21/2022, 3:54 PM
👍
h

happy-kitchen-89482

10/21/2022, 3:55 PM
f

future-oxygen-10553

10/21/2022, 3:55 PM
I think we can probably take a stab at the implementation as discussed in the issue and then see what problems arise
6 Views