Hi everyone, looking for advice in porting projects under separate repos with setup.py files into a monorepo. First question: Do I need to port the setup.py files into pyproject.toml files? Does Pants require pyproject.toml?

It does not, but it also treats these files as completely secondary. You use them to publish a distribution and nothing else generally. Pants directly works with your project python files and understands their dependencies (both 1st and 3rd party) without looking at those sorts of files at all.

Yes, we publish to an internal PyPi server.

For what purpose though? When all projects are in a monorepo will there be a need for that?

In the monorepo we want to have certain packages published at their own cadence.

Ouch, ok. Yes, but it's generally not tenable / sort of defeats the point of being in the monorepo.

Okay.

Is this a social problem or a real technical concern?

The packages that should be built on their own cadence -- should these be separate "pants monorepos"? Social vs technical: More social, I think.

Yeah. I've spent a good deal of time in monorepos where it was basically social. It's unfortunate but a common real problem fwict.

Yes. Since I have you here, what are your thoughts on the "hello world" Pants project on github for Python? It's a small project, but is one takeaway that Pants can/should be used for small, single-module repos?

Pants should have ~0 to do with size or structure of a repo. Where it does it's broken.

Regarding setup.py/pyproject.toml: Neither of these are need because Pants will infer dependencies, correct? This is interesting. So then the entry point for developers into the Pants system is just pants.toml?

Correct. And then, for Python, somewhere to write down 3rd party constraints. That can be requirements.txt or several other forms.

Great. Thanks @average-flag-94768. I'll go back to the docs now that a few things are cleared up. Thanks for your help.

To clarify, Pants will happily build wheels and publish them to PyPI (or an internal server). It can use your existing setup.py for this or generate them. John's point is that you should be very sure you need to do this. Often it is done because Python has not historically had a good story for managing multiple units/packages in a single repo, but Pants provides that. So if you must publish because you have consumers outside the repo, that must stay outside the repo then do so, of course. But it'll be much easier for you not to whenever you don't have to.

I'm watching your Youtube video on this now @happy-kitchen-89482 🙂

At least try and minimize the number of wheels you need to publish, it is such a headache compared to deploying a fully self-contained binary built at some SHA

Agreed.

Pex is that standalone single-file binary format, it is great for deploying services etc

you could consider deploying pex files to your instances that host your jupyter environment and build virtual environment from them, I've been doing that as a really clean way to basically ship an environment to a remote worker if I need it to be extendable

@swift-river-73520 that's a compelling argument and I'll look into it. Thank you.