One thing I'm evaluating is related to developer ergonomics. There's a few things that we've done in some other projects, as well as some tooling thoughts. Going to make a few posts here to allow for proper threading, sorry for any spammyness 🙂 🧵 First is that we're used to having the ability to specify different dependencies in development vs. our release environments. Specifically, this is useful for dependencies such as

psycopg2

which have significant burdens for local development (especially on some OSes) while realizing possible wins when the code is deployed. What I've done in my poetry projects is to have

pg8000

setup as a dev dependency and then have a specific extras section for a production only dependency after flagging it as optional.

[tool.poetry.dependencies]
psycopg2 = { version = "~2.9.3", optional = true }

[tool.poetry.dev-dependencies]
pg8000 = "~1.29.1"

[tool.poetry.extras]
production-only = ["psycopg2"]

This has reduced the burden on our developers while maintaining api compatibility and the code is largely unaware of the distinction. (I only have one place where I have to do multiple imports and that's for a try/except block). From what I can see there's not an obvious way to do a similar dichotomy when using pants and this is a considerable regression that I'm having to weigh. Is there anything I'm missing? Could a plugin enable such a pattern?

Hmm, musing aloud here: can this be achieved by having two resolves/lockfiles, one for dev and one for prod, I wonder?

I don't think multiple lockfiles/resolves is necessary. You can easily use transitive exclude on the

pex_binary

+ the "prod" dep

That too, yes

@important-psychiatrist-39230 see https://www.pantsbuild.org/docs/targets#dependencies-field

You can use the prefix

!!

to transitively exclude a dependency, meaning that even if a target's dependencies include the bad dependency, the final result will not include the value.

is relevant (and is also relevant for your below thread about

rich

. Macros are here: https://www.pantsbuild.org/docs/macros

I'l reread this in the morning, but I'm not really tracking how I would use the transitive exclusions to switch between the two.

The idea is not switching between the two, the idea is instead excluding dev dependencies from production artifacts. You'd add excludes to

pex_binary

targets, for example, that depended on `python_sources`targets that had the dev-only deps baked in.

I don’t think you’d want to add those deps as if they were tool requirements though? That would emulate what poetry et al do, but it’s not very idiomatic for Pants.

Agreed - I'm just trying to point out how we model dev dependencies, which is tool-by-tool wherease things like Poetry model it as 1 big bag of all the tools.

To figure this out more precisely, I’m curious about how your code switches between the two? AFAICT to use pg8000 you import from

pg8000

and to use psycopg2 you import from

psycopg2

? What does the code that disambiguates the two look like?

We're sorely missing someone adept at both worlds here to play Rosetta Stone

Yeah, most python tooling works by throwing the tool requirements and your code’s requirements into one venv and assuming they don’t collide. “dev” vs “prod” is implied by some external info like an env var.

Presumably on the `pg8000`thing, django-ish? Db hidden behind pluggable driver.

With pants whether something is “dev” or “prod” is implied by the target type

Yeah, and making ~duplicate targets is getting easiser all the time but still a crappy way to do things afaict.

Not sure we need duplicate targets though

run is both as is repl

All this to say, @important-psychiatrist-39230 if you check out macros and find it's too brittle, we can help you write a very simple plugin which should do this across the board (exclude/include the dev/prod dep based on target type)

Hey guys catching up. 🙂

BTW in dev you can

./pants run services/swizzler/main.py

(or whatever the entry point file is) and that is a lot faster than running the target (running the target is the same as building the pex and then running it)

@happy-kitchen-89482 we're also having a parallel related convo here: https://pantsbuild.slack.com/archives/C046T6T9U/p1664838855556019 I had an interesting idea where someone could model "dev_dependencies" in a plugin: https://pantsbuild.slack.com/archives/C046T6T9U/p1664904603042689?thread_ts=1664838855.556019&cid=C046T6T9U (and the subsequent message)

as odd as it is, I'm not sure I want to tell devs to run the entry point directly. especially given the inherent verbosity of the command that'd create. I think it'd need to be something like this:

pants run services/swizzler/src/swizzler/main.py:app

no dev on my side is going to think that's great 😕

You used to be able to use a `target`target like an alias to shorten up paths but this appears to no longer work. You can add global aliases in pants.toml by configuring `[cli.alias]`as described here https://www.pantsbuild.org/docs/reference-cli#alias and dog-fooded here https://github.com/pantsbuild/pants/blob/7acbcda329db6fb9a2ebcd0a547c3e0ab123ba0c/pants.toml#L80-L82 I think those are the only historical and present means of dealing with typing too much.