:thread: I have a similar dev dependency pain poin...
# general
i
๐Ÿงต I have a similar dev dependency pain point that I'd like to find a better solution for. While working out a logging setup, I'm settling on using
structlog
to help generate structured json based logs. However, during development those logs are effectively useless. In general this is fine because I can vary the configuration to output easy to parse logs when you're in running locally. However, to get some extra high quality formatting in development we need to have an extra library available:
rich
To have
pants run
work and load up rich into the dev time pex file, I've had to add this to one fo the build files
Copy code
python_sources(
    dependencies=['//:deps#rich']
)
This now works fine when running locally. However, that library has -zero- reason to exist in our hosted environments, but it will continue to be part of the pex file given the above configuration. Is there a lever to manage this that doesn't lead to significant overhead and duplication?
b
i
I'm not sure what you're asserting I'm missing. The underlying issue is that I need that line when I'm doing
pants run
but when I do
pants package
that dependency has no value and just chews up space.
e
You can't link to the blue callout boxes; so I think Josh meant the "Ignore dependencies with
!
and
!!
" callout box there given his response on another of your threads. In other words, apply transitive excludes to `pex_binary`targets to rid production of things like '//deps#rich' not needed there.
๐Ÿ‘ 2
i
gotcha. Yeah I guess that'd work but that means my dev run commands couldn't use the same target as prod does
b
gotcha. Yeah I guess that'd work but that means my dev run commands couldn't use the same target as prod does
They already will in a way. In Pants you can run the source file
./pants run path/to/file.py
which is faster than running the
pex_binary
. Prod will be using the
pex_binary
, whereas devs can chose but usually prefer the speed of running the source. So it's a good natural seam to have the
pex_binary
targets transitively exclude the dev dep and add the prod dep. At least, that;s my experience.
The only difference between the two targets, if you overly-simplify things is just the packaging/running + whatever metadata you tie to the
pex_binary
. Anything else is a Pants bug IMO
i
That's going to be a regression in developer experience for us. Right now we can just use
poe dev
and it starts the dev server, there's no knowledge of how it's running needed. While that domain expertise isn't a Bad Thing โ„ข๏ธ, it will result in a more weaker development experience. ๐Ÿ˜•
b
Not sure I follow completely ๐Ÿ™‚
i
A meta thing I'm getting in a few spots here is that we should be using a different target in dev versus production. By having people running source targets in dev the evocation of said targets becomes more complicated and less turn key.
e
A problem we run into in conversations is that they often devolve into technological discussions: A: "I am used to doing X (with tool Y)" B: "Ah, with with Pants you do Z" e\Etc. What's needed is a focus extracting the what I want to get done from A and leaving the parts specific to tool Y out and then deciding if / how Pants can support that. Basically we only get workflows supported well from what I can tell if the active Pants devs actually use those workflows themselves. When this isn't the case, we get gaps.
Afaict we suffer from a lack of manpower / time to devote to proactively ferreting out gaps in dev experience from other tool sets. Information like yours helps make us aware retroactively that certain dev experiences are not supported, but then there's the issue of actually getting to work and closing those gaps.
In your case in particular - FWICT we have no active Pants devs that have used Poetry extensively.
I've used tox extensively, but that's the only other Python tooling I've used besides Pants. That makes for some pretty massive blind spots.
b
I used to use
poetry
extensively at my last gig. I think in a few dimensions its just a fundamentally different way of doing things ๐Ÿ˜• Poetry (and even tox if configured) makes a venv with everything in it, whereas Pants is "nothing more than exactly what is specified (implictly or explicitly)". That being said, I can see a world where
dev_dependencies
exists as a field. I think the logical hurdle is just how to know when it should be considered or ignored.
That'd actually be completely doable as a plugin, if having a flag was the delineator:
Copy code
python_source(
    ...,
    dependencies=[...],
    dev_dependencies=[...],
)
then:
Copy code
./pants run/package/etc... path/to/target
represents "prod" (E.g. ignore dev deps) and
Copy code
./pants --dev-deps run/package/etc... path/to/target
would mix those in.
@important-psychiatrist-39230 if you're willing to file an issue I can braindump the outline of the solution. And then if you want help implementing the plugin, let us know. We'd love to have that be added to the repo โค๏ธ
i
I'm also just wondering if there should be a distinct
dev
goal, which is more in line to how a lot of ecosystems think of it.
b
FWICT It'd have to be a flag, since we don't have subgoals, and you'd want to at least support
run
and
package
(that being said, I'd absolutely love subgoal support, but that's yet to be thoughtdumped how it'd be possible)