i think i've found a bug, or it may be expected behavior, but i'd like yalls opinion before i file it on gh: whenever i add a dep to

requirements.txt

, that causes nearly all targets in my repo to be counted as an affected transitive dependency in

./pants --changed-since= --changed-dependees=transitive *

commands. I suspect the target list is all targets that are using dependency inference, but i haven't confirmed this yet. Is this behavior expected? It seems like all dependnecy inference targets are depending on

requirements.txt

as a whole, which is bad because it means any change in requirements will trigger tests and rebuilds of everything in the repo

Ah, yeah that’s not great. Expected, but not desirable The issue is that the python_requirements macro adds a dependency on the concrete file. So when you change that file, every 3rd party req ends up being transitively changed I think we’ve had an issue open for a long time for how to have macros better handle reading from files and how to cache it correctly. That python_requirements() macro is one of the only things still left over using v1 APIs

is the workaround to add the requirements to my BUILD file? i can script that lol, i'm fine with it, what is it,

python_requirement_library()

?

Or copy all this https://github.com/pantsbuild/pants/blob/master/src/python/pants/backend/python/macros/python_requirements.py, but remove 84-87 and 100 register like this https://github.com/pantsbuild/pants/blob/ea2c5d6848dba328fbdde193f793905f103f3c72/src/python/pants/backend/python/register.py#L44-L52, but use a unique name

i feel like just updating my root BUILD file is safer, is there something i'm missing there?

Like, adding a

python_requirement_library

to your BUILD file for everything? I think you will still hit a similar issue - when you change a BUILD file, every target defined in that BUILD file is invalidated because we can’t detect which targets you changed (we don’t persist to disk what the target’s metadata was before)

hmm okay i guess i'll take having to restart pantsd then

moving the declarations to BUILD files would avoid needing to restart pantsd.

i'll play around with it

but as Eric said, you’d need to move each declaration into its own BUILD file to avoid the issue with `--changed`… that one goes a bit deeper even than the macro support. i think that i know roughly how to solve it (executing

--changed

logic against our runtime graph rather than the build graph), but have been waiting for the plugin API to settle down a little bit before tackling it.

That’d be awesome!

—changed

is one of the most used features in pants, so making it more precise is highly leveraged

yeah i honestly don't care about macro support, those are just details to me;

--changed-*

working right is everything that matters to me. It's the difference between a working and non-working monorepo solution

no, that would work to solve both issues.

Nope, it’d still work fine! Only downside is that now you have a bunch of BUILD files and it might be annoying to maintain and possibly confusing whenever you need to add an explicit dependency But you could use a naming scheme like • 3rdparty/python/django • 3rdparty/python/ansicolors Because there’s only one target in each directory, you can leave off the target name and use the default of the directory. So to add an explicit dep, it’s simply the path to the folder, no colon needed

agree it's a bit ugly, but i like ugly solutions that fit my use cases better than pretty ones that don't. Thanks for the tip. I can add my feedback to the linked GH issue if it helps, or should I add another that focuses on the

--change

invalidation granularity?

yea, it’s likely medium/long term enough that i didn’t want to derail this thread with it.