Hey guys How s it going I m working on a PR to restrict cert Pants #general

Hey guys! How’s it going? I’m working on a PR to r...

curved-vase-73456

11/12/2019, 6:14 PM

Hey guys! How’s it going? I’m working on a PR to restrict certain types of dependencies between packages in our repo and using foursquare tags plugin for that https://github.com/foursquare/fsqio/tree/master/src/python/fsqio/pants/tags, that automatically tags and validate dependencies. But also made we wonder if there a way maybe in current pants functionality or supported pants plugins to do that. Also, wonder if

target-tag-assignments

can take in directories and tag all targets recursively and if it can if there’s a way to run it before plugins like fsq one.

aloof-angle-91616

11/12/2019, 6:15 PM

these are fantastic questions!!! i am in a meeting for the next hour and a half but will get back to this i swear!

average-vr-56795

11/12/2019, 6:16 PM

@red-balloon-89377 has some slightly unfinished code somewhere which does this, but he’s on vacation until later this week

curved-vase-73456

11/12/2019, 6:16 PM

@aloof-angle-91616 ty so much! Appreciate it and surely wait for enlightenment around these topics 😃

average-vr-56795

11/12/2019, 6:17 PM

https://github.com/pantsbuild/pants/pull/6541 has work in progress, not sure how out of date it is

aloof-angle-91616

11/13/2019, 6:08 PM

we have a task called

validate_tags.py

in our monorepo which we haven't upstreamed for no particular reason

aloof-angle-91616

11/13/2019, 6:09 PM

ooooooh, this is very interesting (looking at the foursquare tags plugin now)

aloof-angle-91616

11/13/2019, 6:12 PM

i spent some time this weekend hacking away at a

--query

option, discussed in https://github.com/pantsbuild/pants/issues/7346 and implemented across a couple open PRs right now, and this seems like it could share implementation with that

aloof-angle-91616

11/13/2019, 6:13 PM

ok so as @average-vr-56795 mentioned, we've been calling the type of validation that you're looking for "dep banning" internally, which is different than our internal

validate_tags.py

task

aloof-angle-91616

11/13/2019, 6:18 PM

@curved-vase-73456 i think the full answer to your question is: (1) we definitely absolutely want to support this feature in upstream pants (2) one of our contributors has the dep banning implementation in https://github.com/pantsbuild/pants/pull/6541 that does something similar to the foursquare tags plugin that we'd maybe be able to pick up soon, but it's not clear whether we'll be able to spend twitter time on it immediately, so may have to happen more slowly in free time (3) it sounds like we might want to align our interface with the foursquare tags plugin if we hadn't previously been aware of it!

Also, wonder if
target-tag-assignments
can take in directories and tag all targets recursively and if it can if there’s a way to run it before plugins like fsq one.

Could you elaborate on this? I think this is definitely possible, especially given the infrastructure developed in the

--query

PR for manipulating targets by tag, but not sure I'm 100% clear on what you're asking. Could I ask you to make it into an issue?

curved-vase-73456

11/13/2019, 6:25 PM

as sure @aloof-angle-91616, so the way fsq plugin works is

path

tags

, while

target-tag-assignments

is working in reverse direction

tag

paths

. Which I wanted to combine, like instead of

Copy code

[tag]
by_prefix:  {
    'src/jvm/com/thesigma':['sigma_jvm', 'dependees_must_have:sigma_jvm'],
    'test/jvm/com/thesigma': ['sigma_jvm'],
  }

I’d like to write

Copy code

[tag]
by_prefix:  {
    'src/jvm/com/thesigma':['dependees_must_have:sigma_jvm'],
  }

[target-tag-assignments]
tag_targets_mappings: {
    'sigma_jvm': ['src/jvm/com/thesigma', 'test/jvm/com/thesigma'],
  }

But, also would to expand this for not only prefix match but a more flexible targets selection smth like

src/jvm/com/thesigma/::/service

but the later could only be possible if the original plugin also supports it

curved-vase-73456

11/13/2019, 6:27 PM

so the 2nd thing doesn’t work when I run

./pants validate-deps

(had to rename original

validate

to smth else to avoid scope dups error) while the 1st one is fine

aloof-angle-91616

11/13/2019, 6:27 PM

(had to rename original
validate
to smth else to avoid scope dups error)

ugh, annoying, sorry

curved-vase-73456

11/13/2019, 6:28 PM

does this make sense?

👍 1

aloof-angle-91616

11/13/2019, 6:29 PM

yes!

aloof-angle-91616

11/13/2019, 6:29 PM

thinking

🙏🏼 1

aloof-angle-91616

11/13/2019, 6:43 PM

basically,

--query

and other v2 rules run "before" any v1 tasks, but they also can't assign tags to targets that can then get picked up by a later task. two ways i can think of to fix this: (1) use buildozer (https://github.com/bazelbuild/buildtools/blob/master/buildozer/README.md) to assign tags to targets "manually", before running pants, then use the foursquare plugin to act upon those tags (2) extend the foursquare plugin task to read from "tags" stored by a previous pants task and interpreting it specially (3) once

--query

ships, building dep/tag validation on top of that

👌🏼 1

curved-vase-73456

11/13/2019, 10:57 PM

thank you, I’ll probably combine 1 and 2 in some way to make it work. Also, is there a reason this plugin cannot be like added to pants almost directly (after cleaning some fsq specific hard-coded things and like modifying scope to

validate-deps

or smth)?

curved-vase-73456

11/13/2019, 10:57 PM

I mean to pants repo

curved-vase-73456

11/13/2019, 11:00 PM

I get that supporting third party things is an extra burden but like it works and it’s a useful feature… maybe I heavily underestimating amount of work it takes to properly bring it over

Open in Slack

Previous Next