https://pantsbuild.org/ logo
h

hundreds-breakfast-49010

02/05/2020, 10:15 PM
I wanted to get peoples' opinion on the following idea for V2 repl:
what if we don't actually have one top-level
repl
goal that tries to figure out which language's repl to run based on the targets
but instead we have many goals of the form
repl-<language-name>
, i.e. explicitly requiring the user to specify which repl they want
👍 1
or maybe even
repl-<repl-name>
in cases where there are multiple supported repl programs for a single language
this has the downside of introducing potentially many top-level goals
but has the upside that users could then regularly type things like
./pants repl-ipython ::
and have pants filter out all the targets that are incompatible with a python repl
(and in the long term we might want to introduce an new, intermediate level of goal namespacing that allows all `goal_rule`s of the form
repl-*
to be collapsed into a single entry in
./pants goals
or otherwise have similar option sets, which would mitigate the "potentially large numbers of
repl-<something>
goals in the goal list" problem)
h

hundreds-father-404

02/05/2020, 10:26 PM
If we registered every V2 backend by default, I’d be concerned about how much this adds to
./pants goals
. Because we don’t do that and you must opt in to certain backends, I’m not concerned about how many new goals it introduces It would be nice to be able to do
./pants repl-python ::
, meaning load every single Python target in the project.
./pants repl ::
doesn’t make sense because which REPL should you use? We couldn’t safely assume anything. Alternatively, we could do something like
./pants repl --language=python ::
, but if you had to do that every single time then that is a smell that we should have distinct goals
h

happy-kitchen-89482

02/05/2020, 11:26 PM
Yep, this was the conclusion I inched towards over lunch convo with Greg. Unlike say
test
or
run
, where the behavior is well-defined in all cases (I think), it's not clear what
./pants repl ::
should mean in various cases (e.g., what if you have a python target with a dep on a scala target for some reason, or what if you cannot induce the repl type from the language, e.g., you can look at java code in jshell but also in the scala repl, so languagesXrepls can be many-to-many)
Whereas
./pants repl-py
or
./pants repl-ipython
is crystal clear, and you can just ignore the non-python targets in the target set, as you would in any other use of
::
Or maybe the goals should just be
./pants python
./pants ipython
and so on
I'm not fussy about the names
h

hundreds-father-404

02/05/2020, 11:37 PM
./pants ammonite
,
./pants java
.
./pants ipython
,
./pants python
,
./pants jupyter
. I think that makes sense? Originally I was thinking
./pants repl-python --ipython
or
./pants --ipython-enabled repl-python
, but I think
./pants ipython
is better. It also naturally expresses the mutex, that you can only have one of
python
,
ipython
, and
jupyter
h

hundreds-breakfast-49010

02/06/2020, 1:26 AM
https://github.com/pantsbuild/pants/pull/9077 is a PR for a
repl-python
goal, and other types of
repl-*
goals would look much like this
w

witty-crayon-22786

02/06/2020, 4:14 AM
the current behavior (explode if ambiguous) has ~never been an issue, afaict
i'm not sure if that's because people don't glob over large swaths of their code, or what.
lots of goals is ... fine, i guess. but doesn't really seem necessary.
it should be possible to use UnionMembership to filter "repl-able" roots, and then ... group by target language?
a

aloof-angle-91616

02/06/2020, 3:45 PM
i was thinking this could use the
--only
argument like we have with
lint
? not sure if that's way off base
w

witty-crayon-22786

02/06/2020, 5:34 PM
yea, that too.
h

hundreds-breakfast-49010

02/06/2020, 7:18 PM
@aloof-angle-91616 I don't have a problem with just creating one top-level goal called
repl
and using an argument to allow the user to select which of several repls they might want
I don't think I"d call it
--only
though
so, maybe something like
./pants repl --name=ammonite ::
or
./pants repl --name=python ::
and that would do the same thing as
./pants repl-ammonite ::
or
./pants repl-python ::
a

aloof-angle-91616

02/06/2020, 7:19 PM
we may not have to do it now, but conceptually it seems similar to what we're trying to do with
--only
for linting so it might be useful to eventually converge on the same option name
h

hundreds-breakfast-49010

02/06/2020, 7:20 PM
the important thing is that the user would always specify which repl they want somehow, and pants would just filter every target that isn't compatible with that repl (and also it would be expected that multiple repls are compatible with the exact same targets, which is why we ask the user to specify)
a

aloof-angle-91616

02/06/2020, 7:40 PM
and also it would be expected that multiple repls are compatible with the exact same targets
when would this be the case?
h

hundreds-breakfast-49010

02/06/2020, 7:42 PM
as benjy mentioned, both scala and java repls can interpret java source files
and some languages have multiple repls
e.g. python vs ipython (vs. jupyter maybe)
a

aloof-angle-91616

02/06/2020, 7:42 PM
are scala and java repls different?
h

hundreds-breakfast-49010

02/06/2020, 7:42 PM
I believe so
scala is also a language that has at least two repls I know about,
sbt repl
and
ammonite
a

aloof-angle-91616

02/06/2020, 7:43 PM
ammonite could potentially work on both though, and ammonite also has the ability to import things from coursier on the fly, which is nice
h

hundreds-breakfast-49010

02/06/2020, 7:43 PM
coursier?
a

aloof-angle-91616

02/06/2020, 7:43 PM
coursier is like ivy but cooler
h

hundreds-breakfast-49010

02/06/2020, 7:43 PM
in general I would expect that any JVM repl might be able to handle source files from any other JVM language
and that's a case for requiring that the user always specify what repl they want
a

aloof-angle-91616

02/06/2020, 7:44 PM
yes, and it seems like it might be easier in that case for pants to support a single jvm repl instead of trying to support multiple jvm repls
h

hundreds-breakfast-49010

02/06/2020, 7:44 PM
once we know what repl they want, pants can figure out what is and is not compatible with it and only present compatible source files to the repl
but it can't automatically guess what repl someone wants from the source files
w

witty-crayon-22786

02/06/2020, 7:44 PM
The difference between "lots of goals" and "options" is probably just whether you can make the options compact enough.
But a distinct advantage of options is that it lets you set a default for the repo
h

hundreds-breakfast-49010

02/06/2020, 7:45 PM
@witty-crayon-22786 yeah that's a point in favor of one
repl
goal with options
w

witty-crayon-22786

02/06/2020, 7:45 PM
Which users could override on a case by case basis
Getting the options right will be interesting though, for sure.
a

aloof-angle-91616

02/06/2020, 7:45 PM
but it can't automatically guess what repl someone wants from the source files
if we have only a single jvm repl (e.g. ammonite), then this does become possible i believe?
h

hundreds-breakfast-49010

02/06/2020, 7:46 PM
I don't think we want to only support one jvm repl
w

witty-crayon-22786

02/06/2020, 7:46 PM
It is similar to the "--only" case (select named $thing from a dynamic list based on backend) I think
Agreed with Greg: there will be multiple repls per language.
👍 1
a

aloof-angle-91616

02/06/2020, 7:46 PM
and also, i would tend to assume setting up a jupyter/ipython notebook would be a different goal than asking for a repl, because jupyter notebooks require e.g. connecting to or setting up some server, which feels like it requires different configuration than a repl
h

hundreds-breakfast-49010

02/06/2020, 7:46 PM
I'm not fond of --only in this context, becuase to me it implies that you could in principle run multiple repls simultaneously, and you just happen to be restricting yourself to one
👍 1
a

aloof-angle-91616

02/06/2020, 7:46 PM
sure, if we're willing to do the extra work to support repls that nobody might use
w

witty-crayon-22786

02/06/2020, 7:46 PM
Java is getting its own repl in one of its upcoming releases
h

hundreds-father-404

02/06/2020, 7:47 PM
Java is getting its own repl in one of its upcoming releases
already has it as of Java 9 iirc
h

hundreds-breakfast-49010

02/06/2020, 7:47 PM
I think we should expect that different shops or individual users will have different preferences for one repl over another
👍 1
w

witty-crayon-22786

02/06/2020, 7:47 PM
9 is sortof a fantasy version still for us, heh. But yea
h

hundreds-breakfast-49010

02/06/2020, 7:48 PM
like, I might personally want to use ammonite for scala, even if the default is the sbt repl
👍 1
I don't think pants should get in the way of that choice
h

hundreds-father-404

02/06/2020, 7:48 PM
I think we should expect that different shops or individual users will have different preferences for one repl over another
Further, Pants should be pluggable enough that plugin authors can hook up their favorite REPL easily
👍 1
a

aloof-angle-91616

02/06/2020, 7:48 PM
if we're willing to do the work to maintain several different repls before we know whether anyone specifically wants to use them, sure
we currently allow pluggability by allowing people to override the classpath and the main class for the jvm repl
h

hundreds-father-404

02/06/2020, 7:49 PM
if we’re willing to do the work to maintain several different repls before we know whether anyone specifically wants to use them, sure
Not necessarily that we will be the ones to support multiple REPLs. Rather that Pants has an infrastructure to allow multiple
👍 1
h

hundreds-breakfast-49010

02/06/2020, 7:49 PM
pants alreayd needs to support more than one repl program, insofar as python and scala/java are different languages and one's gonna require python (or ipython) and the other is going to require one of ammonite/the java repl/sbt repl
a

aloof-angle-91616

02/06/2020, 7:49 PM
is it likely that a single repo will want to have multiple repls available at once? in a way that can't be configured via separate pants.ini.* files?
h

hundreds-breakfast-49010

02/06/2020, 7:50 PM
I do actually think the case where one individual user might want to use their own repl would be common enough
but in general a multi-language repo needs to support multiple repls for different languages simultaneously
like, if we just assume one language per repl, we still would like it if
./pants repl-py ::
and
./pants repl-scala ::
worked right without the user having to figure out exactly which targets are scala-only and which ones are python only
👍 1
but we have no reason to make that one-language-per-repl assumption anyway
👍 1
a

aloof-angle-91616

02/06/2020, 7:52 PM
hm yea ok
h

hundreds-breakfast-49010

02/06/2020, 7:53 PM
I think I do actually like
./pants repl --only=python ::
better, although I'm still not sure if
--only
is the best name for that option
h

hundreds-father-404

02/06/2020, 7:56 PM
If we go with an option, I think it would need to be something like
./pants repl --name
instead of
--only
.
--only
works well with
fmt
and
lint
because those goals usually run all linters, so you are saying only run this one. Here, you should never run more than one REPL at a time. Rather than not one vs. many REPLs, the question is which REPL.
a

aloof-angle-91616

02/06/2020, 7:57 PM
yeah the above seems right to me now
h

hundreds-breakfast-49010

02/06/2020, 7:57 PM
yeah
--name
is fine by me
h

hundreds-father-404

02/06/2020, 7:58 PM
The reason I don’t love
./pants repl --name
is that you would always need to specify the name (unless we try to infer it, which complicates everything). When an option must always be defined, then that’s a smell to be that it should be a distinct goal
./pants ipython
is more discoverable than
./pants repl --name='ipython'
. Naively, I would try running
./pants repl
without thinking to check
./pants help repl
and seeing that I need to define
--name
.
👍 1
a

aloof-angle-91616

02/06/2020, 7:58 PM
ok, but
--name
could be an enum option which could be discoverable in
./pants help repl
?
i also am pretty ok with individual goals, but
--name
feels maybe more discoverable?
h

hundreds-father-404

02/06/2020, 8:00 PM
ok, but --name could be an enum option which could be discoverable in ./pants help repl?
Agreed, but there is still the friction of needing to know to run
./pants help relp
in the first place. I can’t think of any other goal that requires always passing an option, outside of
./pants login
Users are familiar with
./pants $goal path/to:target
. Most users would try doing this the first time they run
repl
, I suspect. There’s some tension to fail and say “You need to also pass `--name='python'`”
a

aloof-angle-91616

02/06/2020, 8:01 PM
well, we can always have
--name
default to something reasonable?
h

hundreds-father-404

02/06/2020, 8:02 PM
well, we can always have --name default to something reasonable?
What would that be? In a monolingual repo and in a multilingual repo? I’m thinking before any Pants config is set up. This is a brand new Pants user who is just trying out the tool
a

aloof-angle-91616

02/06/2020, 8:03 PM
how does a user discover which repls are available?
h

hundreds-father-404

02/06/2020, 8:04 PM
./pants goals
h

hundreds-breakfast-49010

02/06/2020, 8:04 PM
it might not be a bad idea to just have
python
,
ipython
, etc. all be top-level goals
that implies that there's one top-level goal for every repl a given installation of pants supports
a

aloof-angle-91616

02/06/2020, 8:04 PM
yeah
h

hundreds-father-404

02/06/2020, 8:04 PM
And we will only show the REPLs for the registered backends. If you don’t have
pants.backend.java
, we won’t show
ammonite
as a goal. This is very similar to how we handle
setup-py
a

aloof-angle-91616

02/06/2020, 8:04 PM
this also then allows appropriate configuration for each repl to sprout instead of trying to jam all that configuration into the same options in the
repl
goal
💯 2
h

hundreds-breakfast-49010

02/06/2020, 8:05 PM
the only thing we lose doing that is that we're no longer calling these goals repls
I can see a user looking through a list of goals and wondering what "ammonite" is, if htey don't normally use scala
a

aloof-angle-91616

02/06/2020, 8:05 PM
we can make sure the goal descriptions have "repl" in them?
💯 1
h

hundreds-father-404

02/06/2020, 8:05 PM
the only thing we lose doing that is that we’re no longer calling these goals repls
In the case of
ipython
and
ammonite
, I think that is totally fine. But I do agree
python
and
java
are not great. Maybe those are
python-repl
?
h

hundreds-breakfast-49010

02/06/2020, 8:05 PM
I had vaguely suggested earlier that we might want some additional level of namespacing that doesn't currently exist in pants
💡 1
a

aloof-angle-91616

02/06/2020, 8:06 PM
hmmmmmmmmmmmm
that's a great idea
h

hundreds-father-404

02/06/2020, 8:06 PM
I can see a user looking through a list of goals and wondering what “ammonite” is,
Keep in mind they will only see
ammonite
iff their repo has
pants.backend.scala
registered
h

hundreds-breakfast-49010

02/06/2020, 8:06 PM
./python repl-ammonite <target>
might be a goal that should live "under" a
repl-
"meta-goal" in some way
that's definitely adding additional complexity to support a relatively minor use case though
h

hundreds-father-404

02/06/2020, 8:07 PM
Eh, that takes us close to what we had with Task scopes afaict
h

hundreds-breakfast-49010

02/06/2020, 8:07 PM
I'm not sure how task scopes work or if they're something we want to keep in v2
h

hundreds-father-404

02/06/2020, 8:08 PM
or if they’re something we want to keep in v2
Generally, we decided no. Tasks don’t fit into a V2 world and all options should live in a
GoalSubsystem
or
Subsystem
a

aloof-angle-91616

02/06/2020, 8:08 PM
i'm definitely just thinking
h

hundreds-breakfast-49010

02/06/2020, 8:08 PM
anyway I think for now I'm going to try to get my PR that assumes one goal called
repl2-python
merged, and we can change exactly what that's called in subsequent work
👍 1
a

aloof-angle-91616

02/06/2020, 8:09 PM
allow goals to have some tag (hierarchical?) which means they display in a specific group in the output of
./pants goals
yeah none of this was supposed to block the current PR
👖 1
h

hundreds-father-404

02/06/2020, 8:09 PM
allow goals to have some tag (hierarchical?) which means they display in a specific group in the output of ./pants goals
Benjy and I briefly discussed grouping the output of
./pants goals
. I do think that would be very helpful
h

hundreds-breakfast-49010

02/06/2020, 8:10 PM
are there any other cases we can think of where we want to be able to add some kind of heirarchy to goals?
sounds like there might be @hundreds-father-404
a

aloof-angle-91616

02/06/2020, 8:11 PM
in general, this kind of hierarchy could be useful for goals which can't be run at the same time as each other (unlike e.g. the various
fmt
and
lint
tasks)
h

hundreds-father-404

02/06/2020, 8:11 PM
Well, to clarify, Benjy and I were not discussing adding a new type of options scope like
repl.ipython
(i.e a task scope). We only discussed grouping the output of
./pants goals
a

aloof-angle-91616

02/06/2020, 8:11 PM
that is also what i thought i was discussing
💯 1
i don't know what a task scope is or why we are worried about adding it back, it's not clear to me how that could be possible in the v2 model
h

hundreds-breakfast-49010

02/06/2020, 8:12 PM
actually now that I think about it I kinda like
repl.python
as a naming schema
h

hundreds-father-404

02/06/2020, 8:13 PM
it’s not clear to me how that could be possible in the v2 model
Same, not possible with the current design and I don’t think there’s a burning need to add it back. Generally, options living on `GoalSubsystem`s and `Subsystem`s seems to work well. For example,
--isort-config
makes more sense to me than
--fmt-isort-config
and at the same time
--lint-isort-config
a

aloof-angle-91616

02/06/2020, 8:13 PM
especially if we remove all the other usages of that naming scheme
h

hundreds-father-404

02/06/2020, 8:14 PM
actually now that I think about it I kinda like repl.python as a naming schema
So, this would get really tricky with backwards compat with V1. V1 liberally uses that naming scheme, such as
repl.py
and
repl.node
. We would have clashes and competing concepts, that in V1 this refers to a task whereas in V2 this refers to a subgoal(?)
a

aloof-angle-91616

02/06/2020, 8:14 PM
repl/py
?
and then
./pants goals
could parse the goal name and generate the hierarchical output?
h

hundreds-father-404

02/06/2020, 8:16 PM
repl/py
Hm, interesting. I believe it would destroy our `arg_spliter.py`being able to disambiguate between goals vs. specs, unfortunately
a

aloof-angle-91616

02/06/2020, 8:19 PM
i believe the same issue exists with single-word goals though?
how can
ArgSplitter
differentiate between
repl
and
repl:repl
?
there are other delimiters we could use too like
repl@py
repl>py
repl\py
(not really this one though because of escaping)
h

hundreds-breakfast-49010

02/06/2020, 8:22 PM
@hundreds-father-404 I don't understand the objection to the
.
syntax
it's already the case that v1 uses tasks and v2 uses goals
☝️ 1
if, say,
./pants --v2 --no-v1 goals
output 'repl.py` and
repl.ammonite
, among others
I'm not sure how that would confuse people
h

hundreds-father-404

02/06/2020, 8:24 PM
In a world where V1 did not exist and we solely had V2,
repl.python
is neat. In today’s world, we currently have
repl.py
and
rep.node
as real tasks that you can use.
./pants repl.py path/to:target
. How do you know that
./pants repl.py
is the V1 task but
./pants repl.python
is the V2 goal?
h

hundreds-breakfast-49010

02/06/2020, 8:24 PM
right now running
./pants --v1 --no-v2 goals
doesn't output any goals with a
.
in the name
ooh I see what you mean
huh, ok
h

hundreds-father-404

02/06/2020, 8:25 PM
right now running ./pants --v1 --no-v2 goals doesn’t output any goals with a . in the name
Which is exactly the issue. Currently
goal.subname
always means one thing. We would be changing this so that
goal.subname
sometimes means a goal and sometimes means a task
a

aloof-angle-91616

02/07/2020, 1:31 AM
isn't that why we do deprecations though? and also, v1 vs v2 is like the big breaking change in general -- it seems reasonable to me to use that distinction to play around with the space left by losing v1
h

hundreds-breakfast-49010

02/08/2020, 12:31 AM
@aloof-angle-91616 for now I think we could just have a convention of using a
-
instead of a
.
-
repl-py
is not harder to type or recognize than
repl.py
, even though I think I like the look of the latter a little better. also we can start out by just making sure to name different repl goals using the
repl-*
convention without actually changing any code yet
👍 1
w

witty-crayon-22786

02/08/2020, 12:39 AM
is not harder to type or recognize than
repl.py
no one has to type repl.py right now
because an unambiguous repl is used
a lot went on in this thread... i can catch up on it, but if someone wants to summarize the arguments on the ticket, that would be appreciated.
h

hundreds-breakfast-49010

02/11/2020, 8:32 AM
w

witty-crayon-22786

02/11/2020, 3:58 PM
thanks: will look after morning meetings!