@rough-dog-96241 Thanks for chiming in. We only transitioned to monorepo a week back so definitely looking for all opinions I can get. I haven't actually checked out pants or even considered it because we are reasonably happy with sbt. But after yours and stuhood's high words, maybe I will.

Wow

Turns out that the PythonEval task is super-broken

Proof:

and

python_library(
  name = 'project_tree',
  sources = ['file_system_project_tree.py', 'project_tree.py', 'project_tree_factory.py',
             'scm_project_tree.py'],
  dependencies = [
    '3rdparty/python:pathspec',
    '3rdparty/python:scandir',
    '3rdparty/python:six',
    ':build_environment',
    'src/python/pants/util:dirutil',
    'src/python/pants/util:memo',
    'src/python/pants/util:meta',
    'src/python/pants/util:objects',
  ]
)

And

scm_project_tree.py

has

from pants.base.deprecated import deprecated

But

src/python/pants/base:deprecated

is not in the dep closure:

It even succeeds if I delete

deprecated.py

It only fails if I delete

deprecated.pyc

from the source tree (!)

😮 ew

Meaning that the eval task’s pythonpath is polluted, rendering the whole thing invalid

I noticed this while testing the new-python-pipeline version of PythonEval, which kept failing on real tasks in our repo

So spoiler alert: We probably have a lot of missing deps.

Hmm, this is true of PEX in general, it looks like. The PYTHONPATH isn’t scrubbed.

… and all our python source roots are on the PYTHONPATH when running pants from sources

If I build a pants.pex and run that, the eval fails as it should

So that’s what we need to do in CI

@salmon-fish-51828 I spent about a year in an SBT monorepo. I just basically can't un-recommend that strongly enough.

Very hard to not wind up redoing a ton of work each time you build anything. Gets slow. Its various features are all things that can be added into pants if you miss them. Pants is good at the fundamentals: only do what needs to be done, and no more. And some of the benefits of sbt, like zinc, are actually used by pants!

@rough-dog-96241 Appreciate your advice. I'm hesitant to move away from sbt because: 1) sbt has a large community. 2) I understand sbt reasonably well. Having been in that position earlier, mots people who don't like sbt don't like it because they don't understand it. (I'm not making a comment on you, more on the old me). 3) I like the feature: compilation on change 4) Certain projects support only sbt (like scala.js). I don't like that it is very slow and I don't know the feature set of pants but I'm thinking of taking things very slow. I don't want to disrupt our current setup too much so that team can evaluate monorepos in isolation as to how it is helping us. But I definitely get your point and I will actively monitor how much and when sbt is trying to hold us back

@salmon-fish-51828 Sounds reasonable. Honestly, I've been lurking around this slack for quite some time (over a year, for sure), and was unwilling to take the plunge myself. Seems daunting. A lot of folks (I think, anyway) use pants next to other build tools. (I think the Foursquare blog post suggested that they first started using it just for unit tests or some such.) Personally, in retrospect, I wish I'd have dived in sooner. I had the opposite experience with SBT -- at first, I loved it, because things just magically work. The bigger the project became, and the more I learned about it, the more I grew to feel like I was fighting against it. The move away from Build.scala hurt me because my IDE stopped auto-completing stuff for me. 🙂 I tried to port that project to pants and it was hard -- too many built in SBT assumptions that required refactoring and I didn't want to shake the team with that kind of wave. I changed companies about 8 months ago, and this time, we're pants from day one, and I just build support in any time we need something. I was a python developer in a past life, and there are plenty of examples to base your code on, so I've found plugin development pretty easy so far. Regarding (3), above, pants runs zinc in a hot JVM in the background, so even though you have to re-type

/pants run ...

or

./pants compile

or whatever, it picks up where it left off quite quickly, which is fine for our vimmers. I use the IntelliJ plugin for quick feedback, and it's awesome -- compared to sticking your whole sbt monorepo into intellij, which slows it to a crawl. Regarding (4), I don't know much about scalajs, but that sounds like a code generator, which could be supported similar to pants's support for thrift, I think. But I might be wrong. 🙂

Can't argue with the size of the SBT community, but it's not the size that matters man! Or something. 🙂

Maybe a year from now I'll hate it as much as I grew to hate SBT. Here's to hoping not.

Foursquare has some compile-on-change functionality in Pants, but I can’t remember exactly how we made that work.

Pants has scalajs support BTW

cc @salmon-fish-51828 @rough-dog-96241 ^

Back in 2011 we hosted a Scala meetup and one of the talks was by one of the SBT contributors, who claimed that even though symbolic DSLs have a bad rep, he thinks SBT was “DSLs done right”, while the slide showing the 10 different flavors of sideways-chrismas-tree operators was still on the screen. The room erupted in laughter...

Another point to keep in mind is that Pants is intrinsically supportive of heterogeneous codebases. SBT won’t do much for you outside the JVM domain.