https://pantsbuild.org/ logo
Join Slack
Powered by
I flagged this a couple of years ago - but what is...
# development
w
I flagged this a couple of years ago - but what is the reason for Java/Scala backends to be enabled in the main pants repo? https://pantsbuild.slack.com/archives/C0D7TNJHL/p1660451460300279 Every time I run pants on a fresh repo (on python code), I get stuck with 
coursier
stuff - which doesnt play well with Lulu network blocking
I'm guessing that, if we're using the Java tooling to handle jvm stuff - we need to have it loaded, but I guess I'm not fully sure why it runs and downloads everything, when it's not targeted. I'm trying to reduce cycle time when dev'ing in Pants itself - and in external repos, we don't grab tooling unless it's necessary for the goal we're doing. But, not in the main repo?
p
It is needed if pants needs, for some reason, to enumerate all targets. That means running target generators and dependency inference.
I believe there's an issue about allowing people to skip an entire language to avoid that language's tooling. It's just an issue though, no one has done a PR afaik.
w
Ah yes, I remember the conversation around it a couple years ago - not that I could necessarily grok it back then 😆 So, the 20 java deps are just related to dep inference? If so, that's wild. There was a monthly meeting where we discussed this a while ago, and lazy loading backends (even by configuration settings) came up. After call-by-name, this feels like my next big ticket item. Whatever causes me the most pain on a day-by-day basis 😆
Iteration on Pants takes me (on my ridiculously fast mac) like 30-60s per cycle, which is why I dev plugins outside of the repo. Just too slow to iterate
p
True. It's very annoying.
Most of what I do is modify the python backend, so an external plugin is not really a viable option for me 😕
w
Call-by-name is migrating each backend one at a time - most of my time is waiting for pants to start
In all fairness, I haven't yet looked into any performance enhancements, so I shouldn't be complaining
p
And because we're modifying pants itself, almost any change requires restarting pantsd.
💯 1
w
My hope is that call-by-name might speed things up, as rule-graph gen does less work. That could be a nice win, once I get these extraneous 
implicitly
calls trimmed
Java... It's always Java! [Shakes fist at clouds] Note: 
git diff
incoming, in case anyone wants to apply these in dev Before: Fix typo in function comments 
pants fmt src/python/pants/goal/migrate_call_by_name*  0.58s user 0.06s system 2% cpu 30.932 total
31 seconds every time I touch my code - it's also 31 seconds for a full scheduler reload (e.g. change bootstrap settings) After: Fix typo in function comments 
pants fmt src/python/pants/goal/migrate_call_by_name*  0.60s user 0.06s system 5% cpu 12.149 total
Full scheduler refresh: 11 seconds After after: Fix typo 
pants fmt src/python/pants/goal/migrate_call_by_name*  0.59s user 0.06s system 7% cpu 8.605 total
All numbers tested only via 
pants fmt src/python/pants/goal/migrate_call_by_name*
: mainline: 31s no-jvm-toml: 11s no-bsp: 8s aggressive: 6s It's not entirely stable, because, for example - some goals reach into 
./3rdparty/jvm/com/google/guava/BUILD:
for example - so testing doesn;t work yet. But, 3-5x improvement on dev iteration speed basically just by removing backends from pants.toml 🤯 🤯 🤯 I wonder if we can remove the extraneous backends and maybe boot some of the other stuff to 
pants.ci.toml
instead of 
pants.toml
no-jvm-toml.diffaggressive.diffno-bsp.diff
And I think this is good enough for me to get back to work - I'm able to run 
lint
and 
test
which were broken in the previous versions. About 1 second of the 6-7s is spent looking for a python interpreter, which is weird - as I thought that should stay cached across builds 🤷
aggressive-working-test.diff
6 Views
Open in Slack
PreviousNext
Powered by