https://pantsbuild.org/ logo
Join Slack
Powered by
<@U01PZK60W2F>: Thank you for `generate-lockfiles ...
# general
p
@curved-television-6568: Thank you for 
generate-lockfiles --diff
! (I added the option in pants.toml) This is so nice!
❤️ 4
h
woah! Which release is this available in?
p
2.16.0a0
h
It's super great
I think we should turn it on by default
p
I agree, though it does seem to make lockfile generation take significantly longer (but totally worth it).
It takes ~4 min to lock in the st2 repo. Adding diff adds another minute or two. (or at least it normally goes to around 260 seconds, and I saw it go over 320 or so when generating the diff).
e
Hrm. That's too bad. The 
pex3 lock update ...
which Pants does not use, has supported printing this - less prettily - since day 1. It adds ~no extra overhead.
p
We really need to iron down the UX for dependecy management in pants beyond just 
generate-lockfiles
c
Nice. But I’m surprised it adds so much overhead…. did you try locking the same requirements both with and without diff?
p
I can investigate further...
🙏 1
c
@enough-analyst-54434 would’ve been nice if pants made better use of the pex features. Another benefit of this approach though will be that it is agnostic to the lock tool being used, so we can get the same information also for go and java etc..
p
Yeah. I way over estimated: with diff: 
./pants generate-lockfiles --resolve=st2 1.78s user 0.11s system 0% cpu 3:44.55 total
without: 
./pants generate-lockfiles --resolve=st2 1.41s user 0.09s system 0% cpu 3:16.25 total
Just goes to show that I really have no concept of time and I blink at the wrong times.
And that's only one run, so 🤷
c
thanks for checking. I’ll keep an eye on this too anyhow
h
Should this be re-implemented via pex? (we want 
lock update
anyway for incremental lockfile generation - officially now the community's highest priority feature request)
c
if done in pex we need to redo this for go/js/java/etc too… but, we may have some way to communicate the diff data if there turns out to be a perf issue..
where as now what is missing is “only” the diff information that is missing for those other eco systems..
(we want 
lock update
anyway for incremental lockfile generation..
this should not affect the diff logic at all
h
Oh right, multiple languages... never mind then
but yeah, possibly to communicate the diff data
or, we should profile that code to see if we can speed it up, seems like it should be possible
I can't offhand think why this would be computationally intensive, and it's not doing any network requests?
c
exactly, I would expect it to be rather performant as it merely parses the lockfile json and traverses it, basically. There are a bunch of set operations over it, but still would expect it to be reasonable..
p
This lockfile is 259K in size. most of my other lockfiles are in the 10s of K in size.
h
So there's probably some accidentally egregious algorithm in there that could be discovered via profiling
Open in Slack
PreviousNext
Powered by