wow I haven't seen this performance differences (d...
# developmentf
fresh-cat-90827
07/10/2024, 9:49 PMwow I haven't seen this performance differences (doing dependency inference) between 2.17 and 2.18 when I ran the experiments a while ago. As a teaser, my giant repo:
Copy code
$ pants list :: | wc -l
12441🎉 2
fresh-cat-90827
07/10/2024, 9:49 PM2.18 (without Rust parser)
Copy code
$ time pants --no-pantsd --no-local-cache dependencies :: > /dev/null
pants --no-pantsd --no-local-cache dependencies :: > /dev/null 476.89s user 1683.80s system 239% cpu 15:01.16 total Copy code
$ time pants --no-pantsd --no-local-cache dependencies :: > /dev/null
pants --no-pantsd --no-local-cache dependencies :: > /dev/null 71.39s user 19.78s system 145% cpu 1:02.73 totalfresh-cat-90827
07/10/2024, 9:50 PMIf not, that's pretty much awesome.
w
wide-midnight-78598
07/10/2024, 10:02 PM Copy code
Pants 2.17 introduced a new paradigm to dependency parsing for Python by leveraging a Rust-based parser that's called in the same process as Pants itself, instead of farming out to one-python-process-per-file. Copy code
As a result of the switch, cold-cache performance improved by a factor of about 12x, while hot-cache had no difference. Additionally, Pants can now infer dependencies from Python scripts with syntax errors. Copy code
After leaving this defaulted to disabled for a release cycle, Pants 2.18 started defaulting to enabling this.wide-midnight-78598
07/10/2024, 10:02 PMSeems pretty in-line with expectations :)
f
fresh-cat-90827
07/10/2024, 10:15 PMyes, I saw this in the docs, but thought this didn't apply to us since I haven't seen this boost when comparing 2.17 and 2.18 (with Rust parser enabled)
5 Views