clean-city-64472
07/15/2021, 4:04 PMnamed_caches
in CI? We added it to the cache but on BitBucket, the cache is only written the first time it is missed. That means if it's written on a CI run that doesn't touch most of the repo it will be fairly incomplete. We used to be on Gitlab where the cache could be updated after each run so it would continuously expand - but then we had the problem that it would eventually grow too big and the cost of downloading it exceeded the benefit. Is anyone implementing their own cache system using something like S3?witty-crayon-22786
07/15/2021, 4:17 PMWe added it to the cache but on BitBucket, the cache is only written the first time it is missed.are you able to choose the cache key? if so, would suggest choosing it based on a hash of any requirements files you have
clean-city-64472
07/15/2021, 4:18 PMwitty-crayon-22786
07/15/2021, 4:18 PMclean-city-64472
07/15/2021, 4:20 PMwitty-crayon-22786
07/15/2021, 4:20 PMclean-city-64472
07/15/2021, 4:21 PMwitty-crayon-22786
07/15/2021, 4:21 PMnamed_caches
are only relevant for Python resolves when you miss Pants’ process cache (stored locally in ~/.cache/pants/lmdb_store
or remotely).
you should only miss the process cache if you’ve changed the input requirements, or if they have been garbage collected / aren’t large enough.
if you are comfortable re-running resolves from scratch when you change your requirements, using/saving only the process cache in CI environments might work well for you. there is upcoming work to make the size of the process cache much easier to manage.happy-kitchen-89482
07/15/2021, 8:01 PMpowerful-boots-1234
07/15/2021, 8:07 PMbut we are spending 7minutes "resolving contraints.txt" (edited)is resolution here using pip? The most recent versions of pip take far long to resolve deps. https://github.com/pypa/pip/issues/9187
clean-city-64472
07/15/2021, 8:13 PMwitty-crayon-22786
07/15/2021, 9:17 PM--pex-verbosity=1
(https://www.pantsbuild.org/docs/reference-pex#section-verbosity). but i think that in @clean-city-64472’s case the resolve is large enough that it isn’t worth digging into