@hundreds-father-404 We just noticed https://github.com/pantsbuild/pants/issues/9346 - any guesses at a root cause before we dig in?

ohhhh great find! cc @happy-kitchen-89482 @polite-garden-50641 @hundreds-breakfast-49010 this could be one of the culprits for why the cache is being invalidated more than we’d expect So, looking at the constructor for

PathGlobs

in

engine/fs.py

, the solution should be really simple. Change line 84 from

python
 self.globs = tuple(globs)

to

python
 self.globs = tuple(sorted(globs))

Globs are an

Iterable[str]

, so it’s always safe to sort (or we got bigger problems) and sorting strings is cheaper than bad cache misses

yeah that's a super-quick fix

Specifically, the ordering won’t be stable if the passed

globs

were not deterministic, e.g. if you passed a set to the constructor Even if you passed a deterministic data structure like a tuple, we will still get more cache misses than we’d like. target1 with

['f1.py', 'f2.py']

sources and target2 with

['f2.py', 'f1'.py']

would be treated as distinct for no good reason

you wanna write that up and then I can review it @hundreds-father-404?

I suspect it’s only one of many culprits. But, one down gets us closer and it’s extremely low hanging fruit 🎉

Wow, that was fast - thanks @hundreds-father-404!!

good find, thanks folks.

Great stuff

added one comment on https://github.com/pantsbuild/pants/pull/9347 ... we have a test for this, so would be good to get a repro from @average-vr-56795

The repro was literally:

./pants --native-engine-visualize-to=/some/path list 3rdparty:

in the pants repo

yea, Eric followed up on the ticket: i understand now

It was the ordering of the include-elements in the

PathGlobs

inputs vs outputs

Cool 🙂

thanks!