pantsbuild #general

Channels

welcome

github-notifications

development

billowy-alarm-80641

06/24/2015, 5:46 PM

But in general i'm not surprised if multiple args are additive

witty-crayon-22786

06/24/2015, 5:47 PM

type

isn't explicitly labelled as a list

witty-crayon-22786

06/24/2015, 5:47 PM

...in filter

rough-minister-58256

06/24/2015, 5:47 PM

you have to make them that way by specifying the action when defining the arg, so it won’t work ootb for everything. store vs append

witty-crayon-22786

06/24/2015, 5:47 PM

so that could just be a filter

billowy-alarm-80641

06/24/2015, 5:48 PM

yeah, that's just for filter

witty-crayon-22786

06/24/2015, 5:48 PM

Copy code

register('--type', action='append', metavar='[+-]type1,type2,...',
             help='Filter on these target types.')

billowy-alarm-80641

06/24/2015, 5:48 PM

that's what i meant, sorry if it wasn't clear

rough-minister-58256

06/24/2015, 10:22 PM

hey @bored-art-40741 / @proud-alligator-68508, did you guys ever wind up implementing anything based on your ‘Pants Filesystem Snapshotting’ design doc?

proud-alligator-68508

06/24/2015, 10:23 PM

short answer: no.

proud-alligator-68508

06/24/2015, 10:25 PM

long answer:

proud-alligator-68508

06/24/2015, 10:26 PM

motivation was to avoid writing bad thing to cache, eg some artifact written under an incorrect hash. this could happen since hashing inputs and the actual artifact production from those happen at different times, so there’s a race window where an edit could get missed.

proud-alligator-68508

06/24/2015, 10:26 PM

snapshotting all the inputs is really complicated though

proud-alligator-68508

06/24/2015, 10:27 PM

i think instead we decided that it would be easier to listen for edits before hashing, hash, execute, and check if any edits came in before writing to cache

proud-alligator-68508

06/24/2015, 10:28 PM

the naive version might just flip a bit that skips writes entirely

proud-alligator-68508

06/24/2015, 10:28 PM

fancier might be to selectively not write invalidated targets

proud-alligator-68508

06/24/2015, 10:30 PM

simplest version would be tiny — a few lines to start up inotify or fsevents thread which just flips a boolean, and one-line check of that boolean around the cache writes. i think the hardest part is how to handle the fsevents-vs-inotify-based-on-platform and bundling the platform-specific dep

proud-alligator-68508

06/24/2015, 10:31 PM

maybe just require

watchman

be on the PATH for this to be enabled and don’t make pants worry about the impl

rough-minister-58256

06/24/2015, 10:31 PM

gotcha. to improve perf, we’re revisiting the idea of a background service that would monitor fs events and continuously parse and store BUILD file changes (and likely hashes for incremental compile invalidation). sounds like a much different goal than what you guys were aiming to solve at the time, but with a somewhat similar approach.

proud-alligator-68508

06/24/2015, 10:32 PM

ah, yeah, we’ve also explored that a bit

proud-alligator-68508

06/24/2015, 10:33 PM

something to do everything except run scala, so that we can drop straight into scalac when a user decides they’re ready to build

proud-alligator-68508

06/24/2015, 10:34 PM

as well as a git pull, pants would start background downloading artifacts from remote the cache to the local one, and maybe even decompressing so .pants.d has the state ready to go

rough-minister-58256

06/24/2015, 10:35 PM

yeah, seems like a number of optimization possibilities once a general fs-monitoring system is in place

rough-minister-58256

06/24/2015, 10:36 PM

but you guys don’t have anything like that in-place today, correct?

rough-minister-58256

06/24/2015, 10:37 PM

thinking this would be a good discussion for next weeks summit

proud-alligator-68508

06/24/2015, 10:37 PM

sorta. we have a little standalone script

./onchange

proud-alligator-68508

06/24/2015, 10:38 PM

when we were still using ivy, i tried having it run just resolve.

rough-minister-58256

06/24/2015, 10:38 PM

ah interesting

proud-alligator-68508

06/24/2015, 10:38 PM

as a proxy for my “do everything up to complile”

proud-alligator-68508

06/24/2015, 10:39 PM

it didn’t actually work that well —to actually go that route, would need some flag that says “this is background / best-effort run, but if a real run comes along, it should kill this one"