Ergonomic, hermetic, user-friendly software build system for Python, Go, Java, Kotlin, Protobuf, Scala, Shell. Pants lets you fearlessly scale up your codebase!

Pants

Pasted image at 2017_12_08 04_01 PM.png

<@U054MQUPL> uploaded a file: <https://pantsbuild.slack.com/files/U054MQUPL/F8D0H8UCE/pasted_image_at_2017_12_08_04_01_pm.png|Pasted image at 2017-12-08, 4:01 PM>

could try readding that, since self._context_lock still exists

<@U054MQUPL>: mm, yea... maybe. the stack is ```File "/opt/ee/python/2.7/lib/python2.7/threading.py", line 174 in acquire
  File "/Users/stuhood/.pex/install/pantsbuild.pants-1.4.0.dev16+899381073-cp27-none-macosx_10_8_x86_64.whl.46445b2c7acbf09afd084ac31d3da3e261ae5636/pantsbuild.pants-1.4.0.dev16+899381073-cp27-none-macosx_10_8_x86_64.whl/pants/bin/engine_initializer.py", line 96 in create_build_graph``` before <https://github.com/pantsbuild/pants/pull/5178> and inside the engine after it

but yea, you're totally right about this.

so... does the context lock _prevent_ the invalidation thread from successfully getting into the engine?

prob need to plumb All The Locks to context_lock, so we hold them all while forking

ie, no need to acquire the graph lock if there is some other lock above it preventing access during a fork

hopefully it would be a read/write lock... so we could do something like acquire "read" while executing things, but acquire "write"/exclusive while forking ...?

i guess since we already don't really allow for concurrent access at this point, one big lock would be ok for now

yeah. only concurrent would be at most 1 pailgun run + an invalidation event

but the surface area of concurrency needed in the daemon would be .. mostly writes

since thats just the scheduler/product graph warming stage

oh... i didn't realize the singlethreaded pailgun bit

...is that single thread the main thread, by any chance...?

ah, but that doesn't matter, because responder threads are all in the parent process with pipes to the child

<@U054MQUPL>: so... i'm thinking to go ahead and merge <https://github.com/pantsbuild/pants/pull/5178> . it doesn't fix <https://github.com/pantsbuild/pants/issues/5169> , but it doesn't make it any worse, and it actually cleans up the locking situation.

then i can take a deeper look at the context lock.