pantsbuild #general

Heard back from JavaOne organizer: Me; Hello Georges, A lot has happened this year around build tools for large repos following the Google Blaze model. First of all, Buck and Pants have both had solid OSS development activity as evidenced by github stats. Also, Google open sourced Blaze through the Bazel project. For a talk about Pants, I circulated the idea with the other OSS contributors to Pants. We discussed doing a talk about how Pants is being used to edit, compile, test, and deploy Java, Scala, and Python at scale at Twitter, Foursquare and Square. Ideally there would be time to do an overview of the tool (which is inspired by Blaze from Google) and then talk about the kinds of issues working with a repo with hundreds or thousands of projects (sometimes called a "monorepo"). If time is limited, the part we were most interested in was the "case studies from several companies."

ancient-stone-50795

07/31/2015, 11:54 AM

Forwarded to Sharat Chandler: ***Apologies for the long response*** Folks, A session on Pants is pretty interesting in my opinion and maps to what I'd expect a JavaOne attendee would want to learn more about. This falls directly in the category of "Innovation in technology using Java". I'm happy to make room for a session (heck, even a companion birds-of-a-feather [BoF] where more of an informal circular dialog with attendees can take place based on the technical scope of the delivered session). If a session (and/or a BoF) is interesting to you, I will need the following to expedite things along: 1. Full name, e-mail address and mobile number for each speaker (up to two speakers --- if more than two speakers are needed please let me know) 2. Session Title (75 characters maximum including spaces) 3. Public facing description that would appear in the JavaOne content catalog (750 characters maximum including spaces) If I can have this by Friday, I can evaluate and assess to make sure everything is in good shape. If so, I'll get this into our JavaOne content system. Just to set expectations, once I enter all the information into the JavaOne content system, Oracle's Events Team will then take over and do a content sweep of the database next Tuesday and send Acceptance Invitations to all listed speakers. If more time is needed on your part to revert information, I'm okay with that (the content sweep is done every Tuesday). Hopefully, this is enough context for you to take next steps but if you'd like to speak this Friday I'd be happy to make time (available between 10am and 11:30am PDT ---- Next week I'm traveling to the UK for business and will have limited access but still happy to adjust my calendar to speak if desired). Warmest Regards, --Sharat

ancient-stone-50795

07/31/2015, 11:58 AM

Here’s a doc where we can work: https://docs.google.com/document/d/1cJ0E12sUzUWy6mufBeT5KxbhD8d_wskOXUVNFDHqhYM/edit?usp=sharing

ancient-stone-50795

07/31/2015, 11:59 AM

PantsJavaOne.txt

mammoth-autumn-2551

07/31/2015, 12:08 PM

Anyone know if its expected to create 1 nailgun per binary target listed in a

pants binary <java target1> <java target2>

invocation?

ancient-stone-50795

07/31/2015, 1:01 PM

That sounds wrong to me.

mammoth-autumn-2551

07/31/2015, 1:08 PM

hmmm, I can reproduce this on pants 0.0.39, and my exact command is

./pants bundle <target1> <target2> … <targetN> -x --compile-java-partition-size-hint=0 --bundle-deployjar

mammoth-autumn-2551

07/31/2015, 1:08 PM

and it will spawn N nailguns

enough-analyst-54434

07/31/2015, 1:19 PM

Definitely a bug

enough-analyst-54434

07/31/2015, 1:21 PM

BBOL in ~30 minutes, but I can take a look then.

mammoth-autumn-2551

07/31/2015, 1:27 PM

let me know if this is something that java compile configuration can affect… I think ours is pretty vanilla but I’m happy to share

ancient-stone-50795

07/31/2015, 1:35 PM

@mammoth-autumn-2551: I’m trying to reproduce it right now in our environment.

ancient-stone-50795

07/31/2015, 1:36 PM

We are using version 0.0.35-square-20150714-01

ancient-stone-50795

07/31/2015, 1:37 PM

Untitled.txt

ancient-stone-50795

07/31/2015, 1:40 PM

I did a

./pants binary

and saw 3 nailguns left at the end. Now, let me try a development version of pants and see if the results are any different

ancient-stone-50795

07/31/2015, 1:51 PM

-.txt

ancient-stone-50795

07/31/2015, 1:53 PM

At the end of the compile I see 1 instance each of BinaryCreate, AptCompile, JavaCompile, IvyResolve, IvyImports, JaxbGen

ancient-stone-50795

07/31/2015, 1:56 PM

@mammoth-autumn-2551: Can you post the ps output to show the list of nailgun processes you see running?

ancient-stone-50795

07/31/2015, 1:57 PM

I tried again with

./pants bundle …

and I see an additional BundleCreate nailgun, but not one nailgun per target.

ancient-stone-50795

07/31/2015, 2:02 PM

I don’t see any debug output saying that a nailgun is being created and destroyed either (when I re-run the command).

enough-analyst-54434

07/31/2015, 2:05 PM

@ the release_0.0.39 tag for $ ./pants bundle examples/src/java/org/pantsbuild/example/hello/main testprojects/src/java/org/pantsbuild/testproject/bundle I get 1 nailgun per vertical, 5 total

enough-analyst-54434

07/31/2015, 2:05 PM

$ ps -eocmd | grep -v grep | grep nailgun | sed -E "s|^.*-Dpants.nailgun.owner=([^ ]+) .*$|\1|" /home/jsirois/dev/3rdparty/pants/.pants.d/ng/IvyResolve_resolve_ivy /home/jsirois/dev/3rdparty/pants/.pants.d/ng/JavaCompile /home/jsirois/dev/3rdparty/pants/.pants.d/ng/Checkstyle_compile_checkstyle /home/jsirois/dev/3rdparty/pants/.pants.d/ng/BinaryCreate_binary /home/jsirois/dev/3rdparty/pants/.pants.d/ng/BundleCreate_bundle

enough-analyst-54434

07/31/2015, 2:06 PM

@mammoth-autumn-2551: If you can supply those ps details @ancient-stone-50795 mentioned in an issue - that would be great.

enough-analyst-54434

07/31/2015, 2:07 PM

erm, I did not do apples-apples, you did 2 out of the ordinary things with flags there ... collecting my apples

ancient-stone-50795

07/31/2015, 2:08 PM

(I tried adding those flags in my repo, it didn’t change the outcome)

mammoth-autumn-2551

07/31/2015, 2:09 PM

@ancient-stone-50795: @enough-analyst-54434 sure one second

enough-analyst-54434

07/31/2015, 2:10 PM

k - and ditto for me, flags do not change outcome

mammoth-autumn-2551

07/31/2015, 2:13 PM

Copy code

./pants ng-killall
ps -eocmd | grep -v grep | grep nailgun
./pants bundle src/java/com/urbancompass/search:saved_searches_tail_searcher src/java/com/urbancompass/search:search_server -x --compile-java-partition-size-hint=0 --bundle-deployjar
ps -eocmd | grep -v grep | grep nailgun | sed -E "s|^.*-Dpants.nailgun.owner=([^ ]+) .*$|\1|"