pantsbuild #general

Channels

welcome

github-notifications

announce

witty-crayon-22786

07/10/2017, 8:20 PM

thanks

straight-waiter-86090

07/11/2017, 2:53 PM

Hi - I’m looking into some cases where the fingerprint of targets changes depending on what target is compiled. This causes the cache to be less effective in some cases. I don’t have enough information yet to file an issue, but maybe someone knows the cause.

straight-waiter-86090

07/11/2017, 2:54 PM

one example. I have a

nucleus/java:serialization

target that compiles Java code. It’s referenced indirectly by two other targets:

nucleus:world

and

nucleus/java:all

. Both list many other targets as well.

straight-waiter-86090

07/11/2017, 2:55 PM

Depending on whether I compile

nucleus:world

nucleus/java:all

, the fingerprint of

nucleus/java:serialization

changes.

straight-waiter-86090

07/11/2017, 2:58 PM

Copy code

$ ./pants compile nucleus:world
$ ls -ld .pants.d/compile/zinc/current/nucleus.java.serialization/current
lrwxr-xr-x  1 dyoung  SKYDIVE\Domain Users  12 Jul 11 10:58 .pants.d/compile/zinc/current/nucleus.java.serialization/current -> 274e31b840ec

$ ./pants compile nucleus/java:all
$ ls -ld .pants.d/compile/zinc/current/nucleus.java.serialization/current
lrwxr-xr-x  1 dyoung  SKYDIVE\Domain Users  12 Jul 11 10:56 .pants.d/compile/zinc/current/nucleus.java.serialization/current -> b79c0d8ab75b

straight-waiter-86090

07/11/2017, 3:00 PM

this doesn’t happen with all targets, only some.

rough-dog-96241

07/11/2017, 3:30 PM

Is python3 a thing in pants? I've been googling and can't find many leads.

straight-waiter-86090

07/11/2017, 3:57 PM

I think I tracked down my problem before. it happened because I had a dependency on reflectasm, intransitive=False. This referred to one version of asm. If I compiled one of the targets, it additionally brought in a later version of asm. ivy evicted the earlier version and replaced with the later version in this case. This (correctly) make the fingerprint different.

famous-salesclerk-49406

07/11/2017, 4:02 PM

@rough-dog-96241 our code base has both python 2 and python 3 code, and we're using the

compatibility

field to indicate for each target what the target language is. Works well.

rough-dog-96241

07/11/2017, 4:06 PM

ah, cool. So I should write compatibility=['python3'] on a python binary, and the right thing happens?

rough-dog-96241

07/11/2017, 4:06 PM

@famous-salesclerk-49406

famous-salesclerk-49406

07/11/2017, 4:18 PM

For us, we're writing things like

compatibility=['CPython==3.5.1]

compatibility=['CPython==2.7.6', 'CPython==3.5.1']

famous-salesclerk-49406

07/11/2017, 4:19 PM

All environments have exactly those versions of interpreters available (inside a docker container) so we get very specific with the version number, but the usual

>=

stuff would work.

famous-salesclerk-49406

07/11/2017, 4:19 PM

To be more precise, I wrote a little plugin that exposes to the build files the symbols

PY35

and

PY27

that map to the above strings to save some typing.

rough-dog-96241

07/11/2017, 4:46 PM

@famous-salesclerk-49406 Does that mean all those interpreters have to be available at build time?

rich-helicopter-79006

07/11/2017, 7:02 PM

I’m curious how

test-changed

actually works. Does it look at uncommitted files in the local repo and only run tests associated with them?

rich-helicopter-79006

07/11/2017, 7:02 PM

I tried using

changes-since=<some_git_hash>

as well, but didn’t see it execute any tests either

witty-crayon-22786

07/11/2017, 7:06 PM

the rest of the options are important

witty-crayon-22786

07/11/2017, 7:06 PM

in particular: one of them is "include-dependees"

witty-crayon-22786

07/11/2017, 7:07 PM

if you don't include dependees, you'd have to have directly have changed a test in order for it to run.

rich-helicopter-79006

07/11/2017, 7:19 PM

I see. Let me try that. TY

famous-salesclerk-49406

07/11/2017, 8:10 PM

@rough-dog-96241 At build/run/test time there must be at least one available interpreter that is compatible with the target(s) you are operating on. If there's more than one the lowest version is picked up.

rough-minister-58256

07/12/2017, 12:44 AM

anyone around for a quick shipit on https://github.com/pantsbuild/pants/pull/4738

witty-crayon-22786

07/12/2017, 12:53 AM

#shipit

rough-minister-58256

07/12/2017, 12:56 AM

thank u sir

rough-dog-96241

07/12/2017, 1:07 AM

@rough-minister-58256 Last time I bug you about platforms and PEX I promise. 🙂 How does a PEX that's been compiled for multiple platforms decide which to use? Can I tell it with an envvar or some such? (Background: I'm trying to add a gevent-carrying pex to a google-supplied alpine linux docker container. Our linux_x86-64 wheels are built on centos for all out other containers. Alpine, because busybox is so awesome, has weird libc, and gevent can't be moved between them. If I build the wheel in Alpine, the filename comes out the same. So I'm considering various different ways to get a different gevent wheel for alpine and for centos. Is this a crazy idea? (Perhaps a different PEX would be better..)) (whoops, doubly nested parens..)

rough-minister-58256

07/12/2017, 1:12 AM

the pex bootstrap logic does a minimal runtime resolve to determine the subset of packages to use when the pex is run

rough-minister-58256

07/12/2017, 1:13 AM

iirc, it looks at e.g.

Copy code

>>> pkg_resources.get_supported_platform()
'macosx-10.12-x86_64'

rough-minister-58256

07/12/2017, 1:13 AM

to determine the current platform and then resolves for that. don’t believe an env var is plumbed to override it

👍 1

rough-minister-58256

07/12/2017, 1:14 AM

this would be analogous to e.g. the platform inference that happens when you do a

pip install

locally, but at runtime in an environment that may contain supporting dists for multiple platforms