https://pantsbuild.org/ logo
#development
Title
# development
h

hundreds-breakfast-49010

10/10/2019, 12:20 AM
I've noticed what I think is a bug in my recent
pants binary
work
if I run
./pants binary <some target>
(i.e. running the v1 version of it)
and then immediately thereafter run
./pants binary --v2 --no-v1 <same target>
the resulting pex written to
dist
seems to be corrupt
a

aloof-angle-91616

10/10/2019, 12:21 AM
nice
h

hundreds-breakfast-49010

10/10/2019, 12:21 AM
but I can fix the problem by deleting that pex and re-running the v2 command
a

aloof-angle-91616

10/10/2019, 12:22 AM
yeah ok
h

hundreds-breakfast-49010

10/10/2019, 12:22 AM
curious if anyone else is seeing this
a

aloof-angle-91616

10/10/2019, 12:22 AM
i would just make sure to delete any expected output file?
h

hundreds-breakfast-49010

10/10/2019, 12:22 AM
yeah that might be the cleanest way to fix this
a

aloof-angle-91616

10/10/2019, 12:22 AM
i don't think we need to try to merge pex files
not sure what that bug is though it wouldn't be impossible to fix
just maybe not now idk
h

hundreds-breakfast-49010

10/10/2019, 12:22 AM
so this is coming from the underlying rust
materialize_directories
probably doing something wrong
a

aloof-angle-91616

10/10/2019, 12:23 AM
hm
h

hundreds-breakfast-49010

10/10/2019, 12:23 AM
like trying ot merge the newly-created pex with the pex already on the filesystem?
a

aloof-angle-91616

10/10/2019, 12:23 AM
that might be more important then
well, that sometimes can happen when you use zip libraries
and you open a file
and don't specify truncating the existing file
a lot of code isn't written to handle that case and things can break
how corrupt was the pex? was it a valid zip?
h

hundreds-breakfast-49010

10/10/2019, 12:24 AM
Untitled
it did seem to be a valid zip
a

aloof-angle-91616

10/10/2019, 12:24 AM
could you try running with
--no-compile
or is it already doing that
h

hundreds-breakfast-49010

10/10/2019, 12:25 AM
like, pass that flag to the pex?
a

aloof-angle-91616

10/10/2019, 12:25 AM
i feel like i ran into this a week ago with another pex thing i was doing
yes
h

hundreds-breakfast-49010

10/10/2019, 12:25 AM
when I run it manually?
a

aloof-angle-91616

10/10/2019, 12:25 AM
well
no
when it's created
h

hundreds-breakfast-49010

10/10/2019, 12:25 AM
hm what does that flag do?
a

aloof-angle-91616

10/10/2019, 12:25 AM
it avoids compiling pyc files
if you look into that entry in the pex
Copy code
> unzip -p /home/gregs/code/pants/dist/main.pex .bootstrap/pex/__init__.py
what does that print
because i've had pex write pyc files to that file before i think
i don't have the repro on hand, but i could probably find it
h

hundreds-breakfast-49010

10/10/2019, 12:27 AM
Copy code
gregs@gregs-dev-laptop ~/c/p/dist> unzip -p /home/gregs/code/pants/dist/main.pex .bootstrap/pex/__init__.py
.bootstrap/pex/__init__.py:  mismatching "local" filename (.bootstrap/_pex/__init__.pyc),
         continuing with "central" filename version
3
n]@sdS)Nrrrootstrap/_pex/__init__.p<module>s⏎
a

aloof-angle-91616

10/10/2019, 12:27 AM
and
--no-compile
fixed that
there we go
i have no clue what "local" and "central" filenames are
this might be avoidable if we add
--no-compile
h

hundreds-breakfast-49010

10/10/2019, 12:27 AM
hm, ok
a

aloof-angle-91616

10/10/2019, 12:28 AM
but i believe this looks maybe like pex doing something wrong
h

hundreds-breakfast-49010

10/10/2019, 12:28 AM
I wonder if it would be better to just explicitly delete any files with the same name in dist, before overwriting them
a

aloof-angle-91616

10/10/2019, 12:28 AM
but also if we just make sure to delete output files beforehand, that would work not just for this issue
yeah
h

hundreds-breakfast-49010

10/10/2019, 12:28 AM
I dunno if
materialize_directories
lets us do that already
a

aloof-angle-91616

10/10/2019, 12:28 AM
like we can avoid this with any program that interacts with files this is a good long-term choice i think
h

hundreds-breakfast-49010

10/10/2019, 12:28 AM
maybe there's an overwrite flag?
a

aloof-angle-91616

10/10/2019, 12:29 AM
maybe we should always do it whenever we materialize?
it feels like an optimization to not overwrite the file that we may not need write now
right now
h

hundreds-breakfast-49010

10/10/2019, 12:33 AM
hm, is there a way to do this iwth the APIs in the engine already?
in
store/src/lib.rs
, we have
materialize_file
that actually does the disk-writing
but I'm not sure what hte behavior of the flag is if the file already exists on-disk
a

aloof-angle-91616

10/10/2019, 12:36 AM
oh
yeah
also
well
i might try just building the pex with
--no-compile
in pants and see if that fixes the issue
h

hundreds-breakfast-49010

10/10/2019, 12:40 AM
rm'ing the file and building it again (in v2) fixes the issue, I'd rather make the semantics of file materialization be correct
a

aloof-angle-91616

10/10/2019, 12:41 AM
i think the issue might be separate
because the only reason i can think of for the pex to be corrupt is that the file previously existed in dist, and was somehow snapshotted and used as the input to the pex creation
and where we materialize the file doesn't seem like it would affect that
since
--no-compile
fixes the issue, we know it's not because file materialization itself is corrupting the file
h

hundreds-breakfast-49010

10/10/2019, 12:42 AM
hm, that definitely shouldn't be happening
the pex should be getting created in a tmpdir
a

aloof-angle-91616

10/10/2019, 12:43 AM
yes, and if the one in dist was snapshotted and used as the input, then it might also be in that tmpdir
h

hundreds-breakfast-49010

10/10/2019, 12:43 AM
we shouldn't be snapshotting things in
dist
, right?
unless this is a cas bug?
a

aloof-angle-91616

10/10/2019, 12:44 AM
i don't think we should be
i also cannot explain why pex is acting as if the output file is already there
^if that hypothesis is correct
h

hundreds-breakfast-49010

10/10/2019, 12:44 AM
adding that flag fixes the issue
at pex creation time
a

aloof-angle-91616

10/10/2019, 12:45 AM
if we use
--no-process-execution-cleanup-local-dirs
we can look at the output pex, but that doesn't let us see the input
if we run with
-ldebug
we might get the process execution request printed out?
and then we could view the contents of the
input_files
digest with
fs_util
h

hundreds-breakfast-49010

10/10/2019, 1:19 AM
one thing I was doing when I was oriiginally debugging this
was taking the
Digest
we pass to ExecuteProcessRequest
and requesting a
Snapshot
from that and printing the snapshot
which gives:
Copy code
SNAP: Snapshot(directory_digest<Exactly(Digest)>=Digest(fingerprint=d8f1dacca264009f32c6fb5171e9c7f561dd2d2e6fde1c80593848a66872c9db, serialized_bytes_length=168), files<Exactly(tuple)>=('pex', 'source_files/3rdparty/python/requirements.txt', 'source_files/example/__init__.py', 'source_files/example/hello/__init__.py', 'source_files/example/hello/greet/__init__.py', 'source_files/example/hello/greet/greet.py', 'source_files/example/hello/main/__init__.py', 'source_files/example/hello/main/main.py'), dirs<Exactly(tuple)>=('source_files', 'source_files/3rdparty', 'source_files/3rdparty/python', 'source_files/example', 'source_files/example/hello', 'source_files/example/hello/greet', 'source_files/example/hello/main'))
it also seems like the
no-process-execution-cleanup-local-dirs
flag isn't doing anything
and I'm not sure why
a

aloof-angle-91616

10/10/2019, 5:56 AM
check the output! it should print out a directory
it's in
/tmp
now
it's not obvious i had to figure it out