Hello again I am trying to convert my python app targets to

Question

Hello again I am trying to convert my python app targets to archive targets and im running into this error ```15 08 58 38 ERROR Exception caught pants engine internals scheduler ExecutionError File Users briandesimone cache pants setup bootstrap Darwin x86 64 2 1 1 py36 lib python3 6 site packages pants bin local pants runner py line 255 in run engine result = self run v2 goals File Users briandesimone cache pants setup bootstrap Darwin x86 64 2 1 1 py36 lib python3 6 site packages pants bin local pants runner py line 182 in run v2 return self maybe run v2 body goals poll=False File Users briandesimone cache pants setup bootstrap Darwin x86 64 2 1 1 py36 lib python3 6 site packages pants bin local pants runner py line 204 in maybe run v2 body poll delay= 0 1 if poll else None File Users briandesimone cache pants setup bootstrap Darwin x86 64 2 1 1 py36 lib python3 6 site packages pants init engine initializer py line 126 in run goal rules goal product params poll=poll poll delay=poll delay File Users briandesimone cache pants setup bootstrap Darwin x86 64 2 1 1 py36 lib python3 6 site packages pants engine internals scheduler py line 569 in run goal rule self raise on error t for t in throws File Users briandesimone cache pants setup bootstrap Darwin x86 64 2 1 1 py36 lib python3 6 site packages pants engine internals scheduler py line 539 in raise on error wrapped exceptions=tuple t exc for t in throws Exception message 1 Exception encountered ProcessExecutionFailure Process Create db data cache db lexio migrations data cache tar failed with exit code 1 stdout stderr Usage List tar tf lt archive filename gt Extract tar xf lt archive filename gt Create tar cf lt archive filename gt filenames Help tar help``` This doesn t give me much to go on here and even with debug mode on this is the extent of the error message Here is my BUILD file ```python library name= lib dependencies= 3rdparty python alembic 3rdparty python SQLAlchemy lib python core src sources= py tags= code db migration python pex binary name= alembic entry point= alembic config dependencies= lib zip safe=False archive name= lexio migrations data cache packages= alembic files= files format= tar tags= code db migration python output path= db data cache db lexio migrations data cache tar files name= files sources= alembic ini env py versions py ``` Any help here would be appreciated

hundreds-father-404 · Answer

Hm I suspect that we are invoking `tar` in an illegal way on your machine What is your environment and what does `tar version` say

salmon-barista-63163 · Answer

```bsdtar 2 8 3 libarchive 2 8 3``` im in a pyenv running python 3 6 8

hundreds-father-404 · Answer

K the command we run is ` path to tar caf result tar f1 ext f2 ext ` I suspect tar 2 8 doesn t like that for some reason Could you please try running something like the above directly on the CLI

salmon-barista-63163 · Answer

yeah one second

witty-crayon-22786 · Answer

in ` ldebug` there should be a bit more information about the process that is launched it should have the complete argument list

witty-crayon-22786 · Answer

will look like ```12 06 19 39 DEBUG spawned local process as 17884 for Process argv Users stuhood pyenv versions 3 7 7 bin python3 7 pytest pex no header k test cache scope successful src python pants engine process test py env LANG en US UTF 8 LC CTYPE en US UTF 8 PATH Users stuhood src pants build support virtualenvs Darwin pants dev deps py37 venv bin Users stuhood src google cloud sdk bin Users stuhood cargo bin Users stuhood pyenv shims Users stuhood pyenv bin Users stuhood bin usr local bin usr bin bin usr sbin sbin PEX EXTRA SYS PATH src python PEX IGNORE RCFILES true PEX INHERIT PATH false PEX ROOT Users stuhood cache pants named caches pex root PYTEST ADDOPTS color=yes working directory None input files Digest Fingerprint lt 7f3ffe3931443d29371d1517757f49d31bcca776d58b14e3a472e84a0b3abc59 gt 431 output files output directories timeout Some 90s execution slot variable Non```

witty-crayon-22786 · Answer

spawned local process as

salmon-barista-63163 · Answer

yeah i do have that from the debug logs I think its a permissions issue on my dist dir let me check something

salmon-barista-63163 · Answer

and thats a no thought it was because when i ran that above command i had some permission denied errors when trying to use a tar

hundreds-father-404 · Answer

That `tar` process will run in a tmpdir and write the file there Then Pants will capture the result and Pants will write to `dist ` This error is happening when invoking `tar` so permissions on the `dist ` folder is almost certainly a red herring as we haven t hit that step yet

salmon-barista-63163 · Answer

ah got it fair point

salmon-barista-63163 · Answer

```cannot execute binary file```

salmon-barista-63163 · Answer

here is the stderr from trying to execute above command

salmon-barista-63163 · Answer

i just made a tar and then ran what you had above

salmon-barista-63163 · Answer

here is my debug logs if this helps ```spawned local process as 82972 for Process argv usr bin tar caf db data cache db lexio migrations data cache tar db data cache db alembic ini db data cache db env py db data cache db versions 006652de6637 add day of fiscal quarter function py db data cache db versions 0937f76df8bd add topic feeds table py db data cache db versions 25a71be92e10 add fiscal year function py db data cache db versions 2f6e248acf13 update card title and card category to py db data cache db versions 355b0fd2f88b feed items jsonb not null py db data cache db versions 6843323245c1 add fiscal quarter year function py db data cache db versions 6f8531f3d5b6 refactor fiscal quarter year function py db data cache db versions 7103915533d9 adding chart content to topic feed py db data cache db versions 73ff0eea1886 add day of fiscal year function py db data cache db versions 78a04f5f3c39 drop newsfeed v1 tables py db data cache db versions 85902519c1fe add metric column py db data cache db versions a105975e3483 add card category and card label py db data cache db versions c892ac9c500e add day of quarter function py db data cache db versions d0c55c9df3f5 create feed tables py db data cache db versions d81d044d33ac add story config to topic feed items py db data cache db versions fe29d0c81a77 add plpgsql extension py db data cache db alembic pex env PATH usr bin bin usr local bin working directory None input files Digest Fingerprint lt c1958cd0d6a590c150c407b4bbc9e07daf312f5f9705986f0ad885de70372182 gt 166 output files RelativePath db data cache db lexio migrations data cache tar output directories timeout None execution slot variable None description Create db data cache db lexio migrations data cache tar level Debug append only caches jdk home None target platform None is nailgunnable false cache scope Successful ```

hundreds-father-404 · Answer

Thanks what was the full output of running ` usr bin tar caf db data cache db lexio migrations data cache tar db data cache db alembic ini db data cache db env py` directly Prob fine to leave off those other files for now

salmon-barista-63163 · Answer

```Brian DeSimone MacBook Pro NS0311 talos briandesimone$ usr bin tar caf db data cache db lexio migrations data cache tar Usage List tar tf lt archive filename gt Extract tar xf lt archive filename gt Create tar cf lt archive filename gt filenames Help tar help```

hundreds-father-404 · Answer

Okay and then the same thing but running with these permutations please instead of `caf` ` caf` `cf` ` cf` Trying to confirm it s a bad argv

salmon-barista-63163 · Answer

yep i see where you are going one sec

salmon-barista-63163 · Answer

```Brian DeSimone MacBook Pro NS0311 talos briandesimone$ usr bin tar caf db data cache db lexio migrations data cache tar Usage List tar tf lt archive filename gt Extract tar xf lt archive filename gt Create tar cf lt archive filename gt filenames Help tar help Brian DeSimone MacBook Pro NS0311 talos briandesimone$ usr bin tar cf db data cache db lexio migrations data cache tar tar no files or directories specified Brian DeSimone MacBook Pro NS0311 talos briandesimone$ usr bin tar cf db data cache db lexio migrations data cache tar tar no files or directories specified```

hundreds-father-404 · Answer

excellent so the `a` flag must not exist in tar 2 8 That s an easy fix and we can cherry pick it

hundreds-father-404 · Answer

Ahh we do need to recreate the functionality somehow ``` a auto compress c mode only Use the archive suffix to decide a set of the format and the compressions As a simple example tar a cf archive tgz source c source h creates a new archive with restricted pax format and gzip compression tar a cf archive tar bz2 uu source c source h creates a new archive with restricted pax format and bzip2 compression and uuencode compression tar a cf archive zip source c source h creates a new archive with zip format tar a jcf archive tgz source c source h ignores the `` j option and creates a new archive with restricted pax for mat and gzip compression tar a jcf <http archive xxx|archive xxx> source c source h if it is unknown suffix or no suffix creates a new archive with restricted pax format and bzip2 compression ``` That s doable though the user tells us what they wanted to use

hundreds-father-404 · Answer

In the meantime to us getting this fix out you might want to use `zip` as an archive format

salmon-barista-63163 · Answer

i wonder if this is only an issue for me cause im testing on my local we use centos dockers to actually run this in our production pipelines

hundreds-father-404 · Answer

depends if your prod has tar 3+ or not I suspect But this is enough of a gotcha that it s worth us fixing we don t want to assume you have a particular tar version

salmon-barista-63163 · Answer

yeah exactly i agree with you here

hundreds-father-404 · Answer