https://pantsbuild.org/ logo
b

best-florist-45041

12/16/2021, 6:25 PM
Hi, sorry to pile more on the docker train, but noticing a few issues there myself. 0) minor bug:
pants tailor
will create both
python_sources()
and
docker_image()
thus creating a name conflict if it sees a Dockerfile and python code in the same location. 1) Have
[python] interpreter_constraints = [">=3.9"]
, but somehow the pex_binary target, when placed inside the docker image, calls
python3.8
(and fails, because I'm using a 3.9 image). 2) In my situation the entrypoint is defined through a third-party helm chart which needs to know about the target python package to import. Hence I only need a python env with my code in the image. I found using
PEX_TOOLS=1 ./archive.pex venv ...
works perfectly with the corresponding
include_tools
option in
pex_binary
. Except that https://pex.readthedocs.io/en/v2.1.57/recipes.html#pex-app-in-a-container refers to env vars like
PEX_PYTHON=3.9
, which appear to not do anything (like patch the issue (1))
šŸ‘ 1
šŸ™ 2
h

hundreds-father-404

12/16/2021, 6:28 PM
Nothing to apologize for, thank you for sharing this all! Hm, what do you mean with tailor? What names are being used? It should be disambiguating for you
b

best-florist-45041

12/16/2021, 6:30 PM
Copy code
./pants tailor
10:28:59.37 [INFO] Initializing scheduler...
10:28:59.44 [INFO] Scheduler initialized.
Created src/stuff/BUILD:
  - Add docker_image target docker
  - Add python_sources target stuff
āÆ ./pants package src/stuff::
10:29:28.59 [ERROR] 1 Exception encountered:

  DuplicateNameError: A target already exists at 'src/stuff/BUILD' with name 'stuff' and target type 'docker_image'. The 'python_sources' target cannot use the same name.
this directory had two files: Dockerfile + single python file
šŸ‘€ 1
my expectation was running tailor would not put things into a broken state
BUILD file is
Copy code
docker_image()
python_sources()
h

hundreds-father-404

12/16/2021, 6:44 PM
Oh that's a silly bug I think. Thanks for reporting! Will get out a fix today and cherry-pick to 2.8
f

few-arm-93065

12/16/2021, 7:35 PM
I have seen the same bug about pex files requiring 3.8. I think because that is what pants is using under the hood. Any ideas on a workaround for that aside from having both python versions in the docker image?
h

hundreds-father-404

12/16/2021, 7:40 PM
re the pex-binary, can you try using
unzip
on it and look at the
PEX-INFO
file? That has all the metadata for the PEX
f

few-arm-93065

12/16/2021, 7:51 PM
anything in particular I should be looking for in this file?
Copy code
{
  "bootstrap_hash": "a123f8e91baf9753da4866822c19607577f7947d",
  "build_properties": {
    "pex_version": "2.1.54"
  },
  "code_hash": "8a06287e5519b727046aa317d69d062ebfc89950",
  "distributions": {
    "boto3-1.20.21-py3-none-any.whl": "39a44c866fad0505a4f36756ef8982f80b72eaf9",
     etc...
  },
  "emit_warnings": false,
  "entry_point": "...",
  "ignore_errors": false,
  "includes_tools": false,
  "inherit_path": "false",
  "interpreter_constraints": [],
  "pex_hash": "b083681a3c1646ad87398dd31bf1ae755c85fd68",
  "pex_path": null,
  "requirements": [
    "boto3==1.20.21",
     etc...
  ],
  "strip_pex_env": true,
  "venv": false,
  "venv_bin_path": "false",
  "venv_copies": false
}
h

hundreds-father-404

12/16/2021, 7:52 PM
Huh! I'm really surprised
interpreter_constraints
is set to
[]
...What Pants version is this? What happens if you set
interpreter_constraints=["==3.9.*"]
on the
pex_binary
?
b

best-florist-45041

12/16/2021, 7:57 PM
I also see
"interpreter_constraints": []
in PEX-INFO. This is
pants==2.9.0.dev3
and thus
pex==2.1.56
šŸ‘ 1
f

few-arm-93065

12/16/2021, 7:57 PM
pants version 2.8.1rc1 - on mac osx.
I am actually trying to use 3.10.0 - no change when adding interpreter_constraints=["==3.10.0"], to the pex_binary. I am trying to debug this further - adding
Copy code
ENV PEX_VERBOSE=2
to my dockerfile isn't producing any additional output, which is odd. the only error I get is
Copy code
11:58:13.62 [INFO] Completed: Building docker image [redacted]
/usr/bin/env: 'python3.8': No such file or directory
h

hundreds-father-404

12/16/2021, 8:02 PM
Oh, are you both using
platform
on the
pex_binary
perhaps?
b

best-florist-45041

12/16/2021, 8:02 PM
yes,
platforms=["linux_x86_64-cp-39-cp39"]
f

few-arm-93065

12/16/2021, 8:03 PM
Yes,
Copy code
platforms=[
    "current",
    "manylinux2014_aarch64-cp-310-cp310"
]
h

hundreds-father-404

12/16/2021, 8:04 PM
Ah ha. That explains it. Currently if
platforms
is set, we don't set
interpreter_constraints
. John has pointed out in the past that is probably incorrect behavior, which we should change In the meantime, the workaround is to set
shebang="python3.9"
on the
pex_binary
b

best-florist-45041

12/16/2021, 8:15 PM
ok cool. I had to provide the full path for this to work:
shebang="/usr/local/bin/python3.9"
or perhaps more properly,
/usr/bin/env python3.9
is what you meant?
šŸ™Œ 1
h

hundreds-father-404

12/16/2021, 8:19 PM
It can be set to whatever you'd like šŸ™‚ But noting that it's a workaround, opened https://github.com/pantsbuild/pants/issues/13904 for the proper fix
šŸ™Œ 1
f

few-arm-93065

12/16/2021, 8:21 PM
wow, that's awesome, thank you so much!
h

hundreds-father-404

12/17/2021, 6:31 PM
5 Views