https://pantsbuild.org/ logo
Powered by
w

wonderful-iron-54019

09/16/2020, 8:07 PM
Hey folks, wrote my first v2 goal in 1.30 and, while working locally its bombing our CI.
Copy code
Copying src/python_json_logger.egg-info to build/bdist.linux-x86_64/wheel/python_json_logger-0.1.11-py3.7.egg-info
  running install_scripts
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-req-build-whse6jc7/setup.py", line 30, in <module>
      'Topic :: System :: Logging',
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/__init__.py", line 169, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/wheel/wheel/bdist_wheel.py", line 228, in run
      self.run_command('install')
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/command/install.py", line 65, in run
      return orig.install.run(self)
    File "/usr/lib/python3.7/distutils/command/install.py", line 601, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/command/install_scripts.py", line 22, in run
      import setuptools.command.easy_install as ei  # vendor:skip
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/command/easy_install.py", line 69, in <module>
      from setuptools.sandbox import run_setup  # vendor:skip
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/sandbox.py", line 24, in <module>
      import pkg_resources.py31compat  # vendor:skip
  ModuleNotFoundError: No module named 'pkg_resources.py31compat'
going to retry it, but figured i'd post if someone had any insight
h

hundreds-father-404

09/16/2020, 8:18 PM
I can’t tell easily from the stacktrace - is this an error when Thanos is running its subprocess, or this is Pants itself erroring?
w

wonderful-iron-54019

09/16/2020, 8:19 PM
its happening when pants its trying to buildl the thanos.pex going to try to install python3-yaml in the debian container based on some other errors im observing but im not confident that's gonna fix it
almost positive its something to do with the. folllowing thread
h

hundreds-father-404

09/16/2020, 8:22 PM
Okay, so during the subprocess. For some reason, running the equivalent of 
pex --interpreter-constraint='CPython>=3.7' thanos -o thanos.pex
is failing. Likely 
setuptools
is the issue, that it’s expecting a certain version and it’s not there. I recommend “pinning” setuptools by including it in your 
requirements
for the 
PexRequest
, e.g. 
setuptools==44.1.3
setuptools
is really common to cause issues in the Python ecosystem. Lots of Python interpreters come with it pre-installed, so library authors forget to include it in their 
install-requires
metadata. For a couple tools Pants runs, like 
isort
, we automatically install 
setuptools
for the user to work around them forgetting to include it
w

wonderful-iron-54019

09/16/2020, 8:22 PM
hmm ok
h

hundreds-father-404

09/16/2020, 8:23 PM
In particular, the stacktrace makes it look like 
python_json_logger
is the culprit, which is expecting a particular version but is leaving it off
you’d want to make sure whatever version of 
setuptools
you’re using has 
pkg_resources.py31compat
in it, because that import is failing. Which might mean going to the GitHub project and searching for that file
w

wonderful-iron-54019

09/16/2020, 8:59 PM
this is inside my build container:
Copy code
>>>
# pip list | grep setuptools

setuptools                 45.2.0
setuptools-scm             3.4.3
# # python
Python 3.7.9 (default, Aug 18 2020, 06:24:24)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import pkg_resources.py31compat
>>>
this is in my rule
Copy code
thanos_version = options.values.thanos_version
    pex = await Get(
        Pex,
        PexRequest,
        PexRequest(
            output_filename="thanos.pex",
            requirements=PexRequirements(["setuptools==45.2.0", f"thanos[cli]=={thanos_version}",]),
            interpreter_constraints=PexInterpreterConstraints(["CPython>=3.7"]),
            entry_point=options.values.thanos_entrypoint,
        ),
    )
still getting the same error
😞 1
h

hundreds-father-404

09/16/2020, 9:01 PM
Are you able to run the equivalent Pex command in the container, which I think is something like 
pex --interpreter-constraint='CPython>=3.7' thanos setuptools -o thanos.pex -m thanos
? You’d need to install 
pex
, which you can do with 
pip3 install pex
It’ll help isolate if the issue is indeed from Pex/Pip, or from Pants.
w

wonderful-iron-54019

09/16/2020, 9:03 PM
where can i find hte thanos.pex
h

hundreds-father-404

09/16/2020, 9:03 PM
this will create a 
thanos.pex
. 
-o
is short for 
--output-file
. 
-m
is the same as the 
entry_point
w

wonderful-iron-54019

09/16/2020, 9:03 PM
oh
h

hundreds-father-404

09/16/2020, 9:04 PM
and then the 
thanos
and 
setuptools
are the requirements you’re installing. You can specify for example 
thanos[cli]==3.2
w

wonderful-iron-54019

09/16/2020, 9:05 PM
there. an option for a repository url?
thanos is private
h

hundreds-father-404

09/16/2020, 9:07 PM
Yes. 
--repo=https://...
w

wonderful-iron-54019

09/16/2020, 9:07 PM
nice thanks
h

hundreds-father-404

09/16/2020, 9:07 PM
pex --help
has lots of options too For example, 
--constraints
is how we pass the constraints file to Pex But this will get you mostly to the same thing Pants runs
w

wonderful-iron-54019

09/16/2020, 9:11 PM
observing the same error in pex
👍 1
is 44.1.3 the version you use?
should i take this over to the #pex channel?
h

hundreds-father-404

09/16/2020, 10:03 PM
Sorry I was afk. Sure, #pex channel is a good idea
is 44.1.3 the version you use?
If it needs to maintain Python 2 support, yes. setuptools 45 is Py3 only
Does the same command work when you run on your mac? What interpreteres are available in the CI image? You can try out out other interpreters by changing 
--interpreter-constraints
to be more precise, or by setting 
--python
to a specific path
w

wonderful-iron-54019

09/16/2020, 10:07 PM
my mac can. run the command correctly
👍 1
trying to run i the container with a specific python
same error
gonna cross post on #pex
👍 1
Powered by