https://pantsbuild.org/ logo
Join Slack
Powered by
Hey folks, wrote my first v2 goal in 1.30 and, whi...
# development
w
Hey folks, wrote my first v2 goal in 1.30 and, while working locally its bombing our CI.
Copy code
Copying src/python_json_logger.egg-info to build/bdist.linux-x86_64/wheel/python_json_logger-0.1.11-py3.7.egg-info
  running install_scripts
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-req-build-whse6jc7/setup.py", line 30, in <module>
      'Topic :: System :: Logging',
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/__init__.py", line 169, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/wheel/wheel/bdist_wheel.py", line 228, in run
      self.run_command('install')
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/command/install.py", line 65, in run
      return orig.install.run(self)
    File "/usr/lib/python3.7/distutils/command/install.py", line 601, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/command/install_scripts.py", line 22, in run
      import setuptools.command.easy_install as ei  # vendor:skip
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/command/easy_install.py", line 69, in <module>
      from setuptools.sandbox import run_setup  # vendor:skip
    File "/root/.cache/pants/named_caches/pex_root/pip.pex/6bd62cbe8c9f1ae82b35f976967538908799cbd6/.deps/setuptools/setuptools/sandbox.py", line 24, in <module>
      import pkg_resources.py31compat  # vendor:skip
  ModuleNotFoundError: No module named 'pkg_resources.py31compat'
going to retry it, but figured i'd post if someone had any insight
h
I can’t tell easily from the stacktrace - is this an error when Thanos is running its subprocess, or this is Pants itself erroring?
w
its happening when pants its trying to buildl the thanos.pex going to try to install python3-yaml in the debian container based on some other errors im observing but im not confident that's gonna fix it
almost positive its something to do with the. folllowing thread
h
Okay, so during the subprocess. For some reason, running the equivalent of 
pex --interpreter-constraint='CPython>=3.7' thanos -o thanos.pex
is failing. Likely 
setuptools
is the issue, that it’s expecting a certain version and it’s not there. I recommend “pinning” setuptools by including it in your 
requirements
for the 
PexRequest
, e.g. 
setuptools==44.1.3
setuptools
is really common to cause issues in the Python ecosystem. Lots of Python interpreters come with it pre-installed, so library authors forget to include it in their 
install-requires
metadata. For a couple tools Pants runs, like 
isort
, we automatically install 
setuptools
for the user to work around them forgetting to include it
w
hmm ok
h
In particular, the stacktrace makes it look like 
python_json_logger
is the culprit, which is expecting a particular version but is leaving it off
you’d want to make sure whatever version of 
setuptools
you’re using has 
pkg_resources.py31compat
in it, because that import is failing. Which might mean going to the GitHub project and searching for that file
w
this is inside my build container:
Copy code
>>>
# pip list | grep setuptools

setuptools                 45.2.0
setuptools-scm             3.4.3
# # python
Python 3.7.9 (default, Aug 18 2020, 06:24:24)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import pkg_resources.py31compat
>>>
this is in my rule
Copy code
thanos_version = options.values.thanos_version
    pex = await Get(
        Pex,
        PexRequest,
        PexRequest(
            output_filename="thanos.pex",
            requirements=PexRequirements(["setuptools==45.2.0", f"thanos[cli]=={thanos_version}",]),
            interpreter_constraints=PexInterpreterConstraints(["CPython>=3.7"]),
            entry_point=options.values.thanos_entrypoint,
        ),
    )
still getting the same error
😞 1
h
Are you able to run the equivalent Pex command in the container, which I think is something like 
pex --interpreter-constraint='CPython>=3.7' thanos setuptools -o thanos.pex -m thanos
? You’d need to install 
pex
, which you can do with 
pip3 install pex
It’ll help isolate if the issue is indeed from Pex/Pip, or from Pants.
w
where can i find hte thanos.pex
h
this will create a 
thanos.pex
. 
-o
is short for 
--output-file
. 
-m
is the same as the 
entry_point
w
oh
h
and then the 
thanos
and 
setuptools
are the requirements you’re installing. You can specify for example 
thanos[cli]==3.2
w
there. an option for a repository url?
thanos is private
h
Yes. 
--repo=https://...
w
nice thanks
h
pex --help
has lots of options too For example, 
--constraints
is how we pass the constraints file to Pex But this will get you mostly to the same thing Pants runs
w
observing the same error in pex
👍 1
is 44.1.3 the version you use?
should i take this over to the #pex channel?
h
Sorry I was afk. Sure, #pex channel is a good idea
is 44.1.3 the version you use?
If it needs to maintain Python 2 support, yes. setuptools 45 is Py3 only
Does the same command work when you run on your mac? What interpreteres are available in the CI image? You can try out out other interpreters by changing 
--interpreter-constraints
to be more precise, or by setting 
--python
to a specific path
w
my mac can. run the command correctly
👍 1
trying to run i the container with a specific python
same error
gonna cross post on #pex
👍 1
Open in Slack
PreviousNext
Powered by