I guess if I was dockerizing a build, as well caching the result from 1 would be great

Out of curiosity, what does using

bdist_pex

make easier for you over just using

pex

directly?

old habbits I guess

I've never really understood the use case.

I guess if I do the 'new thing' and not even have a

setup.py

at all I should be using just

pex

directly

I think a long time ago, I used

bdist_pex

and never ever questioned it until today

probably a good time to stop since I think distuitls plugins are not 'cool' anymore

Ok.

pex project/ ...

Should be equivalent (or

.

If the project dir is where you're at already).

perfect!

Aha! Ok. I thought you knew that but had a reason.

I might kill bdist_pex in 3.0 if there are zero use cases it adds.

you should, I don't think it adds value and I think the other use cases for bdist style stuff like

bdist_rpm

are going away as well

just to follow up here, I think that if you want to get rid of

bdist_pex

you should remove it from the docs first

so just to discuss some more, after the data I added to the ticket.

using

--pex-repository

with a pex of the requirements results in:

pex.resolver.Untranslatable: Cannot resolve local projects from PEX repositories. Asked to resolve /Users/zmanji/code/git-squash from ./dist/requirements.pex.

using it with

--requirements-pex ./dist/requirements.pex

seems to work fine however

this command seems to be pretty fast

pex . --requirements-pex ./dist/requirements.pex --no-build --wheel --no-transitive --inherit-path=false --script git-squash -o dist/out.pex

is this what you were suggesting?

Nope. Like so: Create the repository:

$ time pex . -r requirements.txt -c git-squash -o git-squash.pex

real	0m5.389s
user	0m4.663s
sys	0m0.524s

Use it:

$ time pex git-squash==0.0.1 -r requirements.txt --pex-repository git-squash.pex -c git-squash -o git-squash.2.pex -c git-squash

real	0m0.339s
user	0m0.302s
sys	0m0.035s

You get the same pex in this case (a full resolve, not a subset):

$ ./git-squash.2.pex -h
usage: git-squash.2.pex [-h] branch

positional arguments:
  branch      The upstream branch to squash commits of the current branch on to.

optional arguments:
  -h, --help  show this help message and exit
$ diff <(pex-tools git-squash.pex info -i4) <(pex-tools git-squash.2.pex info -i4)
$

But you could get a subset:

$ time pex GitPython -o GitPython.pex

real	0m0.791s
user	0m0.652s
sys	0m0.093s
$ time pex GitPython -o GitPython.2.pex --pex-repository git-squash.pex 

real	0m0.317s
user	0m0.289s
sys	0m0.029s
$ diff <(pex-tools GitPython.pex info -i4) <(pex-tools GitPython.2.pex info -i4)
^jsirois@gill /tmp/git-squash (master *) $ ./GitPython.2.pex 
Python 3.9.5 (default, May 24 2021, 12:50:35) 
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from git import Repo
>>> repo = Repo()
>>> repo.rev_parse("HEAD")
<git.Commit "505cf06c598409db9d119a521480e43de3fa8908">
>>> 
now exiting InteractiveConsole...

The

--pex-repository

resolve speed is going to be pretty much constant. PEX file build times will become dominated by zip times.

And if you mess up, Pex will let you know:

$ time pex GitPython isort -o GitPython.2.pex --pex-repository git-squash.pex 
A distribution for isort could not be resolved in this environment.

real	0m0.197s
user	0m0.177s
sys	0m0.021s

Thanks a lot John!

https://twitter.com/pantsbuild/status/1403048496404602884 Pants Build retweeted: On June 24-25, join us at @build_meetup conference online for two talks of talks and discussion panels about build… https://twitter.com/i/web/status/1403048496404602884 Twitter

https://twitter.com/semaphoreci/status/1403038045331468288 Pants Build retweeted: A good test is: 1. Deterministic: always produces the same output for given input 2. Fully-automated: endlessly &… https://twitter.com/i/web/status/1403038045331468288 Twitter

Can someone help me debug the code that pants is providing to pylint? The

./pants dependencies ...

output looks correct, but pylint is complaining that it can't find some modules (modules that are listed in the

./pants dependencies

output). What's more, if I only include the targets in that directory, pylint has a different set of errors. And if I do a very narrow selection of files to lint (including files that had errors with a larger selection of targets) then I don't get any errors at all. I thought maybe it was overlapping source roots, so I temporarily deleted

//conftest.py

, the relevant target from

//BUILD

, and the

/

source root, but that didn't resolve it. Here's a bunch of

./pants ...

output: https://gist.github.com/cognifloyd/f428a423a07e1e859d4e2625c0cd66f1 And here's the code base I'm working with: https://github.com/st2sandbox/st2/tree/pants

https://twitter.com/semaphoreci/status/1404849961771180035 Pants Build retweeted: Building #Python apps is a complex jigsaw of virtual envs, pip, pipenv, setuptools, poetry commands... Meet… https://twitter.com/i/web/status/1404849961771180035 Twitter

I've done something very wrong but I'm not sure what. First I tried to upgrade from pants 2.4 to 2.5 by updating the version in my

pants.toml

. When I then ran

./pants typecheck

I got:

ERROR: Could not find a version that satisfies the requirement mypy==0.818

I killed the daemon and did an

rm -rf ~/.cache/pants

but to no avail. So I tried to downgrade back to 2.4 but I get the same error but now with

mypy==0.800

. What did I do wrong and how do I fix it?

I'm also wondering how to use pip installed type hints. For example, I can add

grpc-stubs

to my requirements but they're not automatically used. I tried adding

//:grpc-stubs

to the

dependencies

for the targets using grpc but that didn't do the trick either.

Hey, trying the upgrade to

2.5.0

from

2.4.0

. Everything seems to be working great except for pylint when running pylint with

--changed-since=<branch>

I'm getting

ModuleNotFoundError: No module named 'pkg_resources'

Runs great when running without

--changed-since

Have a strange issue. I have a pants build that includes TensorFlow 1.15 which transitively depends on gast==0.2.2. That exists on pypi for Python 3.7 and I can enter a py3.7 venv and

pip install -r constraints.txt

without issues. However, when I try to

pants package

I get:

stderr:
ERROR: Could not find a version that satisfies the requirement gast==0.2.2
ERROR: No matching distribution found for gast==0.2.2
pid 794090 -> /home/oliver/.cache/pants/named_caches/pex_root/venvs/7f48ea14fe9a31ad7cc2422df7a949fe8899372d/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /home/oliver/.cache/pants/named_caches/pex_root --log /tmp/process-executionMrBaUB/.tmp/tmpmhp64qj_/pip.log download --dest /tmp/process-executionMrBaUB/.tmp/tmpq73a30sv/macosx_10_14_x86_64-cp-37-cp37m --platform macosx_10_14_x86_64 --implementation cp --python-version 37 --abi cp37m --only-binary :all: --constraint constraints.txt absl-py==0.13.0 astor==0.8.1 attrs==21.2.0 backoff==1.10.0 cachetools==4.2.2 certifi==2021.5.30 chardet==4.0.0 gast==0.2.2 google-api-core==1.30.0 google-auth==1.31.0 google-cloud-core==1.7.0 google-cloud-firestore==2.1.3 google-pasta==0.2.0 googleapis-common-protos==1.53.0 grpcio==1.38.0 h5py==2.10.0 idna==2.10 importlib-metadata==4.5.0 iniconfig==1.1.1 keras-applications==1.0.8 keras-preprocessing==1.1.2 labelbox==2.6.0 markdown==3.3.4 ndjson==0.3.1 numpy==1.18.5 opencv-python-headless==4.5.2.54 opt-einsum==3.3.0 packaging==20.9 pluggy==0.13.1 proto-plus==1.18.1 protobuf==3.17.3 py==1.10.0 pyasn1-modules==0.2.8 pyasn1==0.4.8 pydantic==1.8.2 pyparsing==2.4.7 pytest==6.2.4 pytz==2021.1 requests==2.25.1 rsa==4.7.2 six==1.16.0 tensorboard==1.15.0 tensorflow-estimator==1.15.1 tensorflow==1.15.5 termcolor==1.1.0 toml==0.10.2 typing-extensions==3.10.0.0 urllib3==1.26.5 werkzeug==2.0.1 wheel==0.36.2 wrapt==1.12.1 zipp==3.4.1 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
None

Hey Pants Team, saw a new error from

isort

when running

./pants fmt --changed-since=origin/master

. The BUILD file in the changed targets has

skip_isort=True

and it seems pants is executing

isort

with 0 files which causes an error.

07:22:13.36 [DEBUG] Completed: Run isort on 0 files.
07:22:13.36 [DEBUG] Completed: (Waiting) Run isort on 0 files.
07:22:13.36 [DEBUG] Completed: Format with isort
07:22:13.36 [DEBUG] Completed: `fmt` goal
07:22:13.36 [DEBUG] computed 1 nodes in 2.802568 seconds. there are 448 total nodes.
07:22:13.36 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/Users/njgrisafi/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.0_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 234, in _run_inner
    return self._perform_run(goals)
  File "/Users/njgrisafi/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.0_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 173, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/Users/njgrisafi/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.0_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 195, in _perform_run_body
    poll_delay=(0.1 if poll else None),
  File "/Users/njgrisafi/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.0_py37/lib/python3.7/site-packages/pants/init/engine_initializer.py", line 136, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/Users/njgrisafi/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.0_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 530, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/Users/njgrisafi/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.0_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 494, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),

Exception message: 1 Exception encountered:

Engine traceback:
  in select
  in pants.core.goals.fmt.fmt
  in pants.backend.python.lint.python_fmt.format_python_target
  in pants.core.goals.fmt.enrich_fmt_result
  in pants.backend.python.lint.isort.rules.isort_fmt
  in pants.engine.process.fallible_to_exec_result_or_raise
Traceback (most recent call last):
  File "/Users/njgrisafi/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.0_py37/lib/python3.7/site-packages/pants/engine/process.py", line 254, in fallible_to_exec_result_or_raise
    description.value,
pants.engine.process.ProcessExecutionFailure: Process 'Run isort on 0 files.' failed with exit code 1.
stdout:


                 _                 _
                (_) ___  ___  _ __| |_
                | |/ _/ / _ \/ '__  _/
                | |\__ \/\_\/| |  | |_
                |_|\___/\___/\_/   \_/

      isort your imports, so you don't have to.

                    VERSION 5.5.5


Nothing to do: no files or paths have have been passed in!

Try one of the following:

    `isort .` - sort all Python files, starting from the current directory, recursively.
    `isort . --interactive` - Do the same, but ask before making any changes.
    `isort . --check --diff` - Check to see if imports are correctly sorted within this project.
    `isort --help` - In-depth information about isort's available command-line options.

Visit <https://pycqa.github.io/isort/> for complete information about how to use isort.


stderr:
Error: arguments passed in without any paths or content.

Has anyone seen this before?