brash-baker-91190
01/13/2022, 9:23 PMProcessExecutionFailure: Process 'Building pytest_runner.pex' failed with exit code 1.
Followed by a bunch of these:
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/maier/.cache/pants/named_caches/pex_root/unzipped_pexes/f00d5546a767d36a195e863e80031df5797b7eb8.
Needed manylinux_2_28_x86_64-cp-37-cp37m compatible dependencies for:
1: boto3
Required by:
FingerprintedDistribution(distribution=grapl-common 0.0.6 (/home/maier/.cache/pants/named_caches/pex_root/installed_wheels/7f5c555e91114b099cc3946517adba787c2ede3e/grapl_common-0.0.6-py3-none-any.whl), fingerprint='a76df1c3f9ecd1ae5ae2352850c933b30140ba93')
But this pex had no 'boto3' distributions.
2: botocore
Required by:
FingerprintedDistribution(distribution=grapl-common 0.0.6 (/home/maier/.cache/pants/named_caches/pex_root/installed_wheels/7f5c555e91114b099cc3946517adba787c2ede3e/grapl_common-0.0.6-py3-none-any.whl), fingerprint='a76df1c3f9ecd1ae5ae2352850c933b30140ba93')
But this pex had no 'botocore' distributions.
3: mypy-boto3-cloudwatch
Required by:
FingerprintedDistribution(distribution=grapl-common 0.0.6 (/home/maier/.cache/pants/named_caches/pex_root/installed_wheels/7f5c555e91114b099cc3946517adba787c2ede3e/grapl_common-0.0.6-py3-none-any.whl), fingerprint='a76df1c3f9ecd1ae5ae2352850c933b30140ba93')
But this pex had no 'mypy-boto3-cloudwatch' distributions.
4: mypy-boto3-dynamodb
Required by:
FingerprintedDistribution(distribution=grapl-common 0.0.6 (/home/maier/.cache/pants/named_caches/pex_root/installed_wheels/7f5c555e91114b099cc3946517adba787c2ede3e/grapl_common-0.0.6-py3-none-any.whl), fingerprint='a76df1c3f9ecd1ae5ae2352850c933b30140ba93')
But this pex had no 'mypy-boto3-dynamodb' distributions.
(plus 8 similar missing dependencies)
Any pointers on where to look? I've regenerated my pytest lockfile. Thanks in advance.hundreds-father-404
01/13/2022, 9:25 PMpytest_runner.pex
is actually the fusion of a few PEXes, including the pytest.pex
(your tool lockfile) and the requirements.pex
(your code). It uses the --pex-path
feature which allows you to merge PEXesbrash-baker-91190
01/13/2022, 9:25 PMgrapl-common
, which exists within this monorepo) that is a dependency of the code under test. The tests for grapl-common
run just fine.
• with the exception of boto3
and botocore
(numbers 1 and 2 in the output above), all the missing dependencies are mypy-boto-XXX
type libraries and typing-extensions
Not sure if this is useful, but this is what stands out to me at this point.hundreds-father-404
01/13/2022, 11:27 PMbrash-baker-91190
01/13/2022, 11:29 PMnamed_caches
directory aside (creatively renamed it to named_caches_orig
) and running my test yields the same failure.hundreds-father-404
01/13/2022, 11:30 PMnamed_caches
and using --no-local-cache --no-pantsd
? Want to make sure this isn't from a "corrupted" Pex cache or anything, which has happened in the past iirc.brash-baker-91190
01/13/2022, 11:32 PMhundreds-father-404
01/14/2022, 10:45 PMbrash-baker-91190
01/14/2022, 10:51 PMhundreds-father-404
01/14/2022, 10:58 PMmanylinux_2_28_x86_64-cp-37-cp37m
by chance. @polite-garden-50641 found that upgrading to Pants 2.10 is resulting in this error message for us:
Failed to resolve requirements from PEX environment @ /home/toolchain/.cache/pants/named_caches/pex_root/unzipped_pexes/11c095ad69b7a3c1a757dbfcd20347104c1a6840.
Needed manylinux_2_28_x86_64-cp-38-cp38 compatible dependencies for:
1: py>=1.8.2
Required by:
FingerprintedDistribution(distribution=pytest 6.2.5 (/home/toolchain/.cache/pants/named_caches/pex_root/installed_wheels/d8dbe72f24eaeea8b51e0f353336f9bd4915dc4a/pytest-6.2.5-py3-none-any.whl), fingerprint='c81a0f09ee54ea9496a98d7b1b12d41286248226')
But this pex had no 'py' distributions.
To confirm, do you get this issue on macOS? (if you have access)I'm wondering if this is an issue with manylinux_2_28_x86_64-cp-37-cp37m by chance.Probably red-herring. Asher also has the issue on a mac
polite-garden-50641
01/14/2022, 11:05 PMbrash-baker-91190
01/14/2022, 11:27 PMhundreds-father-404
01/14/2022, 11:29 PMbrash-baker-91190
01/14/2022, 11:46 PMpants
invocation hits the specific test files that trigger this)
git clone <https://github.com/grapl-security/grapl>
git checkout cm/pants-2.9
./pants --tag="-needs_work" test --pytest-args="-m \"not integration_test\"" src/python/graplctl/tests:tests
hundreds-father-404
01/14/2022, 11:49 PMbrash-baker-91190
01/14/2022, 11:50 PMhundreds-father-404
01/19/2022, 6:19 PMenough-analyst-54434
01/19/2022, 6:58 PMbrash-baker-91190
01/19/2022, 6:59 PMenough-analyst-54434
01/19/2022, 7:20 PMdiff --git a/src/python/graplctl/graplctl/BUILD b/src/python/graplctl/graplctl/BUILD
index 1e8a0ce9..050a642b 100644
--- a/src/python/graplctl/graplctl/BUILD
+++ b/src/python/graplctl/graplctl/BUILD
@@ -1,14 +1,7 @@
python_sources(
name="graplctl_lib",
- dependencies=[
- "src/python/grapl-common:grapl-common",
- ],
)
pex_binary(
entry_point="cli.py:main",
- # TODO: Once <https://github.com/pantsbuild/pants/pull/13690> lands,
- # remove this work-around
- # See also <https://pantsbuild.slack.com/archives/C046T6T9U/p1637597807103900>
- dependencies=["!!src/python/grapl-common:grapl-common"],
)
brash-baker-91190
01/19/2022, 7:28 PMenough-analyst-54434
01/19/2022, 7:31 PMbrash-baker-91190
01/19/2022, 7:32 PMenough-analyst-54434
01/19/2022, 7:35 PMbrash-baker-91190
01/24/2022, 6:55 PM/bin/bash: line 1: ./analyzer-executor.pex: Permission denied
I'm continuing to bisect things to figure out at what point in the 2.9 dev/rc line our system starts to fall over. Our test process is kind of long, though, so I'm just picking at this between other tasks (we're not blocked or anything; Pants 2.8 continues to work like a champ for us in this repository). Just wanted to keep y'all in the loop; I'll report back if/when I find anything.enough-analyst-54434
01/24/2022, 7:12 PMthey're not starting up in Nomad... just seeing errors like this:That looks like it can only really be that the 755 bits added to/bin/bash: line 1: ./analyzer-executor.pex: Permission denied
analyzer-executor.pex
by Pex when it creates the PEX have been wiped out. That shouldn't be related to any Pex or Pants upgrades. Are you sure there are no other variable in play for that error re perms / cp, etc?brash-baker-91190
01/24/2022, 7:15 PM-rwxr-xr-x
permissions, but in 2.9, they get -rwxr--r--
... in our container, our PEX files are owned by root, but run by a separate user. Easy enough fix in our containers. Is this an expected change in behavior, though?enough-analyst-54434
01/27/2022, 3:54 PM$ rm -rf dist && ./pants package src/python/pants/bin::
07:52:22.51 [INFO] Completed: Installing 3rdparty/python/lockfiles/user_reqs.txt
07:52:28.06 [INFO] Completed: Extracting 15 requirements to build src.python.pants.bin/pants.pex from lockfile.pex: PyYAML<7.0,>=6.0, ansicolors==1.1.8, fasteners==0.16.3, humbug==0.2.7, ijson==3.1.4, packaging==21.3, pex==2.1.65,... (168 characters truncated)
07:52:28.08 [INFO] Wrote dist/src.python.pants.bin/pants.pex
$ ls -l dist/src.python.pants.bin/pants.pex
-rwxr--r-- 1 jsirois jsirois 53595441 Jan 27 07:52 dist/src.python.pants.bin/pants.pex
That seems worth a bug @brash-baker-91190brash-baker-91190
01/27/2022, 3:54 PMenough-analyst-54434
01/27/2022, 3:58 PMbrash-baker-91190
01/27/2022, 3:59 PMenough-analyst-54434
01/27/2022, 3:59 PMbrash-baker-91190
01/27/2022, 3:59 PM