https://pantsbuild.org/ logo
#general
Title
# general
c

calm-ambulance-65371

04/19/2022, 5:06 PM
Hello, I'm running into some issues with pants
2.10.1rc0
(and
2.10.0
) where a
python_distribution
will generate a
.tar.gz
which gets enumerated by
isolate_local_dist_wheels
and subsequently fails when
unzip -Z1
returns a non 0 exit code because it has no idea what to do with a
.tar.gz
I'm not sure if this is from something in our config/BUILDs, but this seems like a pretty critical failure
The problem also goes away if you change internal dependencies to only depend on the actual targets, not the
python_distribution
, however there's a lot of dependencies I'd have to clean up, and relying on the dist is conceptually cleaner regardless
h

happy-kitchen-89482

04/19/2022, 5:13 PM
c

calm-ambulance-65371

04/19/2022, 5:14 PM
Yeah, that appears to be exactly the problem
h

happy-kitchen-89482

04/19/2022, 5:15 PM
If so, the workaround as described there is to set
sdist=False
on the python_distribution, so it doesn't generate the sdist in the first place. But that is only a hack, this bug (as you can see from my comments on it) is very befuddling because we filter on the
.whl
suffix, so I cannot understand how this is happening at all...
Do you happen to have an easy repro of this to share?
e

enough-analyst-54434

04/19/2022, 5:20 PM
The direct implication is that DigestSubset has a bug. Has that been explored?
c

calm-ambulance-65371

04/19/2022, 5:20 PM
unfortunately it's basically my companies monorepo, I can try to throw something together. Alternatively, attempt to debug live
h

happy-kitchen-89482

04/19/2022, 5:21 PM
yeah I mean the simplest explanation, from staring at the code, is a DigestSubset bug, but I have not been able to reproduce it.
c

calm-ambulance-65371

04/19/2022, 5:22 PM
is there a way to a repo of pants instead of the distributed ones?
ie so I can toss in some prints/pdbs?
h

happy-kitchen-89482

04/19/2022, 5:23 PM
That would be great! Yes, there is a way to run Pants directly from its sources
(scrambles for the link)
😂 1
c

calm-ambulance-65371

04/19/2022, 5:24 PM
neato, please hold
Specifically, if
wheels_snapshot.files
contains anything that doesn't end with
.whl
then that is pretty bad...
c

calm-ambulance-65371

04/19/2022, 5:30 PM
waiting on cargo doing it's thing, but yeah I added log statement there. Could I expect breakpoints to work or is that not going to play nicely with the Rust runtime?
breakpoints meaning
import pdb; pdb.set_trace()
h

happy-kitchen-89482

04/19/2022, 5:30 PM
Breakpoints should work, but your stacktraces will be uninformative
Because they will disappear into the rust engine
c

calm-ambulance-65371

04/19/2022, 5:31 PM
it'll probably eat the stdout/stdin and not actually be interactive then 🙂
h

happy-kitchen-89482

04/19/2022, 5:32 PM
If the print statement shows that the filtering isn't working, one quick thing to try is to replace that line with:
Copy code
wheels_digest = await Get(Digest, DigestSubset(dist.digest, PathGlobs(["**/*.whl"])))
wheels_snapshot = await Get(Snapshot, Digest, wheels_digest)
And see if that makes a difference. It shouldn't, but we're already through the looking glass at this point.
w

witty-crayon-22786

04/19/2022, 5:36 PM
https://pantsbuild.slack.com/archives/C046T6T9U/p1650389498493749?thread_ts=1650388000.877219&cid=C046T6T9U if you run with
--no-pantsd
you’ll be more likely to succeed in `pdb`*, yea. as to
print
statements: they’ll be rendered, but prefer log statements in order to get better formatting
c

calm-ambulance-65371

04/19/2022, 5:37 PM
Copy code
12:36:04.35 [DEBUG] Wheels snapshot: ['<redacted>-py36.py3
7.py38.py39-none-any.whl', '<redacted>.tar.gz']
uh oh 😬
from this statement: on line 57
logger.debug("Wheels snapshot: %s", wheels)
the above code fix didn't change anything
Copy code
wheels_digest = await Get(Digest, DigestSubset(dist.digest, PathGlobs(["**/*.whl"])))
wheels_snapshot = await Get(Snapshot, Digest, wheels_digest)
this one
Where does the logic for resolving
DigestSubset
live?
h

happy-kitchen-89482

04/19/2022, 5:49 PM
@calm-ambulance-65371 Really appreciate the debugging help here BTW!
c

calm-ambulance-65371

04/19/2022, 5:49 PM
np
❤️ 1
h

happy-kitchen-89482

04/19/2022, 5:49 PM
If you want to throw some print statements in the rust code that would be awesome, but potentially also noisy
👍 1
c

calm-ambulance-65371

04/19/2022, 6:03 PM
is there a way to eliminate concurrency?
nvm figured it out
rust build is so sloooow
c

curved-television-6568

04/19/2022, 6:18 PM
c

calm-ambulance-65371

04/19/2022, 6:38 PM
I am unfamiliar with the globbing library used, but here's the output from my print:
Copy code
Globs:PreparedPathGlobs {
    include: [
        PathGlobIncludeEntry {
            input: GlobParsedSource(
                "**/*.whl",
            ),
            globs: [
                DirWildcard {
                    canonical_dir: Dir(
                        "",
                    ),
                    symbolic_path: "",
                    wildcard: Pattern {
                        original: "*",
                        tokens: [
                            AnySequence,
                        ],
                        is_recursive: false,
                    },
                    remainder: [
                        Pattern {
                            original: "**",
                            tokens: [
                                AnyRecursiveSequence,
                            ],
                            is_recursive: true,
                        },
                        Pattern {
                            original: "*.whl",
                            tokens: [
                                AnySequence,
                                Char(
                                    '.',
                                ),
                                Char(
                                    'w',
                                ),
                                Char(
                                    'h',
                                ),
                                Char(
                                    'l',
                                ),
                            ],
                            is_recursive: false,
                        },
                    ],
                },
                Wildcard {
                    canonical_dir: Dir(
                        "",
                    ),
                    symbolic_path: "",
                    wildcard: Pattern {
                        original: "*.whl",
                        tokens: [
                            AnySequence,
                            Char(
                                '.',
                            ),
                            Char(
                                'w',
                            ),
                            Char(
                                'h',
                            ),
                            Char(
                                'l',
                            ),
                        ],
                        is_recursive: false,
                    },
                },
            ],
        },
    ],
    exclude: GitignoreStyleExcludes {
        patterns: [],
        gitignore: Gitignore {
            set: GlobSet {
                len: 0,
                strats: [],
            },
            root: "",
            globs: [],
            num_ignores: 0,
            num_whitelists: 0,
            matches: None,
        },
    },
    strict_match_behavior: Ignore,
    conjunction: AnyMatch,
    patterns: [
        Pattern {
            original: "**/*.whl",
            tokens: [
                AnyRecursiveSequence,
                AnySequence,
                Char(
                    '.',
                ),
                Char(
                    'w',
                ),
                Char(
                    'h',
                ),
                Char(
                    'l',
                ),
            ],
            is_recursive: true,
        },
    ],
}, 
Digest:Digest {
    hash: Fingerprint<934af979fdb5a823fc6de3e173933a839503470e4b796a608731c01a741ab70f>,
    size_bytes: 350,
}
sorry guys, I don't know rust very well, so I can't do much beyond this. If you think of anything you need me to print and share here, let me know. I'm happy to do that
I added in the
sdist=false
to my
python_distributions
and it seems to get me past that. Now I'm seeing another error:
Copy code
ERROR: Package 'common' requires a different Python: 3.8.10 not in '>=3.6'
seems kinda odd lol
is toolchain labs hiring? 😛
h

happy-kitchen-89482

04/19/2022, 7:36 PM
Re the debugging, it would be great to print
path_stats
and
files
in the rust code, right at the end before the call to
DigestTrie::from_path_stats
👍 1
That error is odd indeed!
c

calm-ambulance-65371

04/19/2022, 7:41 PM
there's perhaps some confusion, I'm working off of the 2.10.1rc0 branch, so
snapshot_glob_match
looks a bit different: https://github.com/pantsbuild/pants/blob/release_2.10.1rc0/src/rust/engine/fs/store/src/snapshot_ops.rs#L502
h

happy-kitchen-89482

04/19/2022, 7:48 PM
Oh, huh
c

calm-ambulance-65371

04/19/2022, 7:52 PM
perhaps need to backport that merge?
h

happy-kitchen-89482

04/19/2022, 7:52 PM
Looks like this problem may go away in 2.11.x
Not sure if we can backport it cleanly. @witty-crayon-22786?
c

calm-ambulance-65371

04/19/2022, 7:53 PM
I'll see if switching to 2.11.0rc3 works
cargo builds aggressively
h

happy-kitchen-89482

04/19/2022, 7:55 PM
You can go back to consuming 2.11.0rc3 from a built distribution, no?
w

witty-crayon-22786

04/19/2022, 7:56 PM
it will not backport at all, unfortunately. sits on top of tons of changes =/
c

calm-ambulance-65371

04/19/2022, 7:57 PM
yes, but not if I want to debug print 🙂
it seems
BinaryPathRequest
moved 😐
into src/python/pants/core/util_rules/system_binaries.py
h

happy-kitchen-89482

04/19/2022, 8:10 PM
If things work in 2.11.0rc3 then we won't need debug prints 🙂
I strongly suspect this was the issue
c

calm-ambulance-65371

04/19/2022, 8:44 PM
I'm having trouble with
DuplicateNameError
due to
python_requirements
apparently there's a naming conflict but I can't give the
python_requirements
a different name 😕
Copy code
DuplicateNameError: A target already exists at 'common/BUILD' with name 'common' and target type 'python_requirements'. The 'python_distribution' target cannot use the same name.
Further, it seems we can't support multiple
python_requirements
targets at the same level (rather common for things such as test requirements) in pants
2.11
this is a complete blocker for upgrading, and considering there's a pretty critical issue in
2.10
and possibly older versions, I'm not sure what we can do at this point
h

happy-kitchen-89482

04/19/2022, 9:20 PM
Ah, there is an annoying upgrade dance re
python_requirements
, @hundreds-father-404 knows the details. I think it's in the error message?
c

calm-ambulance-65371

04/19/2022, 9:20 PM
there's not really any error message
docs on what I have to do would be appreciated
h

happy-kitchen-89482

04/19/2022, 9:27 PM
Yeah, sorry about that, for reasons I can't recall off hand that upgrade had an extra bit of complexity. This should help: https://www.pantsbuild.org/v2.11/docs/python-third-party-dependencies#first-turn-off-old-style-macros
Apologies for how truly annoying this is
Once you set
Copy code
[GLOBAL]
use_deprecated_python_macros = false
You can add
name=
to your
python_requirements
to differentiate them
h

hundreds-father-404

04/19/2022, 9:39 PM
Further, it seems we can't support multiple python_requirements targets at the same level (rather common for things such as test requirements) in pants 2.11
Before pants 2.10, this was indeed impossible. It was a major issue we wanted to solve. It is possible in 2.10+ via
use_deprecated_python_macros
. That defaults to true in 2.10, but false in 2.11 There is indeed an awkward dance to migrate to the new system. The instructions should have been logged to you, but here they are https://github.com/pantsbuild/pants/blob/f50c6744243e0d8eac0d8e9fb715a6290585f397/src/python/pants/option/global_options.py#L1653
c

calm-ambulance-65371

04/19/2022, 9:40 PM
huh, we've been stuck on
2.2
for awhile and have had multiple
python_requirements
living along-side each other for awhile. Idk if we've actually been using their generated targets though
also thank you @happy-kitchen-89482!
h

hundreds-father-404

04/19/2022, 9:42 PM
If you are on 2.2 still, then it works as long as they don't include any of the same requirements 🤷 (fixing this has been a major crusade for me the past few months)
What's blocking upgrading past 2.2?
c

calm-ambulance-65371

04/19/2022, 9:42 PM
we're trying to get off 2.2 for a few reasons, it's very old at this piont
1
we were previously blocked by dep issues due to a hard req on python 3.5
now we're free, but we ran into some dep issues with the new pip dependency resolving algorithm
👍 1
h

hundreds-father-404

04/19/2022, 9:58 PM
Please do feel free to keep reaching out for help while upgrading. We've made a lot of substantial improvements since 2.2
c

calm-ambulance-65371

04/19/2022, 9:58 PM
will do, thanks for the help!
❤️ 1
One question:
Copy code
You have set interpreter constraints (`CPython<4,>=3.7`) that are not compatible with those used to generate the lockfile (`CPython<4,>=3.8 OR CPython>=3.6,>=3.8`). You can fix this by not setting `[black].interpreter_constraints`, or by using a new custom lockfile.
any idea why I'm getting that when I just generated the lockfile?
h

hundreds-father-404

04/19/2022, 9:59 PM
What goal are you running? Which pants version also?
c

calm-ambulance-65371

04/19/2022, 9:59 PM
update-build-files --fix-python-macros
2.11.0rc3
use_deprecated_python_macros = false
h

hundreds-father-404

04/19/2022, 10:01 PM
might be this https://github.com/pantsbuild/pants/issues/14912 😕 you can use
--no-fmt
, or try setting
[black].interpreter_constraints
to 3.6+ and regenerate lockfile
c

calm-ambulance-65371

04/19/2022, 10:06 PM
setting the
[black].interpreter_constraints
and regenerating appeared to fix it
👍 1
h

hundreds-father-404

04/19/2022, 10:07 PM
Cool. Then I think it is that one issue. Sorry for the broken window!
c

calm-ambulance-65371

04/19/2022, 10:07 PM
no worries, thanks much for the help!
now to fix all 4 million 3rd party dependencies for our monorepo...
🔥 1
h

hundreds-father-404

04/19/2022, 10:09 PM
Hopefully pants automates the majority of that 😱 benjy has already said this, but again our apologies for this deprecation dance
c

calm-ambulance-65371

04/19/2022, 10:09 PM
it would do something like
common:requests
->
common:requirements#requests
for me?
h

hundreds-father-404

04/19/2022, 10:11 PM
it should! You will need to look closely at the ERROR logs to add
name=
to the
python_requirements()
, but no manual changes for call sites. Just the definition
And do let us know if that is not the case. Several organizations have successfully migrated to the new target generator mechanism, so we think that we have now handled all the edge cases (including pants.toml). But it's possible there are more edges
c

calm-ambulance-65371

04/19/2022, 10:23 PM
looks like it's not happy (?) with my partially manually completed work:
Copy code
17:19:37.39 [ERROR] 1 Exception encountered:

  MappingError: Failed to parse ./interfaces/BUILD:
__call__() got an unexpected keyword argument 'name'
that BUILD looks like:
Copy code
python_requirements(name="requirements")

python_sources()
h

hundreds-father-404

04/19/2022, 10:24 PM
Yes, it is not happy with a partial state. You need to update the global option now
c

calm-ambulance-65371

04/19/2022, 10:25 PM
But then I get:
Copy code
17:25:08.13 [ERROR] 1 Exception encountered:

  ValueError: `--update-build-files-fix-python-macros` specified when `[GLOBAL].use_deprecated_python_macros` is already set to false, which means that there is nothing left to fix.
it lies!
🙂
h

hundreds-father-404

04/19/2022, 10:25 PM
It should be safe to change the global option as long as you have run the update build files goal already. And make sure you preserve those error log messages in case you have not finished them
you only need to run that one time
i.e. don't permanently set
--fix-python-macros
Let me know if you want to hop on a video chat
c

calm-ambulance-65371

04/19/2022, 10:33 PM
I appreciate the offer, but I'm about to log off for the day. I'll pick this back up tomorrow though. Is there some command besides
update-build-files --update-build-files-fix-python-macros
that I should be running to update the dependencies? I may just have gotten myself into a bad state and need to rework things myself 🙂
again, I really appreciate the help!
h

hundreds-father-404

04/19/2022, 10:35 PM
should be only the instructions at https://github.com/pantsbuild/pants/blob/f50c6744243e0d8eac0d8e9fb715a6290585f397/src/python/pants/option/global_options.py#L1653. As long as you follow all those instructions—including changing the global option after running the updater and applying each of the error log changes—should be good to go
Do you already have >1
python_requirements
in a single directory? There is this TODO that could give you trouble https://github.com/pantsbuild/pants/blob/b1fa13f8c0788f11564ef486e2f4fb5f9c56d2e8/src/python/pants/backend/python/macros/deprecation_fixers.py#L128
c

calm-ambulance-65371

04/20/2022, 6:17 PM
ok back at this, fixed up all our 3rd party deps., but now I'm running into some odd errors. Example:
Copy code
4: numpy
    Required by:
      FingerprintedDistribution(distribution=<redacted>-3136395a3be17fe9145c4a8a3d3f2d9c82fa6349 (/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/537d7fc53f4f80dc839310442f7d14a3ee8b01f8ab2d929f987f1ac55cbeaf30/<redacted>-3136395a3be17fe9145c4a8a3d3f2d9c82fa6349-py3-none-any.whl), fingerprint='6e7c538bf420421cd2668a8e1db727ea763104aef000bfa4d08fec0751891228')
      FingerprintedDistribution(distribution=<redacted>-3136395a3be17fe9145c4a8a3d3f2d9c82fa6349 (/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/19769ef6f634442291de1989e8ece3158aa195aa05b985ff08a85c4dab4af640/<redacted>-3136395a3be17fe9145c4a8a3d3f2d9c82fa6349-py38-none-any.whl), fingerprint='d5159cf4c0d5b014f94d9b896d527ffeeb0e2596957a38aba5bd391d9b8548a1')
    But this pex had no 'numpy' distributions.
👀 1
I have a bunch of these for our 3rd party deps
h

hundreds-father-404

04/20/2022, 6:22 PM
huh....if you temporarily rename the folder
/home/noah/.cache/pants/named_caches
?
c

calm-ambulance-65371

04/20/2022, 6:30 PM
same thing
completely wiping ~/.cache/pants doesn't help either 😞
h

hundreds-father-404

04/20/2022, 6:37 PM
How are you installing things? Are you on pants 2.11 and using lock files for example?
c

calm-ambulance-65371

04/20/2022, 6:38 PM
oh, actually, it might be because we're using a
constraints.txt
. Will try removing
nope
I have lockfiles for setuptools, pytest, and black. Our code (now) has no lock files
h

hundreds-father-404

04/20/2022, 6:41 PM
It should be fine to keep using constraints.txt. Soon, it would be good to switch to the new lock files, but it's best to do that as a dedicated change once things are already stable
How about using
-ldebug
and/or
--no-process-cleanup
? I am curious if this is something obvious like that we are not correctly saying to install numpy because eg deps are wrong, vs something else
c

calm-ambulance-65371

04/20/2022, 7:28 PM
I think we might just be somehow stuck in dep hell, not sure it's Pants' fault
h

hundreds-father-404

04/20/2022, 7:29 PM
How so? Were things working before?
c

calm-ambulance-65371

04/20/2022, 7:32 PM
yes, but I think there may have been some silent issues
not sure
Copy code
14:29:35.31 [ERROR] 1 Exception encountered:
ncy id: 3, and concurrency: 1
  ProcessExecutionFailure: Process 'Building pytest_runner.pex' failed with exit code 1.                                       _runner.pex under semaphore with concurre
stdout:
14:00:23.69 [INFO] Preserving local process execution dir /tmp/process-executionCURx
stderr:
Appears to be failing when making the pytest_runner pex
what's odd is some tests pass
nvm, tests aren't running
h

hundreds-father-404

04/20/2022, 7:34 PM
is there more to the error?
c

calm-ambulance-65371

04/20/2022, 7:36 PM
Copy code
Traceback (most recent call last):
  File "/home/noah/.cache/pants/named_caches/pex_root/unzipped_pexes/aecac44135d419f932725eb33af2df4856517e99/.bootstrap/pex/pex.py", line 504, in execute
    exit_value = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/home/noah/.cache/pants/named_caches/pex_root/unzipped_pexes/aecac44135d419f932725eb33af2df4856517e99/.bootstrap/pex/pex.py", line 409, in _wrap_coverage
    return runner(*args)
  File "/home/noah/.cache/pants/named_caches/pex_root/unzipped_pexes/aecac44135d419f932725eb33af2df4856517e99/.bootstrap/pex/pex.py", line 440, in _wrap_profiling
    return runner(*args)
  File "/home/noah/.cache/pants/named_caches/pex_root/unzipped_pexes/aecac44135d419f932725eb33af2df4856517e99/.bootstrap/pex/pex.py", line 560, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/home/noah/.cache/pants/named_caches/pex_root/unzipped_pexes/aecac44135d419f932725eb33af2df4856517e99/.bootstrap/pex/pex.py", line 696, in execute_entry
    return self.execute_pkg_resources(entry_point)
  File "/home/noah/.cache/pants/named_caches/pex_root/unzipped_pexes/aecac44135d419f932725eb33af2df4856517e99/.bootstrap/pex/pex.py", line 728, in execute_pkg_resources
    return runner()
  File "/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/fae568165f31fc90f0e75717c4271e0cfdd33bef60060cee0ed9d51279de33a0/pex-2.1.79-py2.py3-none-any.whl/pex/bin/pex.py", line 740, in main
    do_main(
  File "/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/fae568165f31fc90f0e75717c4271e0cfdd33bef60060cee0ed9d51279de33a0/pex-2.1.79-py2.py3-none-any.whl/pex/bin/pex.py", line 761, in do_main
    pex_builder = build_pex(
  File "/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/fae568165f31fc90f0e75717c4271e0cfdd33bef60060cee0ed9d51279de33a0/pex-2.1.79-py2.py3-none-any.whl/pex/bin/pex.py", line 686, in build_pex
    pex_builder.set_script(options.script)
  File "/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/fae568165f31fc90f0e75717c4271e0cfdd33bef60060cee0ed9d51279de33a0/pex-2.1.79-py2.py3-none-any.whl/pex/pex_builder.py", line 348, in set_script
    distributions.update(PEX(pex, interpreter=self._interpreter).resolve())
  File "/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/fae568165f31fc90f0e75717c4271e0cfdd33bef60060cee0ed9d51279de33a0/pex-2.1.79-py2.py3-none-any.whl/pex/orderedset.py", line 45, in update
    for key in iterable:
  File "/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/fae568165f31fc90f0e75717c4271e0cfdd33bef60060cee0ed9d51279de33a0/pex-2.1.79-py2.py3-none-any.whl/pex/pex.py", line 121, in resolve
    for dist in env.resolve():
  File "/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/fae568165f31fc90f0e75717c4271e0cfdd33bef60060cee0ed9d51279de33a0/pex-2.1.79-py2.py3-none-any.whl/pex/environment.py", line 492, in resolve
    for fingerprinted_distribution in self.resolve_dists(all_reqs)
  File "/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/fae568165f31fc90f0e75717c4271e0cfdd33bef60060cee0ed9d51279de33a0/pex-2.1.79-py2.py3-none-any.whl/pex/environment.py", line 579, in resolve_dists
    raise ResolveError(
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/noah/.cache/pants/named_caches/pex_root/unzipped_pexes/b1f697e212f61b6154e26d9885061a3836dfb691.
Needed cp38-cp38-manylinux_2_31_x86_64 compatible dependencies for:
 1: Flask
    Required by:
      FingerprintedDistribution(distribution=<redacted>-3136395a3be17fe9145c4a8a3d3f2d9c82fa6349 (/home/noah/.cache/pants/named_caches/pex_root/installed_wheels/b7a06d2474b32d3876949dd74da83c93f060b4a4aaf5f4b434a17085a8b40344/<redacted>-3136395a3be17fe9145c4a8a3d3f2d9c82fa6349-py3-none-any.whl), fingerprint='8e7c49e6be260ac829d0c9058bf09b8d90039ba5b010b47d7d04623778ffaf25')
    But this pex had no 'Flask' distributions.
h

hundreds-father-404

04/20/2022, 7:37 PM
Does your constraints file definitely have flask?
lmk if you want to hop on a call
c

calm-ambulance-65371

04/20/2022, 7:38 PM
it's not in the constraints file at all
we only have versions that must be pinned in our constraints as it's company-wide in a monorepo
h

hundreds-father-404

04/20/2022, 7:38 PM
ohhhhhhhhhhh, one sec
c

calm-ambulance-65371

04/20/2022, 7:39 PM
it is pinned to a version in the respective requirements.txt
h

hundreds-father-404

04/20/2022, 7:39 PM
set this to false. It should not be necessary to set that, but I want to see if it fixes it https://www.pantsbuild.org/docs/reference-python#section-resolve-all-constraints
c

calm-ambulance-65371

04/20/2022, 7:40 PM
we already had that set actually
🤔 1
h

hundreds-father-404

04/20/2022, 7:43 PM
This is 2.11.0rc3 right? Maybe try 2.10.1rc0 -- your
python_requirement
changes should still work We made some changes in 2.11 so that lock files generated by pex are much more efficient. Perhaps we broke a code path
👍 1
your python_requirement changes should still work
you may need to explicitly set
[GLOBAL].use_deprecated_python_macros = false
c

calm-ambulance-65371

04/20/2022, 7:56 PM
same issue (I had to do a bunch of other stuff to get 2.10 to work)
😔 1
Will try to get more info
where is the
FingerprintedDistribution
defined?
I'm not finding it in the pants codebase
h

hundreds-father-404

04/20/2022, 9:16 PM
I think that that is pex code
👍 1
bump that using
-ldebug
may be useful: make sure pants is correctly telling Pex to install things. you may need to use
--no-pantsd --no-local-cache
c

calm-ambulance-65371

04/20/2022, 9:17 PM
I confirmed that tests are actually running, it appears to fail at the end. Possibly when trying to do coverage?
👀 1
h

hundreds-father-404

04/20/2022, 9:18 PM
Strange. Could you try turning off coverage to confirm that?
c

calm-ambulance-65371

04/20/2022, 9:21 PM
Did not effect it. The debug log seems to indicate the
test
goal is completing:
Copy code
16:20:23.03 [DEBUG] Completed: `test` goal
16:20:23.03 [DEBUG] Completed: Run Pytest
16:20:23.03 [DEBUG] Canceled: Run Pytest
16:20:23.03 [DEBUG] computed 1 nodes in 41.582558 seconds. there are 16938 total nodes.
16:20:23.03 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building pytest_runner.pex' failed with exit code 1.
stdout:
h

hundreds-father-404

04/20/2022, 9:22 PM
Are you running test on a single file or multiple?
c

calm-ambulance-65371

04/20/2022, 9:22 PM
we have a bunch of modules, I'm running something like
./pants test module::
(excluded the flags)
I'm seeing failing tests for expected reasons, so it's not that it can't run anything
oh weird, the generated .whl files don't appear to honor the
constraints.txt
(unzipped and looking at the METADATA file)
h

hundreds-father-404

04/20/2022, 9:24 PM
It might be helpful if you can make that more precise to figure out the exact file that is failing. If you have the option
--print-stacktrace
set, it should show you a list of rules with the file name in parentheses Note that pants will build multiple pytest_runner.py files. It sounds like some tests work and some down
7 Views