While running in my case `pants package foo ` there s one pa Pants #general

While running, in my case, `pants package foo::`, ...

ancient-france-42909

06/22/2023, 5:45 PM

While running, in my case,

pants package foo::

, there's one particular target that has a problem, is there a sensible way to figure out which one it is, except seeing which completed and which was cancelled, and then trying to infer it from that?

ancient-france-42909

06/22/2023, 5:45 PM

This is the error:

Copy code

20:31:19.40 [INFO] Completed: Building kite.emr_jobs_pants.analyze_inbound_logs/analyze_inbound_logs.pex with 118 requirements: Flask==2.0.3, IP2Location==8.9.0, Jinja2==3.0.3, PyYAML==5.4.1, SQLAlchemy==1.4.47, Unidecode==1.0.22,... (2485 characters truncated)
20:31:19.45 [ERROR] 1 Exception encountered:

Engine traceback:
  in select
    ..
  in pants.core.goals.package.package_asset
    `package` goal

Traceback (most recent call last):
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/core/goals/package.py", line 159, in package_asset
    for field_set in target_roots_to_field_sets.field_sets
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 358, in MultiGet
    return await _MultiGet(tuple(__arg0))
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 165, in __await__
    result = yield self.gets
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/core/goals/package.py", line 109, in environment_aware_package
    BuiltPackage, {request.field_set: PackageFieldSet, environment_name: EnvironmentName}
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/backend/python/goals/package_pex_binary.py", line 202, in built_pacakge_for_pex_from_targets_request
    pex = await Get(Pex, PexFromTargetsRequest, pft_request)
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/backend/python/util_rules/pex_from_targets.py", line 512, in create_pex_from_targets
    sources=sources,
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/backend/python/util_rules/local_dists.py", line 189, in build_local_dists
    additional_args=["--intransitive"],
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/backend/python/util_rules/pex.py", line 800, in create_pex
    result = await Get(BuildPexResult, PexRequest, request)
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 623, in native_engine_generator_send
    res = rule.send(arg) if err is None else rule.throw(throw or err)
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/backend/python/util_rules/pex.py", line 730, in build_pex
    cache_scope=request.cache_scope,
  File "/home/cris.birzan/.cache/pants/setup/bootstrap-Linux-x86_64/2.16.0_py37/lib/python3.7/site-packages/pants/engine/internals/selectors.py", line 118, in __await__
    result = yield self
pants.engine.process.ProcessExecutionFailure: Process 'Building local_dists.pex' failed with exit code 102.
stdout:

stderr:
Could not find a compatible interpreter.

Examined the following interpreters:
1.)                 /home/cris.birzan/.pyenv/versions/3.7.4/bin/python3.7 CPython==3.7.4
2.) /home/cris.birzan/.pyenv/versions/3.7.4/envs/crpython37/bin/python3.7 CPython==3.7.4
3.)               /home/cris.birzan/.pyenv/versions/3.10.9/bin/python3.10 CPython==3.10.9
4.)               /home/cris.birzan/.pyenv/versions/3.11.1/bin/python3.11 CPython==3.11.1
5.)                /home/cris.birzan/.pyenv/versions/3.6.15/bin/python3.6 CPython==3.6.15
6.)                 /home/cris.birzan/.pyenv/versions/3.7.9/bin/python3.7 CPython==3.7.9
7.)                /home/cris.birzan/.pyenv/versions/3.8.16/bin/python3.8 CPython==3.8.16
8.)                 /home/cris.birzan/.pyenv/versions/3.9.5/bin/python3.9 CPython==3.9.5

No interpreter compatible with the requested constraints was found:

  Version matches CPython~=3.10.9,~=3.7.4



Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.


20:31:19.46 [INFO] Canceled: Building kite.emr_jobs_pants.defender_simulation/defender_simulation.pex with 109 requirements: Flask==2.0.3, IP2Location==8.9.0, Jinja2==3.0.3, SQLAlchemy==1.4.47, Unidecode==1.0.22, XlsxWriter==1.1.... (2316 characters truncated)

ancient-france-42909

06/22/2023, 5:46 PM

(I know what causes the error, I just don't know WHICH target causes it 🙂 )

curved-television-6568

06/22/2023, 6:04 PM

perhaps if you turn down https://www.pantsbuild.org/docs/reference-global#process_execution_local_parallelism to 1, and run with

-ldebug

you can spot which package is building when the error hits?

ancient-france-42909

06/22/2023, 6:06 PM

Yeah, that'll take a bit, though.

curved-television-6568

06/22/2023, 6:06 PM

is it that one of your targets have a stray IC?

curved-television-6568

06/22/2023, 6:07 PM

maybe easier to locate that with some

pants peek

and

jq

filtering?

ancient-france-42909

06/22/2023, 6:07 PM

So, one of the targets' deps (or their deps, or their deps deps...) has it

curved-television-6568

06/22/2023, 6:07 PM

or just

grep

ancient-france-42909

06/22/2023, 6:07 PM

Basically, we're in the process of upgrading to 3.10 and a few of our hundreds of targets hasn't been upgraded 🙂

ancient-france-42909

06/22/2023, 6:08 PM

grep definitely won't work, let me see what peek returns

ancient-france-42909

06/22/2023, 6:09 PM

Okay, peek is not very useful, since it only gives me the first level deps.

ancient-france-42909

06/22/2023, 6:10 PM

So I'd have to call it recursively

ancient-france-42909

06/22/2023, 6:10 PM

Oh, hm. I think I can use --changed-since

curved-television-6568

06/22/2023, 6:10 PM

peek takes multiple targets

ancient-france-42909

06/22/2023, 6:10 PM

and then pipe it through a custom script we have 😄

👍 1

curved-television-6568

06/22/2023, 6:11 PM

pants peek ::

will give you everything 😉

ancient-france-42909

06/22/2023, 6:12 PM

Copy code

⠐ 71.36s `peek` goal

🙂

ancient-france-42909

06/22/2023, 6:16 PM

So, 250 seconds, and:

Copy code

$ ./pants peek :: | wc
3093007 3323943 221648106

ancient-france-42909

06/22/2023, 6:16 PM

only 200 MB of stuff 😄

😅 1

ancient-france-42909

06/22/2023, 6:17 PM

Copy code

$ PANTS_PROCESS_EXECUTION_LOCAL_PARALLELISM=1 ./pants -ldebug  package kite/emr_jobs_pants:: 2>&1 | tee pants.out

I'll see what happened tomorrow 😄

ancient-france-42909

06/22/2023, 6:18 PM

if I have enough space on my drive

ancient-france-42909

06/22/2023, 6:30 PM

Hm, I think we might've configured something wrong here, these shouldn't try to build with 3.10. But, yeah, thanks, the parallelism 1 thing did help

🙌 1

billions-printer-97253

06/23/2023, 6:35 PM

git

2 Views

Open in Slack

Previous Next