pantsbuild #general

Channels

welcome

github-notifications

announce

cold-breakfast-93567

06/08/2021, 6:24 PM

Isn't this the right one?

cold-breakfast-93567

06/08/2021, 6:24 PM

How could I further debug this?

proud-dentist-22844

06/08/2021, 9:51 PM

OK. Next fun bit of BUILD file madness around awkward fixture usage. There are several files that glob over the fixtures like this: https://github.com/st2sandbox/st2/blob/pants/st2api/tests/unit/controllers/v1/test_pack_config_schema.py#L24-L26 That glob uses fixtures from this directory (specifically all of the packs that include a

config.schema.yaml

file): https://github.com/st2sandbox/st2/tree/pants/st2tests/st2tests/fixtures/packs For the tests that use individual fixtures, I can import from a python file in the pack's subdir to make pants include the fixture. Is there a way to have a generic target that depends on all the targets with a

*/config.schema.yaml

file (the globs vary slightly, but the one I linked to above is looking for fixture packs that include this file)? If so, I could probably have another subdir (maybe called

config_schema_glob

) that says it depends on all such targets. If not, any good ideas on how to make this pants-friendly?

user

06/09/2021, 8:26 PM

https://twitter.com/semaphoreci/status/1402683413732245504 Pants Build retweeted: Managing the build process for complex #Python apps is hard. @pantsbuild offers an easy-to-use experience that wor… https://twitter.com/i/web/status/1402683413732245504 Twitter

big-xylophone-43403

06/09/2021, 8:42 PM

Hi, I have a fmt/lint issue. Pants seems to ignore the isort

black

profile. Whether its in

.isort.cfg

pyproject.toml

. Also, what version of Pants support the

pyproject.toml

config for both the tools?

user

06/09/2021, 9:47 PM

https://twitter.com/pantsbuild/status/1402744068703145991 Curious about build systems for Python? Check out our Media page full of talks, tutorials, interviews, and more. https://t.co/uNhWRI3OeL For experience levels ranging from novice to advanced, we’ve got something intriguing for you! Twitter

big-fall-51153

06/09/2021, 9:54 PM

I'm hitting an issue which I suspect is related to a recent update of my local

pip

(details in thread)

white-address-70112

06/10/2021, 7:31 AM

Hi Experts, Actually i am very new to Pants.. I am facing issues running my pex file in another system . We are using the command (PEX_TOOLS=1 ./pants my_webapp:my_webapp) to package our python project to a pex file. We have developed our project using python 3.7 and i want to bundle and run it as a pex package in a system which has python 2.7 in it without any dependencies. But when i try to run this on the system i m getting the below error:

Copy code

[vagrant@localhost ~]$ ./my_webapp.pex 
/usr/bin/env: python3.9: No such file or directory
[vagrant@localhost ~]$ python --version
Python 2.7.5

So how do i ship venv with the project as a single PEX file, so that this can be executed in any system we want to without any python version requirement? Some expert opinion/advise is much appreciated!! TIA

damp-television-71348

06/10/2021, 3:48 PM

Greetings, I am working on a plugin to create Rust extensions for Python. I am running into the following error when I try to run

./pants test ::

Copy code

10:24:19.53 [ERROR] Please set RUST_BACKTRACE=1, re-run, and then file a bug at <https://github.com/pantsbuild/pants/issues>.
10:24:19.53 [INFO] Filesystem changed during run: retrying `@rule(pants.backend.python.goals.pytest_runner.run_python_test)` in 500ms...
10:24:20.04 [ERROR] panic at 'called `Result::unwrap()` on an `Err` value: "Field `argv` was not convertible to type alloc::vec::Vec<alloc::string::String>: PyErr { ptype: <class \'TypeError\'>, pvalue: Some(\'Expected type that converts to PyString but received BinaryPath\'), ptraceback: None }"', src/nodes.rs:305

witty-crayon-22786

06/10/2021, 4:03 PM

the panic is not great, but the error message shows what went wrong:

Copy code

Field `argv` was not convertible to type: Expected type that converts to PyString but received BinaryPath

witty-crayon-22786

06/10/2021, 4:03 PM

in one of the types that you are constructing, you’re passing a

BinaryPath

to something that expects a

String

witty-crayon-22786

06/10/2021, 4:04 PM

…the

argv

argument in fact

witty-crayon-22786

06/10/2021, 4:09 PM

awesome to hear that you’re working on a plugin! let us know if you have any other questions

👍 2

jolly-midnight-72759

06/10/2021, 4:12 PM

I've read over the documentation I could find on

./pants validate '**'

but I am not sure what it does. When I run it on my v2.2.3 repo I get a

KeyError: 'path_patterns'

error. I found it in

${GH:pantsbuild/pants}/build-support/pre-commit

script. What is it supposed to validate?

damp-television-71348

06/10/2021, 5:07 PM

Thanks, I was able to resolve that problem. I am using

Get(Process...

to build using cargo. I specified

Cargo.toml

as a target source, but I'm getting

Copy code

stderr:

damp-television-71348

06/10/2021, 5:08 PM

Copy code

error: could not find `Cargo.toml` in `/tmp/process-executionIVcwpV` or any parent directory`

witty-crayon-22786

06/10/2021, 5:10 PM

and you captured

Cargo.toml

in the

input_digest

to the

Process

witty-crayon-22786

06/10/2021, 5:11 PM

if you haven’t seen it already, https://www.pantsbuild.org/docs/rules-api-process#process explains the sandboxing a bit

witty-crayon-22786

06/10/2021, 5:11 PM

but all inputs to processes need to be captured using the filesystem APIs: https://www.pantsbuild.org/docs/rules-api-file-system

damp-television-71348

06/10/2021, 5:11 PM

Ah, okay. I'll give that a try

orange-beach-75711

06/10/2021, 5:31 PM

Has anyone seen this error in CI before? Our tests run just fine locally on our desktops but we get this error in CI on 2.5.1rc1 and 2.5.2rc2.

Copy code

Has anyone seen this error in CI before?

Traceback (most recent call last):
  File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.1rc2_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 234, in _run_inner
    return self._perform_run(goals)
  File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.1rc2_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 173, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.1rc2_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 195, in _perform_run_body
    poll_delay=(0.1 if poll else None),
  File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.1rc2_py37/lib/python3.7/site-packages/pants/init/engine_initializer.py", line 136, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.1rc2_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 530, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.1rc2_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 494, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),
pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:
Engine traceback:
  in select
  in pants.core.goals.test.run_tests
  in pants.core.goals.test.enrich_test_result (###)
  in pants.backend.python.goals.pytest_runner.run_python_test (###)
  in pants.backend.python.goals.pytest_runner.setup_pytest_for_target
  in pants.backend.python.util_rules.pex.create_pex
  in pants.backend.python.util_rules.pex.build_pex (requirements.pex)
  in pants.engine.process.fallible_to_exec_result_or_raise
Traceback (most recent call last):
  File "/cicache/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.1rc2_py37/lib/python3.7/site-packages/pants/engine/process.py", line 254, in fallible_to_exec_result_or_raise
    description.value,
pants.engine.process.ProcessExecutionFailure: Process 'Building requirements.pex with 20 requirements: google-api-core==1.29.0; (python_version >= "2.7" and python_full_version < "3.0.0") or python_full_version >= "3.6.0", google-cloud-storage==1.38.0; (python_version >= "2.7" and python_full_version < "3.0.0") or python_full_version >= "3.6.0", grpcio==1.38.0, kfp==1.6.0; python_full_version >= "3.5.3", opentelemetry-api==0.17b0; python_version >= "3.5", opentelemetry-exporter-opencensus==0.17b0; python_version >= "3.5", opentelemetry-exporter-prometheus==0.17b0; python_version >= "3.5", opentelemetry-instrumentation-grpc==0.17b0; python_version >= "3.5", opentelemetry-instrumentation-sqlalchemy==0.17b0; python_version >= "3.5", opentelemetry-instrumentation==0.17b0; python_version >= "3.5", opentelemetry-sdk==0.17b0; python_version >= "3.5", prometheus-client==0.10.1; (python_version >= "2.7" and python_full_version < "3.0.0") or python_full_version >= "3.4.0", protobuf==3.17.2, pytest==6.2.4; python_version >= "3.6", python-json-logger==0.1.11; python_version >= "2.7", pytorch-lightning==1.2.7; python_version >= "3.6", pyyaml==5.3.1; python_full_version >= "3.5.3" and python_version >= "3.6", retrying==1.3.3, setuptools==57.0.0, torch==1.8.1; python_full_version >= "3.6.2"' failed with exit code 1.
stdout:
stderr:
pid 378 -> /cicache/.cache/pants/named_caches/pex_root/venvs/648cade9a5d2ba49503a6052c1f39aef29a05513/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /cicache/.cache/pants/named_caches/pex_root --log /tmp/process-executionsnJ0oz/.tmp/tmpucx12map/pip.log download --dest /tmp/process-executionsnJ0oz/.tmp/tmp3u2ob2jw/usr.local.bin.python3.7 --constraint requirements.txt google-api-core==1.29.0; (python_version >= "2.7" and python_full_version < "3.0.0") or python_full_version >= "3.6.0" google-cloud-storage==1.38.0; (python_version >= "2.7" and python_full_version < "3.0.0") or python_full_version >= "3.6.0" grpcio==1.38.0 kfp==1.6.0; python_full_version >= "3.5.3" opentelemetry-api==0.17b0; python_version >= "3.5" opentelemetry-exporter-opencensus==0.17b0; python_version >= "3.5" opentelemetry-exporter-prometheus==0.17b0; python_version >= "3.5" opentelemetry-instrumentation-grpc==0.17b0; python_version >= "3.5" opentelemetry-instrumentation-sqlalchemy==0.17b0; python_version >= "3.5" opentelemetry-instrumentation==0.17b0; python_version >= "3.5" opentelemetry-sdk==0.17b0; python_version >= "3.5" prometheus-client==0.10.1; (python_version >= "2.7" and python_full_version < "3.0.0") or python_full_version >= "3.4.0" protobuf==3.17.2 pytest==6.2.4; python_version >= "3.6" python-json-logger==0.1.11; python_version >= "2.7" pytorch-lightning==1.2.7; python_version >= "3.6" pyyaml==5.3.1; python_full_version >= "3.5.3" and python_version >= "3.6" retrying==1.3.3 setuptools==57.0.0 torch==1.8.1; python_full_version >= "3.6.2" --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with -9 and STDERR:
None
FATAL: exception not rethrown
/bin/bash: line 191:    59 Aborted                 (core dumped) ./pants --print-stacktrace --tag="-integration_tests" test ::

user

06/10/2021, 5:56 PM

https://twitter.com/pantsbuild/status/1403048496404602884 On June 24-25, join us at @build_meetup conference online for two talks of talks and discussion panels about build systems. Registration is free https://t.co/MO9yfMLpuy. Pants Build long-time maintainer @stuhood presents “Faster Incremental Builds with Speculation” on June 25th! Twitter

calm-ambulance-65371

06/10/2021, 7:35 PM

Is there a way to get additional binaries available in PATH during execution? For the Rust plugin, Cargo depends on

cc

helpful-lunch-92084

06/10/2021, 7:43 PM

I’ve been testing out the upgrade from pants 2.2.3 to 2.3.2 which brings in venv mode test execution and I’m running into some failures. Specifically, it appears that some code using spacy.cli.link functionality causes spacy to write some symlinks inside it’s spacy/data directory. For non-venv stuff, this means it writes directly to a fingerprinted dir under

.cache/pex_root/installed_wheels/

However, with venv stuff, it appears that it pulls that cached wheel to a venv dir but the resulting symlinks are converted to empty directories in the spacy/data dir and then subsequent spacy.cli.link fails because it can’t overwrite a directory. 1. Is there a way to prevent a non-venv execution from writing those symlinks to the installed_wheels dir? E.g. the spacy package is pulled into a temp location from installed_wheels? 2. Is there a way to selectively disable venv test execution for specific targets. (It’s a less ideal situation but I was thinking about excluding those specific targets from venv execution mode to workaround this immediate problem) 3. Any other ideas?

witty-crayon-22786

06/10/2021, 7:46 PM

there is, yea. apologies: i’ll expand the docs in this area.

calm-ambulance-65371

06/10/2021, 7:47 PM

nvm we just added

env

to the

Process

calm-ambulance-65371

06/10/2021, 7:47 PM

used a

BinaryPath

to find the binary we needed, then added

env={'PATH': cc.first_path.path}

witty-crayon-22786

06/10/2021, 7:47 PM

when creating a

Process

, you can set its `env`: but to actually get the environment (which is filtered out to encourage sandboxing), you need to request it via the engine, so that when the environment changes, the the relevant

@rules

are invalidated

witty-crayon-22786

06/10/2021, 7:48 PM

ah, ok. yea, that’s even better!

calm-ambulance-65371

06/10/2021, 7:48 PM

yeah, I figured that would be better 🙂