pantsbuild #general

Channels

welcome

github-notifications

announce

high-yak-85899

05/11/2022, 4:58 PM

Trying to upgrade to 2.11. I am able to run

./pants export ::

fine locally, but my CI system complained when it came time to clone a vcs requirement with "Permission denied" errors for public keys. I know these CI systems have access since they're cloning down our repos to get to this step. Any ideas of things to try? We already have the following in our

pants.toml

Copy code

[subprocess-environment]
env_vars.add = ["SSH_AUTH_SOCK"]

full-ocean-15873

05/11/2022, 5:32 PM

I am seeing an issue with the lockfiles generates by pex when we have an extra index url with credentials added. The lockfile that gets generates strips the credentials out. So when you try to run test cases, it tries to install the packages from the base url of the repo which causes a 401 unauthorized error. I think pants needs to store these credentials else where or make sure to use them from the index url when downloading them. Does anyone know if there is a workaround in the meantime?

✅ 1

wide-midnight-78598

05/11/2022, 10:09 PM

Any thoughts on why I would be getting

symbol not found

error when using the

RuleRunner

? I ran into this in the past and just kinda assumed it wasn't a big deal, but on a different computer, same error. Even more specifically, why is my XCode Python3.8 library used in here, when I'm pointing my environment (and everything else) at homebrew's 3.9?

Copy code

pants % ./pants test src/python/pants/backend/python/lint/black/rules_integration_test.py
18:00:56.67 [ERROR] Completed: Run Pytest - src/python/pants/backend/python/lint/black/rules_integration_test.py:rules_integration_test failed (exit code 2).
============================= test session starts ==============================
collected 0 items / 1 error

==================================== ERRORS ====================================
_ ERROR collecting src/python/pants/backend/python/lint/black/rules_integration_test.py _
ImportError while importing test module '/private/var/folders/2y/y_j1jxgn3tj5rb67_gjb7v700000gn/T/process-executionpSyKcR/src/python/pants/backend/python/lint/black/rules_integration_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
src/python/pants/backend/python/lint/black/rules_integration_test.py:10: in <module>
    from pants.backend.python import target_types_rules
src/python/pants/backend/python/target_types_rules.py:18: in <module>
    from pants.backend.python.dependency_inference.module_mapper import (
src/python/pants/backend/python/dependency_inference/module_mapper.py:21: in <module>
    from pants.backend.python.subsystems.setup import PythonSetup
src/python/pants/backend/python/subsystems/setup.py:19: in <module>
    from pants.option.subsystem import Subsystem
src/python/pants/option/subsystem.py:12: in <module>
    from pants.engine.internals.selectors import AwaitableConstraints, Get
src/python/pants/engine/internals/selectors.py:14: in <module>
    from pants.engine.internals.native_engine import (
E   ImportError: dlopen(/private/var/folders/2y/y_j1jxgn3tj5rb67_gjb7v700000gn/T/process-executionpSyKcR/src/python/pants/engine/internals/native_engine.so, 0x0002): symbol not found in flat namespace '_PyCMethod_New'
- generated xml file: /private/var/folders/2y/y_j1jxgn3tj5rb67_gjb7v700000gn/T/process-executionpSyKcR/src.python.pants.backend.python.lint.black.rules_integration_test.py.rules_integration_test.xml -
=========================== short test summary info ============================
ERROR src/python/pants/backend/python/lint/black/rules_integration_test.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.46s ===============================



✕ src/python/pants/backend/python/lint/black/rules_integration_test.py:rules_integration_test failed in 1.50s.

✅ 1

happy-kitchen-89482

05/12/2022, 5:16 AM

New feature I've been wanting to implement for a while but never got around to it: test sharding. https://github.com/pantsbuild/pants/pull/15417

🙌 2

ambitious-actor-36781

05/12/2022, 9:36 AM

pants pet peeve: so tedious setting up a new project. done three in the last month or two and every one feels like it's been pulling teeth to get things rolling.

👀 1

green-match-66337

05/12/2022, 11:43 AM

Hey! Having issues packaging with

aws-lambda-powertools

as a dependency for my lambda... Giving me this error

aws-xray-sdk 2.9.0 depends on future

green-match-66337

05/12/2022, 11:43 AM

Copy code

ProcessExecutionFailure: Process 'Building template_get/lambda.zip with 1 requirement: aws-lambda-powertools==1.25.10' failed with exit code 1.
stdout:

stderr:
ERROR: Cannot install aws-lambda-powertools because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit <https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies>
pid 30265 -> /Users/andy.challis/.cache/pants/named_caches/pex_root/venvs/4a4903f4a53b18a8968a4b2bac7896c88ae1b1da/cda4085b8dca33309a87eb474877cbd3a6659e25/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /Users/andy.challis/.cache/pants/named_caches/pex_root --log /private/var/folders/44/2rnr2jp97g1ddjr20xccsvv40000gn/T/process-executionYYi5SJ/.tmp/pex-pip-logfc_g6lfa/pip.log download --dest /private/var/folders/44/2rnr2jp97g1ddjr20xccsvv40000gn/T/process-executionYYi5SJ/.tmp/tmpjrsxhnm8/cp39-cp39-linux_x86_64 --only-binary :all: aws-lambda-powertools==1.25.10 --platform manylinux2014_x86_64 --platform linux_x86_64 --implementation cp --python-version 39 --abi cp39 --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
 
 The conflict is caused by:
     aws-xray-sdk 2.9.0 depends on future
     aws-xray-sdk 2.8.0 depends on future
 
 To fix this you could try to:
 1. loosen the range of package versions you've specified
 2. remove package versions to allow pip attempt to solve the dependency conflict




Use `--no-process-cleanup` to preserve process chroots for inspection.

cold-soccer-63228

05/12/2022, 1:00 PM

Is it on the roadmap to support running

vulture

from the

check

goal? I see that today, it's only

mypy

that is invoked. https://www.pantsbuild.org/docs/python-check-goal

✅ 1

cold-soccer-63228

05/12/2022, 1:20 PM

I had two separate questions about the generation of

BUILD

files for

python_tests

targets. See the code snippets below for reference. • Why is a

name

parameter generated at all? Why is this necessary? • Why is it that sometimes a

"tests0"

string is (rarely) used for a name, whereas other times

"tests"

is used?

Copy code

# path/to/some/BUILD
python_tests(
    name="tests0",
)

# path/to/another/BUILD
python_tests(
    name="tests",
)

✅ 1

cold-soccer-63228

05/12/2022, 2:20 PM

I see many of these logs when I run

./pants fmt ::

, without explicitly modifying files while it is running.

Copy code

10:19:26.61 [INFO] Filesystem changed during run: retrying `Fmt` in 500ms...

Is this to be expected as a result of the formatter modifying files?

✅ 1

calm-ambulance-65371

05/12/2022, 3:13 PM

Having some issues with

./pants package ::

. The module in question here packages fine when I run

./pants package module::

, so it's not the

.whl

or the package itself. Looks like it's also trying to print a binary blob, most likely the ziped up

.whl

brainy-solstice-27042

05/12/2022, 5:46 PM

Is there a way to limit the number of cores that the v2 engine will run on? I looked in the docs and didn't see any mentions of methods to specify some value other than "run on every core".

✅ 1

cold-soccer-63228

05/12/2022, 6:55 PM

Is there a way to generate

BUILD

files only in semi-top level folders, rather than in every directory? Something like the following?

Copy code

my_project/
├── awesome_service/
│   ├── internals/
│   |   ├── utils.py
│   |   ├── constants.py
│   ├── client/
│   |   ├── client.py
│   └── BUILD
└── another_awesome_service/
    ├── internals/
    |   ├── utils.py
    |   ├── constants.py
    ├── client/
    |   ├── client.py
    └── BUILD

loud-laptop-17949

05/12/2022, 9:04 PM

Is there a way to turn a python_dependency into a pex? I have a 3rdparty dependency (semgrep) that I want to run from a shell script. It would be awesome if I could tell pants to generate a pex for me.

✅ 1

cold-soccer-63228

05/13/2022, 3:58 AM

Hello, I'm seeing an error related to GraphQL when running

./pants test ::

. It seems to come from the ariadne package, and looks as follows.

Copy code

E   graphql.error.syntax_error.GraphQLSyntaxError: Syntax Error: Unexpected <EOF>
E
E   GraphQL request:238:1
E   237 |
E   238 |
E       | ^
E   239 |

I've pasted the stack trace in the thread to avoid myself from adding a super long message into the channel. I initially suspected this may be some issue related to ariadne not specifying their dependencies correctly. Previously, I've added to the

overrides

field of the

pipenv_requirements

target in the root

BUILD

file in order to resolve those types of issues, e.g.

Copy code

pipenv_requirements(
    ...
    overrides = {
        "google-cloud-pubsub": { "dependencies": [":requirements#setuptools"] },
    }
)

However, the error message is not easy for me to understand in this case, and I can't quite tell what kind of error this is, and whether it is due to the same root cause. I'm wondering if the Pants engineering team has any tips on how I can diagnose what is going wrong here and how I could go about trying to resolve this. I've already run the following, and see that ariadne is at the very least correctly listed as a dependency of my target.

Copy code

➜  ./pants dependencies front_porch/modules/graphql_views/executable_schema.py
//:requirements#ariadne
...

adorable-caravan-6626

05/13/2022, 5:15 AM

hi, i am facing issue in terraform module

Copy code

14:14:18.56 [ERROR] Completed: pants.backend.terraform.goals.check.terraform_check - terraform-validate failed (exit code 1).
╷
│ Error: Could not load plugin
│ 
│ 
│ Plugin reinitialization required. Please run "terraform init".
│ 
│ Plugins are external binaries that Terraform uses to access and manipulate
│ resources. The configuration provided requires plugins which can't be
│ located,
│ don't satisfy the version constraints, or are otherwise incompatible.
│ 
│ Terraform automatically discovers provider requirements from your
│ configuration, including providers used in child modules. To see the
│ requirements and constraints, run "terraform providers".
│ 
│ failed to instantiate provider "<http://registry.terraform.io/hashicorp/google|registry.terraform.io/hashicorp/google>" to
│ obtain schema: unknown provider "<http://registry.terraform.io/hashicorp/google|registry.terraform.io/hashicorp/google>"

rhythmic-morning-87313

05/13/2022, 7:22 AM

Q: Can I export the venv including all resolve targets? (e.g., python-default + mypy + flake8)

rhythmic-morning-87313

05/13/2022, 7:23 AM

I'm experiencing slower mypy execution (with over 400 files) and want to run it using the exported venv. There is an issue to improve mypy performance (https://github.com/pantsbuild/pants/issues/10864) but it seems to take a while to be resolved.

rhythmic-morning-87313

05/13/2022, 7:24 AM

Regarding pantsbuild/pants#10864, there should be a better way to batch like

xargs

once the dependencies are resolved and caches are verified

green-match-66337

05/13/2022, 7:47 AM

How does pants handle dynamic imports with dependency inference? For example if we use a typing library which is bloated but we don't want that in the package for lambda deployment to minimise artefact size?

purple-umbrella-89891

05/13/2022, 8:03 AM

Hi, is there a recipe / best practice for generating Sphinx documentation including apidoc with pants? Outside of pants, I would have to create a venv with Sphinx and run

sphinx-build

and

sphinx-apidoc

within it. Can this be written as a pants target?

👀 1

bitter-ability-32190

05/13/2022, 3:27 PM

Any initial thoughts on how Pants could support pytest plugins which generate files (snapshots)? Namely thinking of `pytest-recording`https://pypi.org/project/pytest-recording/ or the more general https://pypi.org/project/pytest-snapshot/ When configured, it generates snapshot files to be checked into the repo for subsequent run mocking

high-yak-85899

05/13/2022, 9:34 PM

I think I'm running into some confusing behavior with

paths

. I have two dockerfiles with pex binaries that I know are independent of one another. But somehow

paths

is saying that they are dependent. Here's a quick sketch of what I have

Copy code

> src
  > foo
    A.py
    B.py
    Dockerfile
  > bar
    C.py
    D.py
    Dockerfile

A.py

depends on

B.py

and goes into

foo/Dockerfile

as a pex.

C.py

depends on

D.py

and

B.py

and goes into

bar/Dockerfile

. When I do

paths

from

bar/Dockerfile

A.py

, I would expect no link, but somehow I am getting one through the Dockerfiles. Do target tags mess with the dependency tree at all?

hundreds-father-404

05/13/2022, 10:33 PM

if anyone is using

[mypy].extra_type_stubs

, feedback appreciated on https://github.com/pantsbuild/pants/issues/15455

hundreds-father-404

05/13/2022, 11:42 PM

I started adding Sphinx support 🙂 Thoughts on this target modeling?

✅ 1

💯 1

🙌 5

acceptable-night-23367

05/14/2022, 9:30 PM

I'm trying pants for the first time, on a pretty simple flask app (just one

app.py

module, plus some static assets), I've gone through the getting started docs, but can't get it to run the app. It looks like it's starting the initial process correctly, flask (werkzeug) comes up and says it's going to listen on whatever port, and tries to start the child serving process, but that one fails to import flask. It seems like the child process created by werkzeug isn't running with the same environment the parent process is using?

happy-kitchen-89482

05/14/2022, 10:06 PM

Sharding the big CI job was such a win! https://github.com/pantsbuild/pants/pull/15471

🙌 3

🎉 3

tall-jelly-41236

05/15/2022, 4:01 AM

Hello, I am new to Pants and converting my nox project to Pant 🙂 , I am facing a problem when running

pytest-bdd

, My feature files are not accessible, and I have tried as

python_test_utils

and

files

and included them in dependencies for python_test, still no luck,. I am facing file not found issue for

/private/tmp/.tmpBDSdib/tests/bdd/features/integration_instance.feature

I am sure its very small thing I am missing, hope some one can help me here.

✅ 1

rhythmic-morning-87313

05/15/2022, 4:37 AM

Q. I'd like to refer a file in the build root in the test cases executed via

python_tests()

target. It seems that

%(buildroot)s

expansion is not working in BUILD files. How could I achieve a similar thing? e.g.,

Copy code

python_tests(
    name="tests",
    extra_env_vars=[
        "BACKEND_CONFIG_FILE=%(buildroot)s/manager.toml",
    ],
)

rhythmic-morning-87313

05/15/2022, 4:45 AM

Q. How to execute an individual test case when using

python_tests()

with pytest? (i.e., select specific test(s) to run)

✅ 1