pantsbuild #general

Channels

welcome

github-notifications

announce

white-pager-32496

12/20/2022, 11:25 PM

Out of (morbid?) curiosity - what consequences, if any, arise from disabling dependency inference other than losing the feature itself?

curved-manchester-66006

12/21/2022, 4:21 AM

For "fun" dependency resolution reasons (thanks for pipping

pip_version

thru!) this is on

2.16.0.dev3

so feel free to defer. My question is basically a gut check on "this looks like source roots or other common onboarding confusion" or "probably something with the dev release, check back in January". I'm getting variants of this error message:

Copy code

./pants run scripts/hello.py 
Traceback (most recent call last):
...
ImportError: No module named scripts.hello

🧵

✅ 1

average-sugar-68948

12/21/2022, 2:42 PM

Hi folks, I'm getting mypy errors when using protobufs. Anyone have any suggestions? I've enabled the mypy-plugin in the [python-protobuf] section, but still get:

Copy code

./pants check ::
09:37:48.34 [INFO] Completed: Building requirements.pex with 5 requirements: bitarray==2.5.1, numpy==1.23.4, protobuf==4.21.12, simpy==4.0.1, types-protobuf==4.21.0.2
09:37:48.75 [INFO] Completed: Building requirements_venv.pex
09:37:49.18 [ERROR] Completed: Typecheck using MyPy - mypy failed (exit code 1).
fabric/proto/v1/workload_pb2.pyi:6: error: Library stubs not installed for "google.protobuf.descriptor" (or incompatible with Python 3.7)
fabric/proto/v1/workload_pb2.pyi:6: note: Hint: "python3 -m pip install types-protobuf"
fabric/proto/v1/workload_pb2.pyi:6: note: (or run "mypy --install-types" to install all missing stub packages)
...

gray-shoe-19951

12/21/2022, 3:59 PM

Another quick question on cache, as of now, what is the best way to "clean" the cache? I guess there is no tool to manage cache at the moment.

famous-kilobyte-26155

12/21/2022, 4:13 PM

I’m working on a monorepo consisting of code for several flywheel.io “gears”. A gear is basically a Docker image with a particular directory structure and a manifest.json. I’ve got the project configured so that pants will build the docker image. But flywheel has its own internal container registry and the push is handled by a command-line script that must be run in the directory with the manifest. I want to make it so that publishing the gear image uses the flywheel script instead of docker push. Any suggestions on a quick and dirty strategy to make this happen?

tall-plastic-15186

12/21/2022, 5:51 PM

Hi folks, encountering an issue with mypy with django stubs. So

./pants check ::

works as expected and we're able to see the typing errors across all files, however, if we run

./pants check <a specific file>

it's not able to find the settings module:

Copy code

Error constructing plugin instance of NewSemanalDjangoPlugin


Traceback (most recent call last):
  File "/tmp/pants-sandbox-sx8jrg/.cache/pex_root/venvs/452b1bd48fc9bdd09defe81a4d4ed2dfe7aa21c1/acab3aafcd0b37dc1f526605bd765ad366d418c4/pex", line 246, in <module>
    sys.exit(func())
  File "/home/irsoc/.cache/pants/named_caches/pex_root/venvs/s/f8ab7191/venv/lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "mypy/main.py", line 95, in main
  File "mypy/main.py", line 174, in run_build
  File "mypy/build.py", line 193, in build
  File "mypy/build.py", line 248, in _build
  File "mypy/build.py", line 507, in load_plugins
  File "mypy/build.py", line 488, in load_plugins_from_config
  File "/home/irsoc/.cache/pants/named_caches/pex_root/venvs/s/f8ab7191/venv/lib/python3.10/site-packages/mypy_django_plugin/main.py", line 73, in __init__
    self.django_context = DjangoContext(self.plugin_config.django_settings_module)
  File "/home/irsoc/.cache/pants/named_caches/pex_root/venvs/s/f8ab7191/venv/lib/python3.10/site-packages/mypy_django_plugin/django/context.py", line 83, in __init__
    apps, settings = initialize_django(self.django_settings_module)
  File "/home/irsoc/.cache/pants/named_caches/pex_root/venvs/s/f8ab7191/venv/lib/python3.10/site-packages/mypy_django_plugin/django/context.py", line 65, in initialize_django
    settings._setup()  # type: ignore
  File "/home/irsoc/.cache/pants/named_caches/pex_root/venvs/s/f8ab7191/venv/lib/python3.10/site-packages/django/conf/__init__.py", line 79, in _setup
    self._wrapped = Settings(settings_module)
  File "/home/irsoc/.cache/pants/named_caches/pex_root/venvs/s/f8ab7191/venv/lib/python3.10/site-packages/django/conf/__init__.py", line 190, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/home/irsoc/.pyenv/versions/3.10.9/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mymodule'

glamorous-nail-59544

12/21/2022, 7:35 PM

Hey folks, I am moving multiple django microservices into a pants monorepo, so far I managed to get

runserver

working. Next is to get my tests working. So, any chance of getting my existing `APITestCase`s to run without a change? Running

./pants run src/python/worker/manage.py -- test

results in 0 discovered tests. If I specify the tests target

./pants run src/python/worker/manage.py -- test src/python/worker/app/tests

I get

No module named 'worker.app.tests'

I checked both the django sample and the django webinar, but unfortunately this project is running using pytest, and my microservices are not. nevertheless I tried running

./pants test src/python/worker/app/tests/test_api.py

but I think it won't work out of the box, I got the following:

Copy code

django.core.exceptions.ImproperlyConfigured: Requested setting REST_FRAMEWORK, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

What is is the shortest path to get my tests running?

aloof-appointment-30987

12/21/2022, 10:31 PM

How does one save test artifacts outside of the sandbox so they can be uploaded to CI (Github)? I'm trying to troubleshoot a tricky test failure. The test passes when executed locally whether via

pytest

pants test

as well as inside docker using

nektos/act

. It succeeds when run on Github CI (cloud) if via

pytest

but it fails when run via

pants test

on GitHub CI (cloud). I'm trying to save the output of the function under test to a file so I can inspect what is being produced. The file is saved inside the sandbox but I can't seem to find documentation explaining how to make that available to the CI host for upload.

rich-kite-32423

12/22/2022, 3:54 AM

I've separated out my instructions for setting up Windows Subsystem for Linux (WSL) to run Pants: https://github.com/BruceEckel/ConfiguringForPants

👍 2

🙏 2

lively-gpu-26436

12/22/2022, 9:49 AM

Hi all, I have a dependency in my requirements.txt file like this

<git+ssh://git@github.com/rollbar/sqlalchemy.git@bc5c5b21f956a953a8b68523b3092487e7e63bea#egg=sqlalchemy>

and Pants is saying it’s not able to infer the dependency owner for

sqlalchemy

. Do you know how could I fix? 🙏

✅ 1

chilly-holiday-77415

12/22/2022, 12:32 PM

I think I’m having a module mapping and transitive dependency issue: in a standard .venv, when I do

pip install aws-cdk-lib

, it pulls in

aws_cdk

, as well as some transitive dependencies that are submodules -

aws_cdk.asset_awscli_v1

for example - for clarity, as

.venv/lib/python3.9/site-packages/aws_cdk/asset_awscli_v1

. In my

poetry_requirements(name="poetry", module_mapping={"aws-cdk-lib": ["aws_cdk"]})

top level build file I map the different name In my cdk project I declare the transitive deps not being inferred

Copy code

python_sources(
    name="cdklib",
    dependencies=[
        "//:poetry0#aws-cdk-asset-awscli-v1",
        "//:poetry0#jsii",
    ],
)

pex_binary(
    name="cdk_infra",
    entry_point="app.py",
)

and this gets me really close, but still fails with an import error - if I unzip the pex, I can see it’s pulling

aws-cdk-asset-awscli-v1

separately into it’s own

aws_cdk/assert_awscli_v1

, rather than into the initial

aws_cdk

install which is what I think I need. Hopefully that makes sense - I can cut an issue if that’s helpful! I tried mapping

“aws-cdk-asset-awscli-v1”: [“aws_cdk.asset_awscli_v1"]

but that didn’t seem to help.

cold-branch-54016

12/22/2022, 1:10 PM

Does pants already support cross building pex binaries in a container via the new environments feature in the latest dev build? My project depends on a couple of packages that are only available as

sdists

but I need to build pex binaries for mac and linux. I am currently running on

"2.16.0.dev3"

and tried to get it working but it still complains that it can’t find a wheel.

modern-monkey-78364

12/22/2022, 1:45 PM

Hi, I have kind of general question. I remember a few years earlier we had to explicitly mention the dependencies in BUILD files, which changed to pants automatically inferring them from the code/imports and we had kind of empty BUILD files with just

python_sources()

. Now with https://www.pantsbuild.org/docs/reference-python-infer#assets which is false by default, we expect dependencies to mentioned explicitly. Is there a reason why assets is false by default, rather than true?

able-advantage-61346

12/22/2022, 2:04 PM

I have a few questions that have come up after a week of getting to know Pants: • The virtual environment created by

./pants export::

does not include pytest. Because of that PyCharm's builtin test support won't work. I can add pytest to the list of 3rd party dependencies, but I am not sure this is how it's intended? I also get venv's for each tool, but those miss my regular dependencies. • The generated lockfile (based on
requirements.txt
) only pins version of the listed packages. If I want to lock the version of any downstream dependencies, should I instead use Pipenv or Poetry? (not true) In any case, I appreciate the speedy help that I have gotten here and the thorough documentation available 👍

👀 1

wide-midnight-78598

12/22/2022, 2:11 PM

Not sure where this fits on the Readme.com vs Sphinx docs roadmap (thus, not sure if possible today) - but one small quality-of-life improvement I think is to add a search bar to the homepage when you land on

<http://pantsbuild.org|pantsbuild.org>

next to "Getting Started" Not speaking for anyone else, but the 99% of the time I go back to the home page, it's to look at the docs, and probably to search for something. Tailwind does this well I think (I mean, they do basically everything well, honestly): https://tailwindcss.com/

👍🏽 1

proud-dentist-22844

12/22/2022, 3:44 PM

I'm adjusting

interpreter_constraints

. The key piece I'm questioning is in this BUILD file: https://github.com/StackStorm/st2/pull/5850/files#diff-a1de578ad6ec792e46e84af2ab04188980ed060c25f673c018cf0cf4bd142d93 I had interpreter constraints in

pants.toml

for the

pants-plugins

resolve without applying them here. I assumed that the

[python.resolves_to_interpreter_constraints].pants-plugins

would automatically be applied to anything that used that resolve. But, the default comes from

[python].interpreter_constraints

instead, which is not compatible with that resolve. Was my assumption off base? Why do I need to duplicate the constraints in both places? Could we change how default

interpreter_constraints

get calculated so it uses

[python.resolves_to_interpreter_constraints].<resolve>

first before falling back to

[python].interpreter_constraints

? Is there a feature here that I'm misunderstanding?

plain-orange-45099

12/22/2022, 4:41 PM

Hey all 👋 I'm a recent

pants

convert 😅 Looking at taking my existing

pylint

flow over to

pants

, and I'm getting stuck with my

[SPELLING]

configuration. Currently, running

./pants lint ::

will error with

optparse.OptionValueError: option spelling-dict: invalid value: 'en_US', should be in ['']

despite having the

enchant

library installed through

brew

👖 1

curved-microphone-39455

12/22/2022, 6:27 PM

Hello There! (insert Obi-Wan gif here) When activating the

coverage-py

in my in my

.toml

for me pipeline, it always fail because of this error

Copy code

17:20:21.82 [ERROR] 1 Exception encountered:
  ProcessExecutionFailure: Process 'Generate Pytest html coverage report.' failed with exit code 1.
stdout:
Couldn't parse '/tmp/pants-sandbox-wxTI0G/src/libraries/lib/lib/v2/models/_potential_access_points/_deployment_provider_type.py' as Python source: 'invalid syntax' at line 25

Copy code

[test]
report = true
use_coverage = true

[coverage-py]
report = ["html"]
global_report = true

We are using Python 3.10 at this is a Switch statement at this line, do you have an idea to my coverage work with this ?

glamorous-nail-59544

12/22/2022, 7:21 PM

Hey again, I enabled

[python-infer].assets = true

and have an assets dir located in

src/python/service/assets

I am trying to access these assets from python using the path

service/assets/file_name.extension

but getting not such file or directory. So I am wondering, • am I accessing the files correctly using module instead of root? • do I need to create BUILD file inside the assets dir? • if yes, can I define sources using asterisk symbol

**/*.extension

to void adding new source for every file.

aloof-appointment-30987

12/22/2022, 8:56 PM

I've encountered a head scratcher. I have an example repo where I am able to debug without issue, in visual studio, using

./pants test --debug-adapter path/to/target:test

and a remote attach configuration in VSCode. Break points are hit, code is mapped properly. In our company repo, I have integrated pants and tried to use exactly the same configuration but I encounter an error. Any insight as to what is missing? This is the console log. After

Launching debug adapter at '127.0.0.1:5678', which will wait for a client connection...

I execute the vsCode remote attach configuration and receive a

KeyError: 'console_scripts'

Copy code

> ./pants test --debug-adapter realtime/tests/test_eeg_processor.py

15:25:59.09 [INFO] Initializing scheduler...
15:25:59.37 [INFO] Scheduler initialized.
15:26:02.63 [INFO] Launching debug adapter at '127.0.0.1:5678', which will wait for a client connection...
Traceback (most recent call last):
  File "/Users/russellzarse/.cache/pants/named_caches/pex_root/venvs/s/3784c3f3/venv/lib/python3.8/site-packages/importlib_metadata/__init__.py", line 288, in __getitem__
    return next(iter(self.select(name=name)))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/var/folders/_d/zt8y2x055597l63bhdztp82c0000gn/T/pants-sandbox-pE0mN3/./.cache/pex_root/venvs/6de3f2e95f7514eb53f424a095262c59a1b89864/aa2e58b40d603796246ca3ee285dff9c19064a8b/pex", line 236, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/Users/russellzarse/.pyenv/versions/3.8.13/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/Users/russellzarse/.pyenv/versions/3.8.13/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Users/russellzarse/.pyenv/versions/3.8.13/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/russellzarse/.cache/pants/named_caches/pex_root/venvs/s/3784c3f3/venv/lib/python3.8/site-packages/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/Users/russellzarse/.cache/pants/named_caches/pex_root/venvs/s/3784c3f3/venv/lib/python3.8/site-packages/debugpy/server/cli.py", line 444, in main
    run()
  File "/Users/russellzarse/.cache/pants/named_caches/pex_root/venvs/s/3784c3f3/venv/lib/python3.8/site-packages/debugpy/server/cli.py", line 344, in run_code
    eval(code, {})
  File "<string>", line 1, in <module>
  File "/Users/russellzarse/.cache/pants/named_caches/pex_root/venvs/s/3784c3f3/venv/lib/python3.8/site-packages/importlib_metadata/__init__.py", line 290, in __getitem__
    raise KeyError(name)
KeyError: 'console_scripts'

I have to run an errand so I'll be out for about an hour. Thanks in advance for your help!

powerful-florist-1807

12/22/2022, 9:33 PM

Hi Pants experts, one user in my company reported the following error when he runs the pants test command (

./pants -ldebug --python-requirement-constraints=airflow2/src/python/airflow_dags/data/constraints.txt test ...

raise ResolveError(message)

pex.environment.ResolveError: A distribution for python-levenshtein could not be resolved for /Users/karthikganapathi/.pyenv/versions/3.7.10/bin/python3.7.

Found 1 distribution for python-levenshtein that do not apply:

1.) The wheel tags for python-Levenshtein 0.12.0 are cp37-cp37m-macosx_12_0_x86_64 which do not match the supported tags of /Users/karthikganapathi/.pyenv/versions/3.7.10/bin/python3.7:

cp37-cp37m-macosx_10_16_x86_64

... 1334 more ...

Does anyone know what is the meaning the error message? Thanks!

gentle-painting-24549

12/22/2022, 9:57 PM

I’m running into an issue where having multiple files @

<pants_respective_source>/tests/conftest.py

prevents MyPy checks from running - I’m wondering if this is a pants bug / mypy issue / user error. More details and git example in 🧵

Copy code

packages/package-2/tests/conftest.py: error: Duplicate module named "conftest" (also at "packages/package-1/tests/conftest.py")

ambitious-actor-36781

12/23/2022, 12:42 AM

is there a quicker way to do

Copy code

./pants dependees --dependees-transitive //my/target | xargs ./pants test

👍 1

wide-midnight-78598

12/23/2022, 2:17 PM

Does the Python backend have a docs generator? I've seen this question floating around a bit - never saw the conclusion

adventurous-journalist-72920

12/23/2022, 2:26 PM

Hey all! Is there a way to pass a buf config file to buf linter (backend → pants.backend.codegen.protobuf.lint.buf) ? I’ve been trying to set:

Copy code

[buf]
lint_args = [
  "--config buf.yaml"
]

and tried several paths, raw or with targets, without any success. It feels unlikely to work as protos are likely to be declared with

protobuf_sources

that does not accept any other file types than

.proto

. A workaround would be to pass the yaml (or json) content of the config file directly in the config option, but it’s not really maintainable as the config grows. Any idea?

loud-laptop-89838

12/23/2022, 5:20 PM

I'm curious, for those who have more monorepo experience than me, do you see something like this fitting in well? Either in a pants monorepo or just generally? Where pants figures out transitive dependencies, does dependency management become an issue?

abundant-autumn-67998

12/23/2022, 9:04 PM

We're building PEXes with one linux image (the github worfklow's

ubuntu-latest

) and running them in another (

python:3.X-slim

). We use these build flags: •

--platform=manylinux2014_x86_64-cp-{version_tag}-cp{version_tag}

where

version_tag

is something like

"38"

•

--resolve-local-platforms

Binary wheels work great which is expected. Since we

--resolve-local-platforms

and we also ensure the target Python interpreter version is present in the builder-image, those have also worked.. so far. I recently saw a problem with one sdist dependency where the PEX was build successfully, but didn't run and failed with:

Copy code

Needed cp310-cp310-manylinux_2_31_x86_64 compatible dependencies for:
...
1: pendulum

The PEX included the

manylinux_2_35

version instead of the

manylinux_2_31

version for this dependency. I suspect this has to do with the github upgrading their

ubuntu-latest

to from 20.04 to 22.04 (see compatibility). Does my analysis seem correct so far? I'm looking for a solution that lets us reliably build sdists: 1. pin the builder image to ubuntu 20.04. we then rely on github's support for this image 2. run our own docker image during the build - is slow and looking to avoid this option 3. build a backwards compatible sdist. is it even possible to build a

manylinux_2_31

wheel on ubuntu 22.04? 4. any other options?

rich-kite-32423

12/25/2022, 1:22 AM

I've been setting up a starter example, taking things in small steps, and I seem to have discovered a problem (this is all under WSL). The repo is here: https://github.com/BruceEckel/ConfiguringForPants The READMEs in each directory describe the steps that I took. The problem occurs in step2, where even though pants appears to be working,

./pants test ::

produces no results and

./pants run main/main.py

produces errors. Now the mysterious part: after copying the step2 directory to step3, making no other changes, the pants build to starts working inside the step3 directory.

rich-kite-32423

12/25/2022, 1:24 AM

Inside step2, this is the output I get:

Copy code

./pants run main/main.py
18:05:39.93 [ERROR] 1 Exception encountered:

  NoApplicableTargetsException: No files or targets specified. The `run` goal works with these target types:

  * pex_binary
  * python_source
  * python_test

Please specify relevant file and/or target arguments. Run `./pants --filter-target-type=pex_binary,python_source,python_test list ::` to find all applicable targets in your project, or run `./pants --filter-target-type=pex_binary,python_source,python_test filedeps ::` to find all applicable files.

But then when I get it going in step3, and go back to step2, sometimes it will run successfully! Very strange.

rich-kite-32423

12/26/2022, 5:28 PM

OK, I've done a bunch of updating on my WSL configuration instructions, including changing the name/URL to the repo to reflect the focus on WSL. I think it's in pretty good shape now but please point out any issues here. https://github.com/BruceEckel/ConfiguringWSLForPants