pantsbuild #general

Channels

welcome

github-notifications

announce

high-magician-46188

10/22/2022, 8:20 AM

For the record, the solution that worked for me was: 1. Remove the artifacts produced by

pants tailor ::

2. Re-reading all of the docs under “Getting Started”, “Python”, “Writing Plugins” and a bit more. 3. Implement my own script to produce a global

requirements.txt

for Pants. 4. Implement my own script to produce

BUILD

files for Pants. I now have a much better understanding (and progress). My takeaway here for me is that the docs are helpful (in contrast to the usual stuff), so if you have a special use-case (like that fact that we have our own custom build-system that is not well supported by

tailor

), you should probably just trudge through it.

lively-dusk-46231

10/22/2022, 8:40 AM

H, I am trying out pants for the first time. I have a python project where I have the current structure as:

Copy code

projects/
    my_api
        my_api
           web
           schema
           service
        my_api_test
        setup.py
    ... 2 others
libs/
    ... 5 common libraries which are used across the projects

And when I run

./pants tailor

it seems to be creating a BUILD file for every single folder (100+) in

my_api

my_api.web

my_api.schema

my_api_test

, etc. Is that expected ? I was thinking I just needed 5 (libs) + 3 (projects) BUILD files

✅ 1

lively-dusk-46231

10/22/2022, 1:55 PM

I was able to setup my project based on the guidance in the above thread. I then ran

./pants fmt ::

And it successfully ran

black

and

isort

(and I saw it doing it on multiple cores) which was pretty neat. I then re-ran the same command and it ran super fast (10sec). I guessed that pants realized I had not modified these files, and so used some caching to Now, I realized that in one of my

libs/

- the isort config for

known_first_party

was not correct. So, I fixed that in my

pyproject.toml

And reran:

./pants fmt ::

ISSUE: It says nothing needs to be changed. When I format it using:

venv/bin/isort libs/mylib/

it uses the correct configs. When I again run:

./pants fmt ::

- it reformats it back using the older configs

lively-dusk-46231

10/22/2022, 5:42 PM

Facing something else which seems odd with my custom

BUILD

file I have an example structure as:

Copy code

app1
    ... files in app1
scripts
    ... some misc helper scripts
BUILD

BUILD

contains:

Copy code

python_sources(name='source', sources=['app1/**/*.py'])

---- Now, when I do:

./pants list ::

It shows:

Copy code

app1/__init__.py:source
app1/module1/__init__.py:../source
scripts/script1.py <-------------------------- I never mentioned this in my sources. Why is it here ?

❗ 1

✅ 2

lively-dusk-46231

10/22/2022, 7:17 PM

In my project, we need to support python 3.6+ So, we use older dependencies for 3.6 And newer dependencies for 3.7+ Is there an option to do that with the lockfile approach in pants ? When I tried to set my interpreter constraints to 3.6+ bandit caused issues. And on setting bandit version to <1.8 and creating my own lockfile, bandit.lock has bandit==1.7.x So that lockfile could not be used with py3.6 When I set bandit version <1.6 it started working fine. But I would prefer using the newer bandit for latest python versions (Bandit isn't a big concern for me. But many of my runtime dependencies like pandas, network, flask. Etc need to be latest versions to avoid Vulnerabilities from the older versions)

lively-dusk-46231

10/23/2022, 4:47 AM

I found an issue with

bandit

when debugging some version issues with

pants

https://github.com/PyCQA/bandit/issues/960 Do you think it makes sense to change the default

[bandit].version

from

bandit<1.8,>=1.7.0

bandit<1.8,>=1.7.0,!=1.7.2

? If yes - I can create an issue in github

cold-vr-15232

10/23/2022, 11:58 AM

What is the good way to structure a python monorepo so that mypy works well with test code? How should the roots be setup? How should the test utils be configured? The examples I found are for apps without tests, and I have issues with all variants that I tried. For example, if "tests" is not a package, mypy wants unique names of the files across all applications. If it is a package, then there are import problems. This is the structure I have

Copy code

apps
├── demo_app
│   ├── BUILD
│   ├── pyproject.toml
│   ├── src
│   │   └── demo_app
│   │       ├── BUILD
│   │       ├── __init__.py
│   │       ├── main.py
│   │       └── model.py
│   └── tests
│           ├── BUILD
│           ├── test_demo_app_main.py
│           └── util.py

and the roots are

Copy code

apps/demo_app/src
apps/demo_app/tests

nice-florist-55958

10/23/2022, 8:12 PM

Some developers of our codebase will have environments where docker build is not allowed. Is there any way to make packaging docker_image targets fail with a warning instead of error? What is a minimal targets filter to exclude this type from a :: glob (or selection w/ new semantics)?

lively-dusk-46231

10/24/2022, 8:16 AM

I am trying to create some wheels from my codebase using pants. I read through

package

, the

setuptools

integration and

setuptools_scm

parts too Current setup: We have

setup.py

and

pyproject.toml

. With

setuptools_scm

I tried:

Copy code

$ cat BUILD
python_sources(name='source', sources=['my_api/**/*.py'])

vcs_version(
    name='version',
    generate_to='my_api/_version.py',
    template='version = {version}',
)

resource(name='pyproject', source='pyproject.toml')

python_distribution(
    name='my-api',
    dependencies=[
        ':pyproject',
        ':version',
    ],
    provides=python_artifact(name='my-api'),
    sdist=False,
    generate_setup=False,
)

And then did:

./pants package ::

It threw an error about

setuptools_scm

Copy code

LookupError: setuptools-scm was unable to detect version for /tmp/pants-sandbox-hVjRo4/chroot.

So, I removed

setuptools_scm

from my

pyproject.toml

Then again ran:

./pants package ::

This is generating

dist/UNKNOWN-0.0.0-py3-none-any.whl

And I can't figure out why the name/version is incorrect Any thoughts ?

lively-dusk-46231

10/24/2022, 3:21 PM

When figuring out caches in pytest runs Does pants only use the file based dependency ? Or does it use the coverage to figure out which lines of code were run by a testcase, and return the testcase only if those lines of code were modified (something like pytest-testmon)

flat-zoo-31952

10/24/2022, 3:33 PM

Got a weird issue where an engineer at our company has a dev container set up such that

/proc/self/fd/0

is a broken symlink to a non-existent terminal. This breaks Pants with pantsd running locally for them, as pantsd complains that it can't open this terminal. I'm not sure how this situation can arise (I'll dig), but it is a real thing. However I cannot reproduce. My suggested workaround is

export PANTS_DYNAMIC_UI=false

, as running without pantsd makes local usage pretty tedious. Would Pants devs expect this to work?

❗ 1

happy-kitchen-89482

10/24/2022, 4:52 PM

/github unsubscribe pantsbuild/pants discussions

🙌 2

boundless-zebra-79556

10/24/2022, 5:01 PM

pylint is becoming really slow as I pull in more dependencies. Just to run

pylint

, do we need to build requirements.pex and all? Is there anyway I can avoid the building step?

prehistoric-river-46928

10/24/2022, 5:48 PM

I had pants running correctly but now it seems it is not recognizing my python env. I had a virtual environment

c3venv

but that one no longer exists, I created a new one

new_env

but the error indicates pants keeps pointing to the unexisting env. After manually setting the

pants_python

variable in the

pants

finds python but it does not recognize the pants module anymore. Does anyone know how to fix this?

prehistoric-river-46928

10/24/2022, 6:03 PM

Copy code

Exception: Failed to read link "/Users/davidnino/Develop/workera-dev-workspace/src/backend/Users/davidnino/venvs/c3venv/bin/python": Absolute symlink: "/Users/davidnino/Develop/workera-dev-workspace/src/backend/Users/davidnino/venvs/c3venv/bin/python"

bitter-ability-32190

10/24/2022, 8:10 PM

How do I use the option dict add semantics in a config? https://www.pantsbuild.org/docs/options#addreplace-semantics Specifically trying to add to

cli.alias

✅ 2

happy-kitchen-89482

10/24/2022, 8:13 PM

Copy code

[foo]
bar.add = ['baz']

👀 1

bitter-ability-32190

10/24/2022, 11:00 PM

Can I parameterize multiple fields in tandem? e.g. parameterize `python_requirement`'s

requirements

and

modules

in tandem

clever-gigabyte-29368

10/25/2022, 1:25 AM

Hi team, could you please advise which is the earliest version of pants that supports

run_goal_use_sandbox

? I see that’s listed in 2.12 doc (link) but unfortunately it seems not supported

Copy code

InvalidFieldException: Unrecognized field `run_goal_use_sandbox=False`

bored-energy-25252

10/25/2022, 2:59 AM

How can I disable dependency inference for Python: https://blog.pantsbuild.org/dependency-inference/ A use case, I wanna launch a Python repl for interactive usage, I do not want any of my dependencies are missing and I do not want to create a virtualenv manually.

✅ 1

fierce-keyboard-91032

10/25/2022, 5:54 AM

I'm trying to setup a project using an older version of Python (3.7), which is not installed on the machine (the machine is running 3.9). I setup

interpreter_constraints = ["==3.7.*"]

inside

pants.toml

, and

./pants test

and

./pants lint

works well. But whenever I try to do a

./pants run

for some

pex_binary

, I receive the error

/usr/bin/env: 'python3.7': No such file or directory

. It seems to look for a global installation of

python3.7

in my machine, instead of the one installed by pants. Is there some configuration I needed to fix this? The pex binary:

Copy code

pex_binary(
    name="manage",
    entry_point="manage.py",
)

`manage.py`:

Copy code

print('Hello world')

fresh-continent-76371

10/25/2022, 8:05 AM

Hello - i have a scenario, test migrating a repository over to pants, i have found a test which has a transitive dependency that, on import fails in the regex pattern ; in the current non-pants test matrix 3.8, 3.9, 3.10, the particular test succeeds; Presumably because the offending transitive import is not being loaded. The error in question (not germane to the problem) is a bad old regex with now invalid escape sequences.

Copy code

'e': ('[0-9]+\.?[0-9]+(?:e[-+]?[0-9]+)?', float),
E             ^
E   SyntaxError: invalid escape sequence \.

The current way of testing works is :

Copy code

coverage run --module --branch pytest --verbose --junit-xml=pytest_results.xml path/to/tests/test_subject.py

but when I run

Copy code

./pants test path/to/tests/test_subject.py

it fails with the deep transitive error (importing the lib fails - doesn't get to run the test) I have a sense that somehow, "coverage/pytest" is doing a lazy load, and not branching to this area of the test import. (or it is not actually testing like we think it is) and Pants is "not" lazy loading, thus when it hits this super old 3.9+ incompatible "escape sequence" it errors. Does anyone have any insights to this; it is good that Pants has uncovered this old escape sequence in the library that is old, but why do we not see it in the "current" testing method ;

fast-gold-27885

10/25/2022, 8:43 AM

👋 Всем привет!

clever-crayon-70731

10/25/2022, 10:59 AM

I'm facing the exact same problem. Has there been any progress on this? The proposed solution seems really painful.

lively-dusk-46231

10/25/2022, 11:01 AM

I'm a bit confused on how to write dependencies manually. Maybe I don't fully understand the concept of a target (I have read the docs for it though) I have a setup where:

Copy code

$ cat BUILD
python_sources(name='source', sources=['my_api/**/*.py'])

And I am dynamically importing a file:

my_api/config/development_settings.py

I tried:

Copy code

python_sources(
    name='source',
    sources=['my_api/**/*.py'],
    overried={'my_api/config/__init__.py': {'dependencies': ['my_api/config/development_settings.py']}},
)

And it did not work. Then I tried

my_api/config/development_settings.py:../../source

and it seemed to work Question: Why did the "shortcut" not work for me ? Do I need to explicitly add

../../

everywhere ? In another case, I want configs for my pytest to be specified. So, I have something like:

Copy code

python_tests(
    name='tests',
    sources=['my_api_test/**/test_*.py'],
    dependencies=['my_api_test/config/*.py']},
    # Also tried: dependencies=['my_api_test/config/*.py:*']},
    # And also: dependencies=['//my_api_test/config/*.py:*']},
)

This does not work with the error:

Directory 'corridor_api_test/instance' does not contain any BUILD files

freezing-lamp-12123

10/25/2022, 12:27 PM

is it possible to parameterise a target based on output from another target? .. or how would you go about that? …

fresh-cat-90827

10/25/2022, 2:55 PM

I noticed that since upgrade to

2.13.0

from

2.12.0

, we cannot any longer produce Python wheels in a multiple top-level projects layout unless we add the

"/"

to the source root patterns which doesn’t seem to be necessary? Here’s the repo to reproduce. Saw a discussion here but don’t think the reason for this behaviour was discovered.

adamant-magazine-16751

10/25/2022, 3:03 PM

I'm looking for a way to run a python script before packaging a pex_binary. I'd like the script to create a file, that would then be packaged as a resource into the pex. Is writing it as

experimental_shell_command

that executes

python my_script.py

the way to go? What do I do afterwards to access the file?

fresh-cat-90827

10/25/2022, 4:03 PM

is it a limit of the Pants

setup_py

macro(?) or just a limitation of the

peek

command? It doesn’t report the fields in the

provides

field of the

python_distribution

target. See the thread.

clever-crayon-70731

10/25/2022, 4:50 PM

After getting ABI incompatible wheels in my lambda zips, I did the following: 1. Retrieve platform spec from AWS base image 2. Create a target for the platform spec JSON and reference it in

python_awslambda(..., complete_platforms=[":platform"])

However, running

pants package

does not produce a different lambda.zip (I compared checksums) What am I missing?