pantsbuild #general

Channels

welcome

github-notifications

announce

square-pharmacist-60243

05/26/2022, 8:54 AM

Hi. Here is my sub-project structure. I have added

src/py/sub-project

to the roots.

Copy code

src/py/sub-project
    sub_project/
        __init__.py
        __main__.py
        ...
    tests/
        test_sub_project.py

When I run

./pants tailor

, I see two

BUILD

files added, one in

src/py/sub-project/sub_project

, the other in

src/py/sub-project/tests

. Why isn’t there a

BUILD

file in

src/py/sub-project

if that is the root? Now, in

src/py/sub-project/sub_project/BUILD

it has generated a

pex_binary

entry with name

__main__

and

entry_point="__main__.py"

. When I as a developer add a

__main__.py

file under a Python package (

sub_project

in this case), its goal is to provide a standard entrypoint for the package, not entrypoint in itself as if

__main__.py

was in

PYTHONPATH

(which is what name

__main__

for

pex_binary

looks to me). If I want to make

sub_project

package into something that other subprojects can depend on, where do I declare that it “publishes” the sources under name

sub_project

? Intuitively, I would put it under

src/py/sub-project/BUILD

because that directory (

src/py/sub-project

) sees the Python package

sub_project

in its entirety. But this seems to not match what

./pants tailor

generates. If I put it under

src/py/sub-project/sub_project/BUILD

python_sources

, will the contents be under

sub_project

Python package namespace?

sparse-pharmacist-76603

05/26/2022, 12:35 PM

Any manual or best practices for pants and jenkins integration?

happy-kitchen-89482

05/26/2022, 1:46 PM

Remind me, @polite-garden-50641, when we run Django's makemigrations command internally, we don't do it via

./pants run

, right?

careful-mechanic-89327

05/26/2022, 2:32 PM

Hi everyone.

careful-mechanic-89327

05/26/2022, 2:34 PM

I have been using Pants in a MacOs, and until yesterday, everything was fine, but today, I cannot execute pants anymore as Python quit unexpectedly. It is like the python process died , the MacOs give a report, but it is the typical coredump report

careful-mechanic-89327

05/26/2022, 2:34 PM

any idea?

careful-mechanic-89327

05/26/2022, 2:38 PM

Ok, I have used the recommended parameter

--no-pantsd

and it seems to work now

careful-mechanic-89327

05/26/2022, 2:40 PM

sadly when execute some goal I get this error:

careful-mechanic-89327

05/26/2022, 2:40 PM

/Users/runner/.cargo/git/checkouts/lmdb-rs-369bfd26153a2575/6ae7a55/lmdb-sys/lmdb/libraries/liblmdb/:2432: Assertion 'mp->mp_pgno != pgno' failed in mdb_page_touch()

careful-mechanic-89327

05/26/2022, 2:55 PM

good news, just removing

~/.cache/pants

, fix it

creamy-airplane-38079

05/26/2022, 3:25 PM

Hi team, does anyone know of a library ( hopefully in python) that allows you to parse the Starlark BUILD file ? I am looking at some helper functionality to automatically edit/add some target definitions in existing BUILD files.

✅ 1

enough-analyst-54434

05/26/2022, 5:04 PM

Is it expected that given a Pants command line I can no longer tack on a request for help?

proud-appointment-36730

05/26/2022, 5:46 PM

Question about listing dependencies in python projects. How do I list internal dependencies within a project? I have a large monorepo, and it would be super useful to see what other modules in the repo that module A depends on

✅ 1

proud-appointment-36730

05/26/2022, 6:10 PM

IS there any sort of "repo" or awesome list of Pants plugins out there? A visual representation of dependencies (using something like graphviz) would be killer , and I'm sure I'm not the first person to think of this

wooden-thailand-8386

05/26/2022, 8:26 PM

Hey, idk if this is useful or not for others but I’m using both

awslambda

and

google_cloud_function

targets and as the documentation predicted (https://www.pantsbuild.org/docs/awslambda-python) it is failing on MacOS. So I’ve created a little Dockerfile as a workaround for running

pants package

locally. I’ll share in a thread if you folks think it might be useful to put as an “example” on the Docs page.

proud-dentist-22844

05/26/2022, 9:30 PM

I finally took a stab at upgrading from pants 2.7 to 2.11 (via 2.8, 2.9, 2.10 as in the docs). tailor is recommending I add a whole bunch of

pex_binary

targets for files that have an

if __name__ = "__main__":

block. But in many cases, these are just poor-excuses for testing. Is there a way to tell tailor to not add any

pex_binary

targets? Or maybe I'm missing something - why should I let it add them?

✅ 2

millions-psychiatrist-72923

05/26/2022, 9:48 PM

Hey everyone, are there good resources on using pants with AWS CDK? Documentation? Example projects? How to organize stacks? I'm thinking of giving pants a try but it would be very useful to see some examples of how other people are working with it and CDK. An initial google search has been, um, not that useful: https://www.mastermans.com/itemdetail/CDK-874BLK3232

✅ 1

high-yak-85899

05/27/2022, 12:15 AM

I have a scenario where I'm trying to download a pex into a docker container and then execute it. I am able to download it and store it in the docker container. When I go to execute it, it just returns and says "Killed". Any thoughts as to what could be happening?

proud-dentist-22844

05/27/2022, 12:34 AM

For the code in

[pylint].source_plugins

, I would like to use the resolve from

[pylint].lockfile

, but setting

resolve="pylint"

in the source_plugins BUILD file, says that pylint is not a valid resolve name. Is there a good way to tie that code to the tool's lockfile?

rhythmic-morning-87313

05/27/2022, 3:48 AM

Q. In a monorep, can I let the package "A" depend on the package "B" while "A" is targeted for Python 3.10.x and "B" for Python 3.8.x, because "A" runs on a host and "B" runs inside containers (mounted by "A" when "A" creates containers) but they need to be distributed together?

sparse-pharmacist-76603

05/27/2022, 7:05 AM

Can someone help with this error:

Copy code

pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/dpetrov/.cache/pants/na
med_caches/pex_root/unzipped_pexes/aa08b4e865b8aa322f2e33f409ce7348b0188c91.
Needed cp38-cp38-manylinux_2_35_x86_64 compatible dependencies for:
 1: numpy
    Required by:
      nufft 0.0.3
    But this pex had no ProjectName(raw='numpy', normalized='numpy') distributions.

rhythmic-morning-87313

05/27/2022, 8:29 AM

hmm.

./pants-local package :: '!src/ai/backend/kernel:dist' '!src/ai/backend/runner:dist' '!src/ai/backend/helpers:dist'

i thought this would work but not...

✅ 1

fresh-cat-90827

05/27/2022, 10:30 AM

is it possible that Pants overrides the tests discovered by

pytest

when using the

--ignore

option (of

pytest

)? Or does one need to pass the path to the directory differently when run with Pants?

refined-addition-53644

05/27/2022, 12:42 PM

Hi, how does pants handle VCS dependency using pex during test? My tests are taking quite a long time (~30 mins). Is it building this dependency from source for every test ? I am using multiple resolves if that's something that might be relevant.

wooden-thailand-8386

05/27/2022, 3:08 PM

Hello, I might have found an issue but before I open a ticket on Github I wanted to check with y’all. I’m using pants to run a docker container with the following command:

Copy code

./pants run :linux-pyenv --docker-run-args="--rm -it -v $(pwd):/app"

which works fine, flags and volume get assigned as expected. Now when I move that into the

pants.toml

file, which I did as follow:

Copy code

[docker]
run_args = "--rm -it -v $(pwd):/app"

then I get this error:

Copy code

11:02:24.76 [INFO] Initializing scheduler...
11:02:24.95 [INFO] Scheduler initialized.
11:02:55.41 [INFO] Completed: Building docker image linux-pyenv:latest
docker: Error response from daemon: create $(pwd): "$(pwd)" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
See 'docker run --help'.

If I were to guess, this looks like some parsing/casting issue when reading from the toml file and sending it to docker since when I run directly from the command line it works as expected. I’ve already tried single quotes, escaping “$” and parenthesis but nothing seems to work. 😞

✅ 2

busy-vase-39202

05/27/2022, 4:18 PM

Some open CFPs that may be of interest: • DevOpsDays Boston 2022 https://www.papercall.io/dodbos22 • PyCon Latam 2022 https://www.papercall.io/pyconlatam22 • Big Data Day - Dominican Republic - 2022 https://www.papercall.io/bigdataday

brainy-solstice-27042

05/27/2022, 7:22 PM

I'm running into an odd issue that I haven't run into with older versions of pants that I'l hoping someone has experience with. I have a wheel that I've build to run under python 3.6, currently tagged with

cp36-none-any

which works under pants 1.30. After bumping to pants 2.11, all of a sudden pants says that the tag is no longer a listed as a viable candidate. I tried renaming the wheel to include one of the supported tags for python 3.6, however that hits the same error. Is there a good way to tell pants to either not worry about the tag, OR tag the wheel as something that's supported?

flat-byte-50827

05/27/2022, 8:47 PM

I'm sorry in advance if this is a stupid question. I am trying to update the version of pants in our repo. We are currently running 2.8, and I see the stable version is 2.11. Following the advice on the following documentation page, I am trying to update one minor release at a time. So I am starting with trying update to 2.9 https://www.pantsbuild.org/docs/upgrade-tips#check-for-updates-to-the-pants-script I started by updating the pants executable with

curl -L -o ./pants <https://pantsbuild.github.io/setup/pants>

Then I updated the pants.toml file with

pants_version = "2.9.0"

I did

./pants package ::

And it packaged everything fine. But then I tried testing it by opening a repl environment for one of our modules, and I got the following error:

Copy code

Traceback (most recent call last):
  File "/home/jovyan/.cache/pants/named_caches/pex_root/unzipped_pexes/efa8689a6db824683c0e9ee4a5e73ff156ff5c7b/.bootstrap/pex/pex.py", line 475, in execute
    self.activate()
  File "/home/jovyan/.cache/pants/named_caches/pex_root/unzipped_pexes/efa8689a6db824683c0e9ee4a5e73ff156ff5c7b/.bootstrap/pex/pex.py", line 139, in activate
    self._activated_dists = self._activate()
  File "/home/jovyan/.cache/pants/named_caches/pex_root/unzipped_pexes/efa8689a6db824683c0e9ee4a5e73ff156ff5c7b/.bootstrap/pex/pex.py", line 126, in _activate
    activated_dists.extend(env.activate())
  File "/home/jovyan/.cache/pants/named_caches/pex_root/unzipped_pexes/efa8689a6db824683c0e9ee4a5e73ff156ff5c7b/.bootstrap/pex/environment.py", line 322, in activate
    self._activated_dists = self._activate()
  File "/home/jovyan/.cache/pants/named_caches/pex_root/unzipped_pexes/efa8689a6db824683c0e9ee4a5e73ff156ff5c7b/.bootstrap/pex/environment.py", line 673, in _activate
    resolved = self.resolve()
  File "/home/jovyan/.cache/pants/named_caches/pex_root/unzipped_pexes/efa8689a6db824683c0e9ee4a5e73ff156ff5c7b/.bootstrap/pex/environment.py", line 506, in resolve
    for fingerprinted_distribution in self.resolve_dists(all_reqs)
  File "/home/jovyan/.cache/pants/named_caches/pex_root/unzipped_pexes/efa8689a6db824683c0e9ee4a5e73ff156ff5c7b/.bootstrap/pex/environment.py", line 593, in resolve_dists
    raise ResolveError(
pex.environment.ResolveError: Failed to resolve requirements from PEX environment @ /home/jovyan/.cache/pants/named_caches/pex_root/unzipped_pexes/3e4b475b418d085af3053314f8241f840264d4f0.
Needed manylinux_2_31_x86_64-cp-39-cp39 compatible dependencies for:
 2: nltk~=3.6.6
    Required by:
      FingerprintedDistribution(distribution=rml.carbon 0.1 (/home/jovyan/.cache/pants/named_caches/pex_root/installed_wheels/15626b0a0261f72c5c4e829cb0d8707f2b069261/rml.carbon-0.1-py3-none-any.whl), fingerprint='a93da96c5715158b8b7541015377158cad5692c9')
    But this pex had no 'nltk' distributions.
 3: pandas~=1.2
    Required by:
      FingerprintedDistribution(distribution=rml.carbon 0.1 (/home/jovyan/.cache/pants/named_caches/pex_root/installed_wheels/15626b0a0261f72c5c4e829cb0d8707f2b069261/rml.carbon-0.1-py3-none-any.whl), fingerprint='a93da96c5715158b8b7541015377158cad5692c9')
    But this pex had no 'pandas' distributions.
 5: numpy~=1.21
    Required by:
      FingerprintedDistribution(distribution=rml.core 0.1 (/home/jovyan/.cache/pants/named_caches/pex_root/installed_wheels/b5e675ffbf812888399c77f2e4ce4c5e7f6034b8/rml.core-0.1-py3-none-any.whl), fingerprint='6103b379e4791b5c147d607fef034c0a5c6fd799')
    But this pex had no 'numpy' distributions.
 6: cleanco==2.1
    Required by:
      FingerprintedDistribution(distribution=rml.processing 0.1 (/home/jovyan/.cache/pants/named_caches/pex_root/installed_wheels/51eda90b701c0f406efbc2b5818b80669e4d004a/rml.processing-0.1-py3-none-any.whl), fingerprint='36f8f58abea538996450e98addd0032d4d9892be')
    But this pex had no 'cleanco' distributions.
 7: setuptools~=59.4.0
    Required by:
      FingerprintedDistribution(distribution=rml.carbon 0.1 (/home/jovyan/.cache/pants/named_caches/pex_root/installed_wheels/15626b0a0261f72c5c4e829cb0d8707f2b069261/rml.carbon-0.1-py3-none-any.whl), fingerprint='a93da96c5715158b8b7541015377158cad5692c9')
    But this pex had no 'setuptools' distributions.

I had to remove some of the traceback which showed proprietary info, but hopefully the error is clear. I am not sure how to handle these errors, since the requirements were imported fine before the pants update, so I think I might have missed a step or done something wrong. Could anyone please provide some advice on how I can resolve this? Thanks

millions-psychiatrist-72923

05/27/2022, 9:55 PM

What's the proper way of running all scripts with the name

deploy

that changed since

origin/main

? I tried

./pants run --changed-since=origin/main --changed-dependees=transitive ":deploy"

but that gave me

Copy code

You used `--changed-since` at the same time as using target arguments. Please use only one.

✅ 1

cold-sugar-54376

05/28/2022, 2:34 AM

hey all, i’m using a

pyproject.toml

and

poetry_requirements

to define requirements for my pants project. I’m finding that everytime I change/upgrade a dependency (even a dev one such as

pytest

) in

pyrpoject.toml

pants tries to rebuild all my targets (even pex_binaries which don’t depend on it). Any idea why this could be happening? From what I can gather it’s because pyproject.toml ends up being a transitive dependency of every target in the repo

✅ 1