pantsbuild #general

Channels

welcome

github-notifications

announce

polite-vase-75369

12/11/2020, 7:33 PM

i tracked it down to a kafka dependency, but im not sure why my managed dependencies is not overriding their value

witty-crayon-22786

12/11/2020, 8:11 PM

there is a plan for holistically improving dependency management in 2.x… it’s prioritized because it works across languages (and people want it for Python), and i expect that it will be less error prone than the “dependency management” scheme from 1.x… sorry for the trouble.

➕ 2

polite-vase-75369

12/14/2020, 4:23 PM

I’ve been trying to upgrade from 1.16 -> 1.27, made a lot of progress, but im having issues with python lint. I see this step fail

pythonstyle

. I’ll attach the actual stack trace as a response to this message. I added some debugging to the pants python code, and it looks like it is resolving dependencies to the root site-packages directory rather than the individual one per each dependency

[pantsbuild.pants.contrib.python.checks.checker 1.27.0 (/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.4g9T7C/install/lib/python3.6/site-packages), pyflakes 2.1.1 (/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.4g9T7C/install/lib/python3.6/site-packages), pycodestyle 2.4.0 (/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.4g9T7C/install/lib/python3.6/site-packages), six 1.15.0 (/Users/lawrencefinn/.cache/pants/setup/bootstrap-Darwin-x86_64/pants.4g9T7C/install/lib/python3.6/site-packages)]

. Not sure what to do here, maybe i have some dumb flag?

magnificent-vr-79843

12/15/2020, 12:26 AM

According to documentation:

To force your tests to run again, rather than reading from the cache, run ./pants test --force path/...

I am getting

Unrecognized command line flags on scope 'test': --force

Any ideas what can be wrong there?

late-lawyer-50596

12/15/2020, 11:06 PM

Hi, I'm upgrading from 1.3 to 2.0 and getting a ton of

Unmatched glob

warnings. I tried adding

ignore_pants_warnings = [".*"]

pants.toml

but no luck. I also looked at the

files_not_found_behavior

option but it only takes

[warn, error]

and no

ignore

. Are there any ways to completely silence the unmatched glob warnings?

user

12/16/2020, 4:30 PM

https://twitter.com/ndm_haskell/status/1338948012815642630 Pants Build retweeted: Great to hear it was useful! The more build systems teams can share knowledge, the better we all do. And great to s… https://twitter.com/i/web/status/1338948012815642630 Twitter @ndm_haskell Thanks a lot for writing https://neilmitchell.blogspot.com/2020/11/data-types-for-build-system-dependencies.html ! I was investigating a bug in @pantsbuild recently,… https://twitter.com/i/web/status/1338947249712218114 Twitter

salmon-barista-63163

12/16/2020, 6:42 PM

Are there any plans to release plugins for pycharm or vscode? Similar to the intellij plugin?

witty-crayon-22786

12/16/2020, 7:35 PM

https://pantsbuild.slack.com/archives/C18RRR4JK/p1608147321153900

salmon-barista-63163

12/16/2020, 8:19 PM

I am looking for some help on a performance issue i am having with pants 2.0.0. I have what I believe is a unique setup with one of our application servers that we build and run with pants. This specific server spawns other pants processes from within itself. With pants v1 we were able to run multiple

pants run

commands from within a single

pants run

process without issue. With the v2 engine we are CPU and memory bound and the native OS will start killing random processes (as expected) when we reach the ceiling. A little background on how this server works:

Copy code

application server: main application process running with ./pants run
child server: Receives a request to start a "task" or series of "tasks" = multiple ./pants run sub-processes executed in parallel. These subprocesses then exit gracefully after they are completed.

The way this works, we can receive sub process commands to execute dozens of

pants run

“task” commands at a single time. Some things I have tried that do not have effect on performance: 1. pantsd on or off 2. Concurrency limits set in global pants options I am looking for some guidance here as we have tried everything that comes to mind without changing how this application runs. (That is out of our scope at this time as the main concern here is getting to pants 2.0.0) We are currently on pants 1.30.

stale-artist-91164

12/16/2020, 10:26 PM

Question, is it possible to use pants run and pycharm's interactive debugger together? Pants docs looks like it will work great for CICD, but I couldn't find anywhere about here is how you can single step through your code locally while you are developing

polite-zebra-76126

12/17/2020, 2:05 PM

Hi everyone. Enjoying Pants so far, threw together a monorepo for Python modules pretty quickly using the v2 engine. I'm running into an intellisense issue that is making it pretty difficult to work with though. I'm using VS Code, for reference. What does everyone here use or suggest?

refined-dusk-58376

12/17/2020, 9:07 PM

I wanted to link this for people that are having issues with the development process. One of the things I determined at some point was that for monorepos it's helpful to separate each project into their own workspace. I plan on writing some tools for working with Pants at some point, though the work I have on my plate is a bit intense. I can't do it immediately until certain obligations are met. https://marketplace.visualstudio.com/items?itemName=folke.vscode-monorepo-workspace

👀 1

jolly-midnight-72759

12/17/2020, 9:12 PM

Is there a way in pants v1.25 to build a pex with multiple architecture dependencies (i.e. macOS and linux)? In pants v2.1.x? If this is answer in the docs, my apologies. I was not able to find the answer.

hundreds-father-404

12/17/2020, 10:11 PM

Are the pants scripts versioned? It might be helpful to know when it is a good idea to update this.

@jolly-midnight-72759 it’s in GitHub at https://github.com/pantsbuild/setup. Normally, it’s a good idea to update the

./pants

script to get any improvements. We eagerly error if the version of the script isn’t compatible with a certain Pants version.

👍🏽 1

salmon-barista-63163

12/18/2020, 12:28 AM

Is there a way to limit the amount of parallel processes that run with pytest?

salmon-barista-63163

12/18/2020, 4:28 PM

Hey everyone, once again im having performance issues with the v2 engine and looking for suggestions. Here is my setup:

Copy code

circleci
8 CPU
16 RAM

I am trying to run tests via pytest or behave. In this environment we start up our servers and run the tests. we have about 8 webservers running (with pants) and we kick off the tests. I am getting out of memory errors and process killed issues when running. If you have been following what i have been posting about i am having all sorts of performance issues. I am running pants 2.0.1rc4 (with a twist from a release that was cut for me yesterday). There is no way the performance of v2 can be this much worse than v1 as a whole. At this point im thinking im missing or overlooking something. Any suggestions?

stale-artist-91164

12/18/2020, 8:19 PM

@happy-kitchen-89482 Hey, just saw you've added instructions for pycharm, thanks. But wanted to ask, have you considered setting up a dev target like lerna (https://lerna.js.org/ javascript/typscript mono-repo tool). 'lerna bootstrap' create symlinks between all the dependent packages allowing you to develop and debug without any fancy setup. Develop using symbolic links, and then package and run for running unit tests, E2E and deployment.

incalculable-analyst-9027

12/22/2020, 6:33 AM

Hello everyone, the project I'm working on is using pants 1.5.0. As I know, when I run the command ./pants -version for the first time, the Python environment for pants will be set up in ~/.cache/pants/. It works for a long time. But now I encounter a problem when trying to build by pants in a new Mac. The error is Exception message: Could not satisfy all requirements for cffi==1.11.1: cffi==1.11.1(from: pantsbuild.pants.contrib.go==1.5.0->pantsbuild.pants==1.5.0), cffi>=1.12(from: pantsbuild.pants.contrib.go==1.5.0->pyopenssl==17.3.0->cryptography>=1.9) I found this is caused by the upgrading of cryptography from 2.4.2 to 3.3.1 My quesiton is that, can I make the pants fetch cryptography in a fixed version (say, 2.4.2) when I setup the pants in a new Mac? Many thanks in advance!

polite-vase-75369

12/22/2020, 7:06 PM

For python libs, is there a good way to alias a library into a package?

clean-night-52582

12/23/2020, 1:20 AM

Any trick to integrating

ipdb

I tried adding https://www.pantsbuild.org/docs/python-test-goal#running-tests-interactively so added

pytest_plugins.add = *[*"ipdb"*]

got* OSError: pytest: reading from stdin while output is captured! Consider using '-s'

acceptable-guitar-79854

12/26/2020, 2:09 PM

Merry Christmas everyone! In my case, I cannot install pants anymore since the last commit to

./pants

(the virtualenv thingy). I tried prepending

$venv_path/src

to PYTHONPATH but then it complains about

appdirs

Copy code

(pants) cristian@Cristian-XPS13:~/workspace$ ./pants --version
Traceback (most recent call last):
  File "/home/cristian/.cache/pants/setup/bootstrap-Linux-x86_64/virtualenv-20.2.2/src/virtualenv/__main__.py", line 77, in <module>
    run_with_catch()  # pragma: no cov
  File "/home/cristian/.cache/pants/setup/bootstrap-Linux-x86_64/virtualenv-20.2.2/src/virtualenv/__main__.py", line 58, in run_with_catch
    from virtualenv.config.cli.parser import VirtualEnvOptions
ModuleNotFoundError: No module named 'virtualenv'
./pants: line 293: /home/cristian/.cache/pants/setup/bootstrap-Linux-x86_64/2.0.0_py38/bin/python: No such file or directory

Copy code

(pants) cristian@Cristian-XPS13:~/workspace$ ./pants --version
Traceback (most recent call last):
  File "/home/cristian/.cache/pants/setup/bootstrap-Linux-x86_64/virtualenv-20.2.2/src/virtualenv/__main__.py", line 77, in <module>
    run_with_catch()  # pragma: no cov
  File "/home/cristian/.cache/pants/setup/bootstrap-Linux-x86_64/virtualenv-20.2.2/src/virtualenv/__main__.py", line 58, in run_with_catch
    from virtualenv.config.cli.parser import VirtualEnvOptions
  File "/home/cristian/.cache/pants/setup/bootstrap-Linux-x86_64/virtualenv-20.2.2/src/virtualenv/__init__.py", line 3, in <module>
    from .run import cli_run, session_via_cli
  File "/home/cristian/.cache/pants/setup/bootstrap-Linux-x86_64/virtualenv-20.2.2/src/virtualenv/run/__init__.py", line 6, in <module>
    from ..app_data import make_app_data
  File "/home/cristian/.cache/pants/setup/bootstrap-Linux-x86_64/virtualenv-20.2.2/src/virtualenv/app_data/__init__.py", line 9, in <module>
    from appdirs import user_data_dir
ModuleNotFoundError: No module named 'appdirs'

calm-alarm-89082

12/28/2020, 9:29 PM

hi 👋 I'm using the new goal typecheck for running mypy and I'm running it only for some targets but when I run typecheck for a specific target it also runs it for its dependencies. is there a way to tell pants/mypy to ignore them?

witty-crayon-22786

12/29/2020, 5:31 AM

https://pantsbuild.slack.com/archives/C18RRR4JK/p1609219865170600

loud-stone-83419

12/29/2020, 4:52 PM

Hi! I am looking to build a quick scalatest plugin for v1.30 - The Junit plugin is not working for Scalatest 3.2 (I need to change source test files to make it work) I was wondering - Can I use the V2 api even though the scala part will use scala_lib?

high-egg-2153

12/30/2020, 8:51 PM

Hey. Just starting to experiment with pants for a python project. I'm having an issue trying to run anything. When I run:

./pants run accounts_report/app.py

I am getting the following error:

from pkg_resources import get_distribution, parse_version

ModuleNotFoundError: No module named 'pkg_resources'

Some details:

Python 3.8.6

pip 19.3.1

Manjaro Linux

Any help would be greatly appreciated.

witty-crayon-22786

12/31/2020, 2:24 AM

https://pantsbuild.slack.com/archives/C18RRR4JK/p1609381484172700

happy-kitchen-89482

01/03/2021, 1:19 AM

Hi Pants community! Despite its many challenges, 2020 was a huge year for the project, culminating in the long-awaited release of Pants v2! Also noteworthy last year was the launch of our brand new documentation site at https://www.pantsbuild.org/. 2021 will be another busy year, in which we plan to expand Pants's Python user base, grow the contributor community, and implement support for more languages and frameworks. I'm very excited for what's ahead! Thanks to all who contributed this past year, whether patches, documentation, bug reports or feature suggestions. And to all who adopted Pants v2 - thanks for your faith in the project, and my apologies for any bumps in the road. Your feedback allows us to continue to develop and improve the system. Our aim is to be the best build system, with the most supportive community, for Python and more. Your help is crucial in realizing that goal! I hope you all remain safe as we emerge from these challenging times into better days ahead. Happy new year! Benjy

🎉 6

👍🏽 2

jolly-midnight-72759

01/03/2021, 9:36 PM

One of our basic tests is failing on v2.1.1. When I run

pants list ::

it exits with status 1. When I do

for p in $(fd -t d -d 1 .) ; do echo $p ; ./pants list ${p}:: ; done

(i.e. for every directory run,

.pants list ${DIRNAME}::

) and also

.pants list :

I get a list of directories and it exits with status 0. I have print stack trace set to to true and also ran with

-ldebug

. No additional output was provided. Help with debugging would be appreciated.

jolly-lamp-97248

01/04/2021, 3:59 AM

Hi, want to confirm that what's the latest version of pants for a scala project? Is it 1.30? Pants 2 hasn't supported scala yet? Thank you!

loud-stone-83419

01/04/2021, 11:00 AM

Not sure if this is the proper location for feedback: In pants v1 there is in the Junit module a way to have safe args if too long through argfile, which is great but assumes the Junit module is compatible with the argfile format. Since Java CLI is compatible with argfile it would be great if argfile safety could be put at the Java process execution layer so that a module that uses that subsystem doesn't have to handle it. ie as I was doing the integration for scalatest, the scalatest runner don't handle argfiles, so I needed to get around that instead of having the safety provided by a higher level of abstraction. Might be something tricky about caches though that I may not realize. That's mostly voiced in perspective for v2 JVM integration Let me know if I need to clarify 🙂 Thanks for the great work! and happy new year!