Would this be what you’re looking for? https://www.pantsbuild.org/v2.0/docs/rules-api-testing#approach-4-run_pants-integration-tests-for-pants This test utility allows you to run full Pants commands as an integration test. In your test, you could run ./pants setup-py for example. You could then do whatever you want with the result, including inspecting the files that were created.

I have a dependency on torch-geometric which requires specific wheels to be installed depending on the hardware platform as given in https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html. I use poetry to generate requirements.txt file and able to generate it from poetry and it directly uses the https url for the wheel file. When this is used in pants using python_requirements() rule, ends up in a ValueError: Invalid requirement in requirements.txt at line 119 due to value 'https://pytorch-geometric.com/whl/torch-1.6.0/torch_cluster-latest%2Bcpu-cp37-cp37m-macosx_10_9_x86_64.whl'. Is there a support for adding a wheel file url as 3rd party dependency or a workaround for now?

@hundreds-father-404 Could be! Initially, I thought it is only for testing of plugins but it can probably be used to run any script within the project, right? With something like:

run_pants(["run", "src/python/package_A/script"])

Using this approach, is it possible to run multiple binaries at the same time? I will give it a try, thanks for the tip!

Initially, I thought it is only for testing of plugins but it can probably be used to run any script within the project, right?

Typically, it’s designed for running on synthetic test projects created via

setup_tmpdir()

. I imagine you want to run on your actual project, rather than some contrived example, right? This is where things would start to fall apart. Tests run in a chroot (tmpdir) in Pants, meaning that they will only copy over files that are known about through the

dependencies()

field. (Why? This allows us to safely run tests in parallel and remote execution). So, your test would not have access to

src/python/package_A/script

. You could put a dependency on the

python_binary

target in

src/python/package_A/script

, but that won’t do what you’re expecting. That will copy over the Python files with the source roots stripped, like

package_A/script

. It won’t copy over the raw file, and it won’t copy over the BUILD file either. So,

run_pants(["run", "src/python/package_A/script"])

would complain “No BUILD file in src/python/package_A/script”. You could work around that by declaring a

files()

target owning that script, and owning its original BUILD file. A

files()

target doesn’t strip source roots - the files get copied exactly as-is. Then, add a dependency to the

files()

target to the

python_tests()

. That works fine, and we do that for several Pants integration tests. Unless…`src/python/package_A/script` depends on a bunch of other files, like most projects do. You would need a

files()

target owning all of its transitive dependencies..which we’ve found is not feasible to do. So, my suggestion is only helpful if either a) you’re okay with creating a synthetic target, or b)

"src/python/package_A/script"

has very few dependencies. I suspect neither of those are true, unfortunately. -- Given the above, Benjy is thinking through more robust ways to do what you’re asking

Is it possible to use https://github.com/bazelbuild/rules_rust with Pants? Or I need something specific to Pants?

Long story short: there is a database -- a large code base mixing C and Rust (and also some other languages for code generation).

I would like to have a build system which is as reproducible as possible.

But also I'd like to avoid bringing Java into the build process (that's why I'm looking at Pants and Please instead of Bazel).

hey there! I started testing pants 2.0.0a1 and I noticed that

python_app

is causing chaos and there isnt

bundle

as a goal anymore. 😞 Did it become something else or we won’t have support for that on v2.0?

Nevermind, found it

We redesigned it in 1.30.0 and are putting final polish on how coverage works in 2.0.0. See the PR description of <https://github.com/pantsbuild/pants/pull/10100> for how it works now

is there a way to specify a coverage file config .coveragerc for the test ? Found something that this is available in 2.0.0.dev3 . is anything available in the stable version 1.30?

Thanks for the exhausting info!

I imagine you want to run on your actual project, rather than some contrived example, right

Exactly. Unfortunately, scripts have quite a lot of dependencies. Looking forward to hearing about some more robust way 🙂 At the moment I had to disable integration tests. Many thanks!

I just discovered another issue. There is a script S1 depending on the package A. The S1 and A are both within my project. The S1 downloads another script S2 (that also imports something from A) and runs it as a subprocess. Unfortunately, it seems that S2 can't import from A:

ModuleNotFoundError: No module named 'A'

I tried running S1 with ./pants run S1 as well as "./dist/S1.pex" with the same result. How can this be solved? Maybe it is just a matter of PYTHONPATH or something like that? I thought that pex is "just" virtual environment so anything that runs within it should be able to import anything that is there 🙂 This can be "solved" by deploying packages through pypi but we also need to build docker images for our scripts and it would be nice if we can just copy one pex file into each docker image. Also, being able to easily test this functionality during development would be nice.

...ModuleNotFoundError fixed by altering PYTHONPATH of the subprocess like this. Not sure if it might be useful for someone else, but it is simple as this:

pypath = ":".join(sys.path)
myenv = os.environ.copy()
myenv["PYTHONPATH"] = pypath
PROCESS = await asyncio.create_subprocess_exec(
    script_path,
    stdin=asyncio.subprocess.PIPE,
    stdout=asyncio.subprocess.PIPE,
    stderr=asyncio.subprocess.STDOUT,
    env=myenv,

Hello there! I’m adding an internal wheels cache for pants and it’s a self-signed certificate for the url that’s serving it so when I run pants it throws back

WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError("hostname

… seems like using

PYTHONHTTPSVERIFY=0

would “fix” that but I’m not sure how to do that / if there’s a better workaround

Change to disambiguate output under

dist/

. Feel free to bikeshed the scheme: https://github.com/pantsbuild/pants/pull/10735

Hello there! We recently started using Pants as our build system and are still getting used to some nuances. I have a few questions around protobuf and docker support - • When will protobuf support be re-enabled? It looks like it was removed in 2.0.0a2. • Are there any best practices for using pants to generate & run pex binaries inside Dockerfiles? Since we generate our pex files inside the docker container (see the Dockerfile snippet below), there is no caching of dependencies. Ideally, if none of the dependencies or source files change, pex shouldn't need to be regenerated in the container. • Is there a way to enable 'live reloading' while debugging/code changes? It will help improve the development flow a lot! Sample Dockerfile -

FROM python:3.7.7-slim

RUN apt-get update \
    && apt-get install -y \
    curl \
    cmake \
    build-essential \
    && apt-get clean


EXPOSE 3000
WORKDIR /app

RUN curl -L -o ./pants <https://pantsbuild.github.io/setup/pants> \
    && chmod +x ./pants
# Copy src dir and pants config files
COPY ./src /app/src/
COPY BUILD /app/
COPY pants_docker.toml /app/pants.toml
COPY requirements.txt /app/

COPY ./dist /app/dist/

RUN ./pants --version

RUN ./pants binary src/nabu/services/api:server

CMD ./dist/server.pex nabu.services.api.wsgi:app -c src/nabu/services/api/gunicorn.conf.py

Hello, what linux requirements does pants have? I'm trying to run it inside docker.io/centos:8.2.2004 and I get an error when building the linters https://pastebin.com/EBBa13Ey

Hi everyone, I am just getting started with pants (but worked with bazel before). I am trying to do

./pants lint my_project/

and it takes forever to build PEX-es for linters e.g. flake8

I am running behind a proxy

with pants 1.30 I get connection timeout error while with 2.0 it just takes forever

basically it cannot download any requirement/dependency

can someone please help?

I have http_proxy and https_proxy set in my .bashrc and I am setting them again in the pants script

Hi there, I’m trying to use local wheels and I’m not sure if

pants

is actually getting the .whl files from the folder. What I have on my toml file is:

[python-repos]
repos.add = "%(buildroot)s/.wheels

One of the reasons I’m doing this is to try to speed up the test/bundle by not having to download any dependency from the internet (we have tensorflow and other ML libraries) and under our proxy things take forever 😞

We're having some issues updating our scala stack to the latest scalatest version. The issue was solved here: https://github.com/pantsbuild/pants/pull/9707, but the shipped junit-runner was not updated: https://github.com/pantsbuild/pants/blob/1.30.x/src/python/pants/backend/jvm/subsystems/junit.py#L20

Is there a way to override the junit-runner jar used?

Starting to get an error pretty frequently in Pants 1.30 when running

fmt

big stack trace in thread