pantsbuild #general

I have a plan for making the Jenkins sharding more user friendly too, and to check in the config that then updates the jenkins jobs themselves, so it's not just a scary mutable playground

bored-art-40741

02/08/2016, 3:27 AM

Heads up @wide-energy-11069 and IntelliJ friends: I split out http://jenkins.pantsbuild.org/ into a few tabs to make it less noisy. I set Pants CI to the default tab, but made a dedicated tab for IntelliJ CIs

bored-art-40741

02/08/2016, 3:28 AM

I think you can choose which tab is your personal default as an account setting

ancient-stone-50795

02/08/2016, 3:32 AM

@bored-art-40741 Sorry, I'm AFK. I didn't notice anything else when I looked this morning. You updated the docs about the ivysettings.xml example, right?

bored-art-40741

02/08/2016, 3:39 AM

err, the docs?

bored-art-40741

02/08/2016, 3:39 AM

Maybe I missed some docs I should be updating, I haven't updated anything not in regular code

bored-art-40741

02/08/2016, 3:40 AM

I did make the error message that results from a failure to find the resolution report to be much more explicit about why it probably occurred

bored-art-40741

02/08/2016, 3:50 AM

Ah I think maybe I see the doc you're referring to

bored-art-40741

02/08/2016, 3:50 AM

src/docs/setup_repo.md

bored-art-40741

02/08/2016, 3:50 AM

Oh, and publish has its own templated ivysettings.xml. I can fix that up too, I think

bored-art-40741

02/08/2016, 3:51 AM

Though it seems like publish should probably just always go through an isolated cache, since the point is to avoid pollution

wide-energy-11069

02/08/2016, 5:50 AM

@bored-art-40741: it looks good, thanks!

curved-manchester-66006

02/08/2016, 5:00 PM

Is there a "catch" to use pkgutil.get_data in tests? I think from these docs these two statements should be equivilant

Copy code

data = open(os.path.join(os.path.dirname(sys.modules[__name__].__file__), 'resources/dmidecode-basic'), 'rb').read()
        data = pkgutil.get_data(__name__, 'resources/dmidecode-basic')

but only the first one is working for me. I have

Copy code

python_tests(
    name='test_intake',
    sources=globs('test_intake.py'),
    dependencies=[
    ],
  resource_targets=[
    ':resources',
  ],
)
resources(
  name='resources',
  sources=globs('resources/*'),
)

curved-manchester-66006

02/08/2016, 5:02 PM

Where "not working" == returns

None

, which get_data does in 3-4 cases

fancy-queen-20734

02/08/2016, 5:34 PM

@enough-analyst-54434: wrt native support in the JVM, JNI’s being superseded by JNR which is less painful and more performant.

curved-computer-88379

07/12/2023, 7:22 PM

Hey I’m new to pants and have never worked with a monorepo tool before. I am starting a monorepo that currently has folders

src/

and

projects/

where

src/

contains source code and

projects

contain individual projects that may or may not source from

src

. I was thinking that each project either builds an image from the Dockerfile within the project directory, or from the root directory base

Dockerfile

that will likely be compatible across multiple projects. What are some steps I should take to integrate python builds / services that are currently deployed in production using docker, but integrating pants so I can source shared code (versioned) across projects. Example: Say I update the shared code

~/src/utils.py

from version 0.1 to 0.2, but I don’t want to have to worry about maintaining project_1 once it’s been deployed (e.g. if

utils

version 0.2 breaks project_1, so project_1 requires utils version 0.1) For example

Copy code

project_1:
  run.py <- sources from /app/src/utils.py version 0.1
  Dockerfile
  requirements.txt
  ...

project_2:
  run.py <- requires /app/src/utils.py version 0.2 since utils.py was updated
  Dockerfile
  requirements.txt
  ...

powerful-scooter-95162

07/13/2023, 12:32 AM

Hi, I'm having some trouble with pytorch & pants (and just with package management tooling in general lol); I have this line in my Pipfile to install a specific wheel:

Copy code

torch = {version="2.0.1", file = "<https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp311-cp311-linux_x86_64.whl>", platform_system = "== 'Linux'", platform_python_implementation = "== 'CPython'", platform_machine = "== 'x86_64'", python_version = "== '3.11'"}

And that is sufficient for pipenv to generate a lockfile, however when I run

Copy code

pants generate-lockfiles

I get this error:

Copy code

ERROR: No matching distribution found for torch==2.0.1+cu118

Which seems to imply pants is trying to find the dep on pypi (where it is not available), rather than using the specified wheel. I also tried setting version="2.0.1+cu118" in my Pipfile, but that did not help.

melodic-lighter-17242

07/13/2023, 7:46 AM

Hey, Any chance someone can review this PR? On a different matter, I’m seeing an unexpected behavior when running

pants --no-pantsd --no-watch-filesystem --changed-dependents=transitive --changed-since=<commit-sha> publish

I have two packages, X and Y, where X consumes Y (

python_sources()

of Y is declared as a dependency in the

python_sources()

of X). I also have a custom plugin that intercepts usages of

python_artifact

(as part of

python_distribution

) When I run the command, although only X changed, I see my plugin runs on both X and Y. Is that intentional? (I think the

package

step runs on both, while

publish

only runs on X) Thanks!!

acoustic-library-86413

07/13/2023, 7:54 AM

Is there some configuration I can use to force a certain set of tests to run one after the other AND in the same batch with Pants, even when using

pants test ::

? I have one project whose tests I don't have time to refactor for the time being. Setting

batch_compatibility_tag

on the tests does not solve the problem, but running them with

--debug

does. The issue is that they re-use a service running in a Docker container. I can't dynamically change the environment variable used inside the container in order to facilitate usage of PANTS_EXECUTION_SLOT, and as a result I need them to run as if I ran

pytest

normally.

melodic-exabyte-60253

07/13/2023, 10:10 AM

I know this issue is not related to Pants, but it seems that the Toolchain website is down: http://toolchain.com

modern-manchester-33562

07/13/2023, 3:18 PM

Hi! So in the

pants.toml

we can configure

resolves_to_no_binary

and

resolves_to_only_binary

. This is great, but I'm not getting my particular use case to work. Let's say I generally want only binaries to be resolved but in the rare cases of that there is no binary, then and only then, I want to allow sources. Unfortunately, setting the following didn't work:

Copy code

[python.resolves_to_only_binary]
__default__ = [":all:"]

[python.resolves_to_no_binary]
__default__ = ["stringcase"]

Is this possible at all?

late-keyboard-89314

07/13/2023, 4:03 PM

I’m hitting a bit of a wall with a seemingly trivial Docker image build. I have a Go binary target which I’m trying to toss into a Docker image, and when running

pants package

I get the following out of the subprocess stderr:

Copy code

ERROR: use `docker --context=default buildx` to switch to context "default"

I’m not seeing much in the pants log that looks weird, and a cursory google search suggests that this problem is normally related to the

DOCKER_HOST

DOCKER_CONTEXT

env vars. I’m using pants v.216.0, and docker desktop on OSX; v4.21.1 (docker engine 24.0.2). Here’s a

-ldebug

full output: https://gist.github.com/jcrumb-u21/b2836d74bd70d551f0b06ee90e2aaf19 and my full Dockerfile: https://gist.github.com/jcrumb-u21/dd907855b89fa6730e55fb27f829d1f4. I can see that the dependency inference is working and my Go binary is being built at least which is pretty sweet!

✅ 1

average-breakfast-91545

07/13/2023, 5:13 PM

IF, hypothetically, I did want to use psycopg2 on Linux, but psycopg2-binary on Mac, what's the approach? Do I need two resolves?

bland-lion-5427

07/13/2023, 6:57 PM

I'm looking to upgrade a repo from an ancient 1.6.0 version of pants to ideally the newest 2.x version of pants, so first I'm trying to go through 1.30.4 per the guidance here, any clue why this error will crop up when I upgrade to 1.30.4?

Copy code

$ ./pants test.pytest src/test/python::
New python executable in /Users/bmahler/.cache/pants/setup/bootstrap-Darwin-arm64/pants.95cc9e/install/bin/python2.7
Also creating executable in /Users/bmahler/.cache/pants/setup/bootstrap-Darwin-arm64/pants.95cc9e/install/bin/python
Installing setuptools, pip, wheel...done.
Looking in links: <redacted>, <redacted>
Collecting pantsbuild.pants==1.30.4
  Downloading <redacted>/pantsbuild.pants-1.30.4.tar.gz (4.8MB)
    100% |████████████████████████████████| 4.8MB 8.4MB/s
  Installing build dependencies ... done
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    IOError: [Errno 2] No such file or directory: '/private/var/folders/hj/4zbxnmpd4nx174gsg9ly4lpr0000gn/T/pip-install-6nrE5f/pantsbuild.pants/setup.py'

purple-plastic-57801

07/14/2023, 1:54 AM

I am current transpiling a front end react app with a shell command. I need to put these assets into a pex binary… is there a way I could have the output directories of the shell command be a resource? I tried to put the output dir as a resource source but it complained.

✅ 1

powerful-eye-58407

07/14/2023, 7:29 AM

Hello, is there a way to pass "--add-host=host.docker.internal:host-gateway" to docker_environment used for running tests? We have a set of component tests, started by bash script, that 1) runs services using docker compose, exposing tested service on a port that's accessible from host OS 2) triggers

pants tests path/to/tests::

with BUILD file for tests containing env variables like

SERVICE_HOST=hostname

that are read by tests and used to connected to tested service run by compose. Now, those component tests work in docker_environment, which means they are run in docker container with network mode set to bridge by default probably? On Mac, using docker desktop, it's enough to set

SERVICE_HOST=host.docker.internal

to make the connection work. On linux, I need to pass custom option

"--add-host=host.docker.internal:host-gateway"

to docker run. There's pants.toml docker.run_args section, but it looks like this is only used when I do

pants run path/to/docker_target

and not when running environments?