pantsbuild #general

Just getting started - I setup a small example repo to capture what I think I want to do, but where I'm running into questions - https://github.com/rjrudin/pants-monorepo . Hoping someone can help with the initial 4 questions I have in the README file, which - high-level - are: 1) Why are dependencies not shown; 2) Recommendation for running tests in PyCharm; 3) Controlling version of Pytest; 4) How to do what I used to do via Gradle with app/lib dependencies .

refined-addition-53644

01/14/2022, 3:27 PM

I have a general question around how do you onboard your colleagues at work when you introduce a new tool e.g. pants in my case. I wrote a wiki and had a meeting/tutorial, but I still feel like that people will be bit lost when they start using it. I work in a data science team and I feel like not everyone is so inclined to handle the engineering complexity that comes with introducing such tools.

☝️ 1

flat-zoo-31952

01/14/2022, 5:35 PM

Is cross-compilation support on the roadmap for Go integration?

witty-crayon-22786

01/14/2022, 6:36 PM

ok, last chance, for real this time! the close of submissions has been bumped to first thing Monday morning, when we’ll switch to voting. get your ideas in! https://pantsbuild.slack.com/archives/C18RRR4JK/p1641338890053700

👆 2

💡 2

flat-zoo-31952

01/16/2022, 9:00 PM

Is there a way in general in a
dependencies
field to tell the engine "Everything you find with dependency inference, _plus this list_"? Nevermind, this is what it does by default:

To add an explicit dependency, add the target's address to the
dependencies
field. This augments any dependencies that were inferred.

👆 1

flat-zoo-31952

01/16/2022, 10:30 PM

Is there a way to deal with colons in file names? It seems to throw some commands through a loop (in this case the file is named `path/to/wip:_new_rules.yaml`:

Copy code

❯ ./pants --changed-since=HEAD --changed-dependees=transitive list
17:27:29.91 [ERROR] 1 Exception encountered:

  ResolveError: The file or directory 'path/to/wip' does not exist on disk in the workspace, so the address 'path/to/wip:_new_rules.yaml' cannot be resolved.

refined-addition-53644

01/17/2022, 7:28 AM

I am seeing warning message from PEx when using pants in our CI/CD platform on AWS using codebuild. Please see the 🧵 for details

clean-city-64472

01/17/2022, 5:55 PM

I''ve checked docs and searched this Slack and can't see how to redirect terminal output of a pants run to a file. Using

./pants test  my/path/to/test.py -l=debug > /tmp/test.log

doesn't seem to work.

echoing-london-29138

01/18/2022, 5:17 PM

Hi Here, I’m trying to use pants on a docker, but I keep to get this strange error:

Copy code

#12 0.335 Bootstrapping Pants using /usr/bin/python3.8
#12 0.339 Creating the virtualenv PEX.
#12 0.340 Downloading the Pex PEX.
#12 0.348   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
#12 0.348                                  Dload  Upload   Total   Spent    Left  Speed
100   640  100   640    0     0   6736      0 --:--:-- --:--:-- --:--:--  6666
100 3609k  100 3609k    0     0  5631k      0 --:--:-- --:--:-- --:--:-- 5631k
#12 1.033 SHA256 fingerprint of <https://github.com/pantsbuild/pex/releases/download/v2.1.62/pex> verified.
#12 4.946 Installing pantsbuild.pants==2.9.0rc2 into a virtual environment at /root/.cache/pants/setup/bootstrap-Linux-aarch64/2.9.0rc2_py38
#12 5.675 created virtual environment CPython3.8.10.final.0-64 in 384ms
#12 5.675   creator CPython3Posix(dest=/root/.cache/pants/setup/bootstrap-Linux-aarch64/pants.giAPtK/install, clear=False, no_vcs_ignore=False, global=False)
#12 5.675   seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
#12 5.675     added seed packages: pip==21.1.2, setuptools==57.0.0, wheel==0.36.2
#12 5.675   activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
#12 6.283 Requirement already satisfied: pip in /root/.cache/pants/setup/bootstrap-Linux-aarch64/pants.giAPtK/install/lib/python3.8/site-packages (21.1.2)
#12 6.381 Collecting pip
#12 6.434   Downloading pip-21.3.1-py3-none-any.whl (1.7 MB)
#12 6.515 Requirement already satisfied: setuptools<58 in /root/.cache/pants/setup/bootstrap-Linux-aarch64/pants.giAPtK/install/lib/python3.8/site-packages (57.0.0)
#12 6.689 Collecting setuptools<58
#12 6.711   Downloading setuptools-57.5.0-py3-none-any.whl (819 kB)
#12 6.766 Installing collected packages: setuptools, pip
#12 6.766   Attempting uninstall: setuptools
#12 6.766     Found existing installation: setuptools 57.0.0
#12 6.811     Uninstalling setuptools-57.0.0:
#12 6.817       Successfully uninstalled setuptools-57.0.0
#12 7.094   Attempting uninstall: pip
#12 7.094     Found existing installation: pip 21.1.2
#12 7.250     Uninstalling pip-21.1.2:
#12 7.258       Successfully uninstalled pip-21.1.2
#12 7.752 Successfully installed pip-21.3.1 setuptools-57.5.0
#12 8.153 ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.9.0rc2 (from versions: 0.0.17, 0.0.18, 0.0.20, 0.0.21, 0.0.22, 0.0.23, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.39, 0.0.40, 0.0.41, 0.0.42, 0.0.43, 0.0.44, 0.0.45, 0.0.46, 0.0.47, 0.0.48, 0.0.49, 0.0.50, 0.0.51, 0.0.52, 0.0.53, 0.0.54, 0.0.55, 0.0.56, 0.0.57, 0.0.58, 0.0.59, 0.0.60, 0.0.61, 0.0.62, 0.0.63, 0.0.64, 0.0.65, 0.0.66, 0.0.67, 0.0.68, 0.0.69, 0.0.70, 0.0.71, 0.0.72, 0.0.73, 0.0.74, 0.0.75, 0.0.76, 0.0.77, 0.0.79, 0.0.80, 0.0.81, 0.0.82, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.5.0, 1.6.0, 1.7.0)
#12 8.153 ERROR: No matching distribution found for pantsbuild.pants==2.9.0rc2

Here my docker file :

Copy code

# GET the Ubuntu Image from public.ecr.aws/ubuntu/ubuntu (ubuntu oficial ECR) to avoid Docker Hub Limits
FROM public.ecr.aws/ubuntu/ubuntu:20.04

RUN apt-get update

ENV PYTHONUNBUFFERED 1
ARG DEBIAN_FRONTEND=noninteractive

ENV SRC_PATH=./mlp_backend
ENV CONTAINER_CODE_PATH=/code
ENV CONTAINER_BUILD_PATH=/build
RUN apt-get update && apt-get --assume-yes install \
	git \
	xmlsec1 \
	libpq-dev \
	python3-setuptools \
	poppler-utils \
	build-essential \
	python3-dev \
	python3-pip \
	python3-setuptools \
	python3-wheel \
	python3-cffi \
	python3-distutils \
	libcairo2 \
	libpango-1.0-0 \
	libpangocairo-1.0-0 \
	libgdk-pixbuf2.0-0 \
	libffi-dev \
	libcurl4-openssl-dev \
	libssl-dev \
	shared-mime-info \
	supervisor \
	curl \
	--no-install-recommends \
	&& apt-get clean && rm -rf /var/lib/apt/lists/*

RUN mkdir $CONTAINER_BUILD_PATH
COPY ./src $CONTAINER_BUILD_PATH/src
COPY ./mypy_lockfile.txt ./mypy.ini ./pants ./pants.toml ./pytest_lockfile ./pytest.ini $CONTAINER_BUILD_PATH

RUN cd $CONTAINER_BUILD_PATH && curl -L -O <https://static.pantsbuild.org/setup/pants> && chmod +x $CONTAINER_BUILD_PATH/pants
RUN $CONTAINER_BUILD_PATH/pants --version
RUN $CONTAINER_BUILD_PATH/pants package ::
RUN cp -r $CONTAINER_BUILD_PATH/dist /dist/

enough-analyst-54434

01/18/2022, 5:50 PM

bootstrap-Linux-aarch64

-> We don't publish Pants for that architecture yet: https://pypi.org/project/pantsbuild.pants/2.9.0rc2/#files

enough-analyst-54434

01/18/2022, 5:51 PM

You can follow-along, upvote, or even pitch in here: https://github.com/pantsbuild/pants/issues/12183

👍 1

few-arm-93065

01/18/2022, 7:50 PM

Hey all, can anyone point me to the ticket where "interpreter_search_paths" was removed in 2.10.0.dev1? I am trying to find out why and what the alternatives are, and I am having trouble finding it via github search

high-energy-55500

01/18/2022, 8:26 PM

a little open-ended question here, but has anyone successfully been able to get pants working with tensorflow on M1? it looks like neither pants nor pip are able to install tensorflow normally:

Copy code

martim@Argon allpdl % ./pants run src/app:analysis-cli -- standardize 3577                                      
19:08:09.21 [INFO] Completed: Building requirements.pex with 27 requirements: Unidecode<2.0.0,>=1.2.0, arn<0.2.0,>=0.1.5, boltons<21.0.0,>=20.2.1, boto3<2.0.0,>=1.15, botocore<2.0.0,>=1.21.60, click<8.0.0,>=7.1.2, contractions<0.0... (540 characters truncated)
19:08:09.21 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
[...]
    raise ExecutionError(

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building requirements.pex with 27 requirements: Unidecode<2.0.0,>=1.2.0, arn<0.2.0,>=0.1.5, boltons<21.0.0,>=20.2.1, boto3<2.0.0,>=1.15, botocore<2.0.0,>=1.21.60, click<8.0.0,>=7.1.2, contractions<0.0.59,>=0.0.58, dataclasses<0.9.0,>=0.8; python_version < "3.7", django-taggit<2.0.0,>=1.1, django<4.0.0,>=3.1.1, elasticsearch==7.12.0, emoji<2.0.0,>=1.6, jsonschema<4.0.0,>=3.0, marshmallow-dataclass<9.0.0,>=8.2.0, nameparser<2.0.0,>=1.0.6, nltk<4.0.0,>=3.6.5, orjson<4.0.0,>=3.3.1, phonenumbers<9.0.0,>=8.12, psycopg2-binary<3.0.0,>=2, redis-py-cluster<3.0.0,>=2.0.0, requests<3.0.0,>=2.24, sentry-sdk[flask]<2.0.0,>=1.5.0, slackclient<3.0.0,>=2.9, tensorflow==2.5.1, titlecase<2.0.0,>=1.1.1, toposort<2.0.0,>=1.6, usaddress<0.6.0,>=0.5.10' failed with exit code 1.
stdout:

stderr:
ERROR: Could not find a version that satisfies the requirement tensorflow==2.5.1
ERROR: No matching distribution found for tensorflow==2.5.1

pip:

Copy code

martim@Argon allpdl % pip install tensorflow
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)

it seems that the fix requires installing miniforge (see here or here), but: 1. it appears to require using conda to create virtualenvs instead of pyenv+pants, 2. it requires pip-installing

tensorflow-macos

and

tensorflow-metal

instead of

tensorflow

, and i’m not sure it’s possible to create that kind of conditional

pyproject.toml

, even assuming it’s possible to get this working

bitter-ability-32190

01/18/2022, 11:28 PM

Since the remote cache key is influenced by PATH and friends, could remote caching work for multiple dev boxes if Pants was invoked via a script that scrubbed the relevant vars? 🧐

quiet-painter-18838

01/19/2022, 12:05 AM

Hi, I have encountered mypy error.

Copy code

Duplicate module named "src"

Let me share my situation in the thread.

few-gigabyte-4152

01/19/2022, 1:18 AM

Is there a way to configure pants to use scalafix to update from scala 2.12 to 2.13?

echoing-london-29138

01/19/2022, 8:23 AM

Thank you for your answer 🙂

curved-television-6568

01/19/2022, 2:33 PM

I’ve recently noticed that when I have conflicting dependencies, and pip loggs something like 🧵

flat-zoo-31952

01/19/2022, 5:43 PM

Is it possible or reasonable to have BUILD files in directories other than the ones they describe? Like to have some remapping of BUILD files to other paths? This could be used e.g. to maintain a shadow directory of BUILD metadata. Something like where

.build/src/module/BUILD

would map to

src/module/BUILD

hundreds-father-404

01/19/2022, 5:45 PM

I don't think I understand the motivation - what's the value of a shadow directory?

flat-zoo-31952

01/19/2022, 6:04 PM

"centralized" management of pants-related things

👍 1

flat-zoo-31952

01/19/2022, 6:07 PM

I don't think this is possible currently... there seems to be a pretty strong notion of... a target's address comes from the location of the BUILD file it is defined in

➕ 1

hundreds-father-404

01/19/2022, 6:09 PM

Yeah, not possible currently. A target can only "own" source files in the current dir or subdirectory

hundreds-father-404

01/19/2022, 6:11 PM

In the meantime, the best workaround is probably to use a single BUILD file at the root of your project with

**

recursive globs. We don't generally recommend that per https://www.pantsbuild.org/docs/targets#target-generation, but it is totally feasible to do especially thanks to the

overrides

field added in 2.8 (If you haven't read that page in the past two months, I encourage re-reading it. It was rewritten with some conceptual changes in 2.8)

👍🏻 1

happy-kitchen-89482

01/19/2022, 6:14 PM

I got an article published in SD Times, about monorepos: https://sdtimes.com/softwaredev/the-monorepo-approach-to-code-management/

👖 7

👍 4

🙌 4

adventurous-judge-80222

01/19/2022, 6:59 PM

Hello, I'm evaluating pants as a build solution for my team and I wanted to know your roadmap for the docker backend. Its currently labeled

pants.backend.experimental.docker

does this mean it is in beta? When will it go GA?