Hello, I am very new to pants and am trying to create a python 2.7 package so I set

interpreter_constraints=[">=2.7"],

in my

python_distribution

target. running pants package results in this error:

ProcessExecutionFailure: Process 'Run setuptools.build_meta:__legacy__ for 'scm:scm' failed with exit code 1.
stdout:

stderr:
Traceback (most recent call last):
  File "/private/var/folders/tx/j170snrj51x7vsqlr6gjgknm0000gn/T/pants-sandbox-n0BU4K/chroot/../.cache/pex_root/venvs/1286fe9e47a1c2e7c2f0b8f40834df0d199c016c/bbf665a577dacddf63ce4c741b47906e31dbdb6c/pex", line 230, in <module>
    exec ast in globals_map, locals_map
  File "backend_shim.py", line 21, in <module>
    os.makedirs(dist_dir, exist_ok=True)
TypeError: makedirs() got an unexpected keyword argument 'exist_ok'

Probably because 2.7 doesn't have the

exist_ok

arg. Couldn't find anything online about how to get this to work, anyone have any ideas what i'm doing wrong? This is a 2 and 3 compatible package, builds fine if i change it to

>=3.7

Is it possible to set the following setting to a different value for a specific source root (except for using the corresponding CLI flag for running targets in that source asthat does not scale well)?

[python-infer]
string_imports_min_dots = 0

Hi there. When I try to run

./pants package ::

I'm receiving the following Python exception:

13:37:08.36 [INFO] Completed: Building dockerfile_parser.pex from dockerfile-parser_default.lock
13:37:08.36 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Building dockerfile_parser.pex from dockerfile-parser_default.lock' failed with exit code 1.
stdout:

stderr:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)

  ProcessExecutionFailure: Process 'Building dockerfile_parser.pex from dockerfile-parser_default.lock' failed with exit code 1.
...
ModuleNotFoundError: No module named '_sqlite3'

Pants bootstrapped with Python 3.9, but the the above traceback shows that Pants is using Python 3.8 when I run it. This normally wouldn't be a problem, but my system default (3.8) doesn't include sqlite3. Any idea how to get around this and force Pants to use 3.9, which does include sqlite3?

Hello ladies, gentlemen; a few of you may remember my regular streak of questions here 😇 My engagement as a Tweag consultant for Kaiko is coming to an end next week and I would like to thank everyone that helped me while I introduced Pants at Kaiko. Big shoutout to @wide-midnight-78598, @enough-analyst-54434, and @happy-kitchen-89482 🫶 I introduced Pants and we are now successfully linting, formatting, publishing, and testing our code with Pants (we even wrote our custom plugins!); however there are also friction points that remain: 1. I wasn't able to decommission the non-Pants CI, because we use pytorch and pytorch doesn't play nice with multiplatform (MacOS+Linux in our case) lockfiles. See e.g. this thread for the current solution, which I don't like 😞 I know pytorch doesn't make the life of tooling easy, but thought it was worthwhile to share. This blocked us from typechecking our code with Pants. We still do typechecking in the non-Pants CI 🤷 2. There is a choice to be made between a global requirements.txt files, for ease of Pants use; and multiple requirements.txt file, for flexibility of code writers. Kaiko had a preference for the latter, but Pants made it easier for the former; so we are currently using a global requirements.txt file. However, I believe the team did not yet fully understand the consequences, and so it may be hard to stay in this situation in the long run. @happy-kitchen-89482> I think it's worth you reach out to the Kaiko team in a few months, to see how they are doing 😉 Contact Robert Berke. To the rest of the readers, if you need some development/productivity developer/CI work with a focus on quality and excellent communication, reach out to me and Tweag! I'll stay on this Slack for the foreseeable future.

Hi, I was trying to build a docker target and I am getting the following error:

#3 [internal] load metadata for <http://docker.io/library/python:3.8-buster|docker.io/library/python:3.8-buster>
#3 sha256:88586e40fdfcc6826fc75a7a9653c2727184be908b30c80bf7580390d6713f00
#3 ERROR: rpc error: code = Unknown desc = error getting credentials - err: exec: "docker-credential-dev-containers-e0fbf732d956a2c0412935aab1cc18ff0f3b4638": executable file not found in $PATH, out: ``

It fails here:

FROM python:3-8-buster

I don't understand why this happens because using

docker build

works fine. (I am running this inside a devcontainer)

hi, i think this was in docs, but I can’t seem to find it now. Does the pants lockfile mechanism validate the transitive dependencies of third party libraries? E.g. if my

3rdparty/requirements.txt

contain

req1

that say depends on

PyJWT>1

and

req2

depends on

PyJWT<1

will pants error out during the lock generation?

Has anyone example of a docker file build for AWS lambda with a container image python 3.10 (or 3.11)? (no official support yet https://github.com/aws/aws-lambda-base-images/issues/31) I have done this successfully using poetry packages and awslambdaric, but I am having some issues getting this working with PEX files.

What's the easiest way to glob up a bunch of dependencies for a target? I'd like to do something like

dependencies=["//my/dir/**/*_my_file.py"]

, but I keep getting hit with various forms of

DifferingFamiliesError

. I'd like to avoid having to put one

python_sources

really high up that globs like

sources=["//my/dir/**/*_my_file.py"]

if possible.

Have there been any examples of writing a python source that gets a binding from a C++ implementation? I know bazel supports this, but I'm guessing Pants would be difficult without dedicated C++ support.

Hi there, I’m relatively new to pants ecosystem. Has anyone faced the issue with pants failing to build wheels for grpcio in M1? If yes, how did you guys solve it?

Hi, please give a lot more detail, including the command you ran, the relevant parts of relevant BUILD files and requirements files (e.g., requirements.txt/pyproject.toml) and the error message you got (running with

--print-stacktrace

)

Hi Sathvik. This is a shot in the dark, but I've had* issues building

grpcio

on M1 with pants. I had to add

[subprocess-environment]
env_vars = [
  'GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=TRUE',
  'GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1',
  'GRPC_BUILD_WITH_BORING_SSL_ASM=',
]

to my

pants.toml

Aha, thanks @silly-queen-7197, that is likely the issue. Based on the stack trace @future-crowd-69462 DMd me, the error is a compilation error due to attempting to building the sdist, which requires various env vars to be plumbed through to the sandbox.

Hi! I’m evaluating Pants for a potential polyglot monorepo CI/CD solution for managing the build graph and only rebuilding what changed, inferring from git history. My main question is will I be able to use Pants also for my TypeScript projects? According to https://www.pantsbuild.org/page/language-support there is no built-in parsing of source files to tell granularly what changed. Would I be able to do at least a course grained per-project check? Let’s say I have two packages: a library and an app. If the lib changes, both must rebuild. If the app changes, only the app needs to rebuild. If the library changed, I want to publish to a private npm registry. If the app changed, I want to build a docker image and push to a private container registry. Is this possible and how do I 1. build/test etc the right projects? 2. publish to correct registries? a. and how do I authenticate? I’m looking to run this in GitHub Actions primarily. Thanks in advance for any incoming help! 🙏

Is anyone experiencing that typecheck (mypy) fails due to missing

types-six

whenever switching branches or in CI like GitHub Actions? This seems to be resolved if I run the same operation twice, but becoming annoying as CI also suffers from the same issue. (I'm using Pants 2.16.0dev5) Maybe some

pex

issue?

Thanks @silly-queen-7197 @happy-kitchen-89482. I have added the env variables but still this error keeps recurring. Not really sure what’s been happening though. These are my env variables if that could help

env_vars = [
    "SSH_AUTH_SOCK",
    "GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1",
    "GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1",
    "GRPC_BUILD_WITH_BORING_SSL_ASM=1",
]

Are you able to install that package on your M1 using

pip

?

Basically without access to your system, or at least a LOT more information than you've provided, it'll be hard to figure out. So you'll have to do some more digging here. The basic framework of the problem is this: There is no published wheel for grpcio on M1. So Pants has to build the wheel from the sdist (which it does by delegating to Pip).

And that build is failing. So one of two things may be happening: 1) If this does build when you install it via Pip directly, then the difference is that Pants runs Pip in an isolated sandbox, so not all env vars are available, and you have to punch them through as @silly-queen-7197 explained. 2) If this doesn't build even directly under Pip, then you have to debug that first, and then revert to 1).

https://pantsbuild.slack.com/archives/C046T6T9U/p1676817863986189?thread_ts=1676735917.484029&cid=C046T6T9U Yes, I am able to install that package in my M1 with pip.

Have tested building my pants package by adding the env variables as suggested by @silly-queen-7197. No luck with that either.

So this is where you start digging into what is different between the sandboxed pip run that Pants does (via Pex) and the unsandboxed standalone pip run. It's very likely to be one or more missing env vars

for example, try adding PATH to that list

If you run with

--keep-sandboxes=on_failure

then Pants will log the location of the sandbox that failed, and you can cd into it and play around

there is a

__run.sh

in the sandbox that will rerun the process

So you can tweak env vars, then

./__run.sh

until things work

Actually, I think running

./__run.sh

manually will have access to your entire environment

so things might work

and then you have to figure out which env vars are missing when Pants runs in the sandbox