Hitting an issue trying to set up a docker build o...
# general
Hitting an issue trying to set up a docker build on Mac OS ARM (pants
Copy code
There was 1 error downloading required artifacts:
1. dockerfile 3.2 from <https://files.pythonhosted.org/packages/6e/75/1395fac29bd2dcccb51ffae11ba9a19fe159eabf9596bca4cf49c1a2b191/dockerfile-3.2.0.tar.gz>
    ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
This seems to be coming from bootstrapping the docker backend. Where can I find the logs of this process to keep troubleshooting? I’ve got
Hey, did you find a solution here? Someone is reporting something similar on a different requirement, after upgrading to 2.14
I'm wondering if this is an artifact of upgrading Pants (it shouldn't happen, so it's still a bug if so, but wondering if that is the immediate cause)
Can you run with
and see if that gives more info?
@enough-analyst-54434, where can we find the pip logs in this case?
You need to pass
to Pex. It will print the path to stderr and retain it.
Hmm, Pants does not do this today, so ... damn
No solution yet here. Will try to dig into it more today and see if I can get some more helpful logs.
At least run with
. And I wonder if this has to do with Python versions/interpreter constraints
E.g., it's trying to isntall
on an unsuitable interpreter somehow
Here’s what I think is the most relevant excerpt from running with
and `--print-stacktrace`: https://gist.github.com/jcrumb-u21/7163d47b523c4753c9b8f13a80582ce1
If I have time today I’ll poke at the pants code and add the flag John mentioned to the pex call.
Right on the tin it says you need go tooling to build the sdist: https://pypi.org/project/dockerfile/ Do you have that?
I do
Ok ... does that require env vars to work that are not configured to allow leaking through to Pants actions? GO_ROOTS, etc? It's been a while for me.
I don’t believe so, but let me double check I’m able to install an sdist of that package with the same interpreter with
env -i.
I'm pretty sure this is just down to getting a macOS aarch64 wheel built for the dockerfile sdist; and that's trickier in Pants hermeticity shrink wrap.
@#$%! macOS arm
pip install dockerfile
will pull a wheel on macOS ARM, and
pip install --no-binary :all: dockerfile
worked for me as well. Trying it now under
env -i
to see if
is the problem.
Nothing but trouble from M1s!
Whoa - seriously? There exists a pre-built dockerfile M1 wheel? Looking ...
Let me try with python 3.7 since that’s what pants is picking
Yeah - that's the issue
You need 3.8 to get lucky.
since its abi3 though ... hrm, let me see if there is a
bug re this.
And @late-keyboard-89314 your on macOS >= 12 ?
I did manage an sdist install of
with the same interpreter pants selected.
Yeah, running ventura
Ok, well your idea to use
was correct. You need to poke at the
script there and get your hands dirty finding out what's different. Editing that script will allow insertion of things like
, etc.
Although, wait a sec. Why are you using Pants 2.16.x? This:
Copy code
10:49:30.95 [DEBUG] spawned local process as Some(75453) for Process { argv: ["/Users/jay/.pyenv/versions/3.7.11/bin/python3.7", "./pex", "--tmpdir", ".tmp", "--jobs", "6", "--pip-version", "20.3.4-patched", ...
Says you're not using it for the
[python] pip_version = "22.3"
Which ... would be good to try to rule out old
, etc.
And https://gist.github.com/jcrumb-u21/7163d47b523c4753c9b8f13a80582ce1#file-pants-debug-L13 definitely does not have
in it's env; so ignoring why the abi3 wheel is not getting selected, that explains why the sdist fails to build I think.
But! ... remembering my abi3 better - I think abi3 says any Python 3 >= the one the wheel was built with, which 3.7 is not. Let me check that.
Yeah, an abi3 wheel is compatible with the version of Python it was compiled with and subsequent; so that macOS arm wheel on PyPI is only good for >=3.8
If you really need to be using Python 3.7, you'll need to confront the sdist build / go tooling requirement thicket.
On last thing - when debugging using
keep in mind that it is not hermetic like the actual Pants process execution is. It will leak through your environment! So if the
command works, try then blanking out select env vars and seeing if it still works, although this may be complicated by successful caching of wheel builds in
Why are you using Pants 2.16.x?
I need a bugfix that hasn’t hit the 2.14/2.15 release branches yet.
If you really need to be using Python 3.7
I don’t need to be using Python 3.7 at all, it just happens to be on my system and is what pants is selecting when installing this tool. This project actually doesn’t use Python at all at the moment, and the backend isn’t enabled:
Copy code
pants_version = "2.16.0.dev3"
backend_packages = [
It would be perfectly fine in my case for pants to select python 3.8 and use the wheel for this package.
Aha, excellent. Just a sec for steps.
You want the following in `pants.toml`:
Copy code
interpreter_constraints = ["CPython>=3.7,<4"]
lockfile = "anywhere/you/want/dockerfile-anyname-really.lock-but-any-extension-or-none
The 2 things to change there are: 1. The ICs: I presented the default, and that's what's killing you. You probably want to change
to at least `<3.12`but the tighter the better on the upper bound and clearly you want to bump the lower bound. 2. The lockfile path: as should be obvious Then run
./pants generate-lockfiles --resolve dockerfile-parser
, then try to use Pants again.
As an aside to @happy-kitchen-89482 - this presents an interesting class of cases: Python tooling being used for not-Python backend... should lock to scie-pants shipped Python versions to avoid all this. In the current case that would have been 3.9 and there would have been no issue here.
Even in the current world, this IC should match Pants own and be
since the tool does not care about any user Python code, its just a tool to be used internally by rules.
Heyyy, that worked. Tightened the constraint to
and it worked (presumably because it pulled the wheel). I have such a love-hate thing going with ARM Macs.
Thanks for digging into this with me!
You should direct the hate at Apple. The reason things are hard for Apple is they don't allow commodity rackable servers. It all rolls down hill from there. It is expensive in time and $$$ to get OSS projects working for mac arm.
And one way to direct that hate is not buy the damn things!
Yeah, I mean it’s a great platform from a performance/battery life perspective when things work but they certainly don’t make it easy for people to get things working.
They are simply not developer friendly
Only for developing for their own ecosystem of course.
Yet more evidence that we should not ship with loose default interpreter constraints, and force you to pick them explicitly for your repo (and subsequently for tools)
On my tombstone will be engraved "Died of CPython>=3.7,<4"
Universal locks are a horrible idea. They let you not think too easily.