https://pantsbuild.org/ logo
#general
Title
# general
r

rapid-flag-11972

10/08/2020, 4:09 PM
Hello, I’m writing a CI pipeline with gitlab and pants and I’m not able to get linting or formatting to work. I have ensured I have pants installed and working as
./pants --version
gives me the version I’m running (
1.30.0rc0
). When I run
./pants --print-exception-stacktrace fmt ::
pants tells me if changes were made by black, docformatter and isort as it should. So I know that pants is able to format/lint code. However after this I get the following output:
Copy code
14:36:33 00:00 [main]
               (To run a reporting server: ./pants server)
14:37:04 00:31   [setup]
14:37:04 00:31     [parse]
               Executing tasks in goals: bootstrap -> fmt
14:37:04 00:31   [bootstrap]
14:37:04 00:31     [substitute-aliased-targets]
14:37:04 00:31     [jar-dependency-management]
14:37:04 00:31   [complete]
               FAILURE
FAILURE: Problem locating a java distribution: Failed to locate a JDK distribution with minimum_version None, maximum_version None
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 1
I have ensured that my custom docker image has java and is working as I am able to do
java -version
during the same job. Here is the output when checking if java was installed:
Copy code
$ java -version
openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment (build 11.0.8+10-post-Debian-1deb10u1)
OpenJDK 64-Bit Server VM (build 11.0.8+10-post-Debian-1deb10u1, mixed mode, sharing)
The project itself is python based but since pants told me it wasn’t able to find a java installation I made a custom image with openjdk-11 installed. Any help is appreciated, thanks!
h

hundreds-father-404

10/08/2020, 4:15 PM
Hey Shreyas! Welcome, and sorry for these issues you’re running into it. What’s going on is that you’re using the old v1 engine, which did a much worse job at isolating what dependencies you need to run the formatters you care about. Indeed, it shouldn’t matter if you have Java installed. Instead, we recommend using
2.0.0b3
, which solely uses the much improved “v2” engine. https://www.pantsbuild.org/docs/installation Even though it’s a beta, we plan to promote to release candidate shortly, and it generally offers a more polished experience than 1.30.1. (I’m not sure how far you got with BUILD files, but a major improvement is “dependency inference”, which means you can leave off lots of boilerplate you’d normally need to add)
r

rapid-flag-11972

10/08/2020, 4:24 PM
Hi Eric, thanks for the quick reply! The reason we’re using V1 is because V2 doesn’t work with gRPC. Here’s some context: https://github.com/pantsbuild/pants/issues/10497
h

hundreds-father-404

10/08/2020, 4:24 PM
Ah, I’m actually working on fixing that today! We will hopefully include it by tomorrow’s next release
In the meantime, I believe this will fix the Java issue:
Copy code
backend_packages.remove = ["pants.backend.jvm"]
r

rapid-flag-11972

10/08/2020, 4:26 PM
That would be amazing! thanks Eric! Ah okay I’ll try the fix for now.
o

orange-beach-75711

10/08/2020, 4:29 PM
@hundreds-father-404 - that's great to hear that v2 might be getting protobuf grpc support soon. I'm really excited about the release since that is the only blocker for us from moving to v2. fyi - I just replied to your question on the linked Github issue.
❤️ 1
h

hundreds-father-404

10/08/2020, 4:31 PM
Thanks! Pardon for the delay with us adding it back. It’s one of the very last blockers for the 2.0 release
@rapid-flag-11972 you might need to also include
['pants.backend.native', 'pants.backend.codegen.protobuf.java', 'pants.backend.codegen.thrift.java', 'pants.backend.codegen.thrift.python']
in
backend_packages.remove
. We had this not-great idea in v1 of activating every backend by default, and it was really difficult for us to deprecate
1
Update: just got gRPC working 🙂 we plan to release 2.0.0rc0 today or this weekend, and it will include both MyPy Protobuf and gRPC support
2.0.0rc0 is released 🙂 Change
pants_version = "2.0.0rc0"
in
pants.toml
. See https://www.pantsbuild.org/docs/protobuf for instructions on gRPC, and https://www.pantsbuild.org/docs/how-to-upgrade-pants-2-0 for a migration guide. We’re happy to help with the upgrade
r

rapid-flag-11972

10/12/2020, 1:51 PM
That’s awesome! thanks a lot Eric 😄
❤️ 1
o

orange-beach-75711

10/13/2020, 8:49 PM
@hundreds-father-404 - I'm currently in the process of upgrading to
2.0.0rc0
from
1.30.0rc0
. I'm hitting a roadblock when I run any pants goals. • Here is the ldebug output of
./pants fmt ::
- https://pastebin.com/su8tGdWJ • Here is our pants.toml file - https://pastebin.com/YSvGBUXm Any thoughts on what we may be getting wrong?
h

hundreds-father-404

10/13/2020, 8:51 PM
Thanks for the ldebug. It looks like Pants is using the macOS installed Python, which is often horribly broken Try setting the option
[python-setup].interpreter_search_paths
so that system Python is not used: https://www.pantsbuild.org/docs/python-interpreter-compatibility#changing-the-interpreter-search-path
One of my coworkers is having the same issue with you and setting
interpreter_search_paths = ["<PYENV>"]
did not fix it, which I’m trying to help him debug. I suspect the
<PYENV>
expansion isn’t working properly. If that doesn’t work, try setting to an explicit path like
/Users/eric/.pyenv/versions/3.8.5/bin
(Also if it keeps not working, continuing to send
-ldebug
is extremely helpful)
o

orange-beach-75711

10/13/2020, 9:06 PM
Thanks for the quick turnaround Eric! Looks like neither
<PYENV>
, nor setting an explicit path worked. Here's the latest ldebug output - https://pastebin.com/UseybiSR Latest
pants.toml
- https://pastebin.com/4q1zu6dM Fyi, this is my python version -
Copy code
14:05:23 paritosh at Paritoshs-MacBook-Pro-2 in ~/workspaces/phaidra/nabu on feature/pants_upgrade U $ which python
/usr/local/opt/pyenv/shims/python
14:05:24 paritosh at Paritoshs-MacBook-Pro-2 in ~/workspaces/phaidra/nabu on feature/pants_upgrade U $ python --version
Python 3.7.7
h

hundreds-father-404

10/13/2020, 11:15 PM
Okay, so it looks like Pants is setting up everything how we expect it to set it up.
[python-setup].interpreter_search_paths
is properly being applied. However, Pex is not behaving quite like I’d expect. This might take us a little time to figure out, but it’s our highest priority to fix - we’ve gone through several iterations the past two months on getting this right, and looks like it’s still Not Quite Right, gr. One more favor, can you please run the same command with
-ldebug
and
--pex-verbosity=9
? (Warning it will be a ton of output)
o

orange-beach-75711

10/14/2020, 12:19 AM
No worries Eric! Please find the output with
--pex-verbosity=9
attached.
h

hundreds-father-404

10/14/2020, 12:21 AM
Thank you! I see precisely where the issue is in PEX, and it’s easier to fix than I feared. Only now need to verify this is the right fix, that I’m not breaking PEX behavior when you use PEX stand-alone. Will try to get
rc1
out tomorrow with this fixed one way or another :)
In the meantime, this workaround might work, but it will probably simply trade one error for another error. Set
[pex].executable_search_paths = ["/path/to/pyenv_interp/bin", "/bin", "/sbin", "/usr/sbin"]
(without
/usr/bin
, which has the problematic Python interpreter)
o

orange-beach-75711

10/14/2020, 1:19 AM
I surprisingly get the same error as before with the following search paths -
interpreter_search_paths = ["/usr/local/opt/pyenv/versions/3.7.7/bin", "/bin", "/sbin", "/usr/sbin"]
- https://pastebin.com/Y2QXRVGy
h

hundreds-father-404

10/14/2020, 1:20 AM
Not
[python-setup].interpreter_search_paths
, but
[pex].executable_search_paths
o

orange-beach-75711

10/14/2020, 1:32 AM
oh my bad!
that worked!!
I'll go through other goals now and make sure I'm able to get everything else to work. Thanks so much for the live help Eric. Really appreciate it! Will wait for the rc1.
❤️ 1
h

hundreds-father-404

10/14/2020, 1:35 AM
Yay! I think everything will work fine, unless any third-party requirements you have require something discoverable on
/usr/bin
in order to build themselves Fingers crossed they don’t, until we can fix
[python-setup].interpreter_search_paths
@orange-beach-75711 to close the loop, you can upgrade to 2.0.0rc1, then remove your setting for
[pex].executable_search_paths
, and set
[python-setup].interpreter_search_paths = ["<PYENV>"]