https://pantsbuild.org/ logo
#general
Title
# general
q

quick-translator-54728

08/25/2021, 2:22 PM
hey everyone, happy wednesday! i’m not sure if this is where we ask questions, but I was hoping somebody could help me with this issue.. I’m trying to import a project into Intellij and I see errors like this — would ANYBODY possible know how to resolve this? thank you so muchh
👋 1
h

hundreds-father-404

08/25/2021, 2:26 PM
Hi Kati! Definitely the right spot for questions :) Hm, I think there might be two issues here? First is that Pants itself fails to run because it can't resolve the contrib dependency, and second is that causing IntelliJ to not fail. Let's focus on that first problem What happens when you run
./pants --version
?
q

quick-translator-54728

08/25/2021, 2:36 PM
Screen Shot 2021-08-25 at 7.35.58 AM.png
👍 1
h

hundreds-father-404

08/25/2021, 2:45 PM
Cool, thanks. Now how about running
,/pants export
directly rather than using the IntelliJ plugin to run it? I'm curious if this is an issue with how the plugin is invoking things, or pants export never is working for you
q

quick-translator-54728

08/25/2021, 2:47 PM
oh wait actually, when I run
./pants --version
in the repo that i’m having a problem with, I get
h

hundreds-father-404

08/25/2021, 2:48 PM
Okay solid, that will be easier to debug. I suspect your ./pants bash script might be out of date. You can run this to redownload
curl -L -o ./pants <https://static.pantsbuild.org/setup/pants|https://static.pantsbuild.org/setup/pants>
q

quick-translator-54728

08/25/2021, 2:49 PM
should that command take like 0.2 seconds to run lol, I ran it and it finished so fast and the same issue shows up
h

hundreds-father-404

08/25/2021, 2:51 PM
it should be fast, it's only downloading a file. If you run
git status
, does it show changes made to
./pants
?
Also what OS are you using?
q

quick-translator-54728

08/25/2021, 2:54 PM
ah yeah pants and pants.toml
i’m on big sur
h

hundreds-father-404

08/25/2021, 2:54 PM
pants.toml
changed?
q

quick-translator-54728

08/25/2021, 2:54 PM
yep!
h

hundreds-father-404

08/25/2021, 2:54 PM
I wouldn't expect the download to have changed that. How is it different? (
git show
will tell you)
q

quick-translator-54728

08/25/2021, 2:56 PM
oh nevermind, i had previously changed the pants_version value in pants.toml to 1.27.0, so that was my change when i was attempting a solution
w

witty-crayon-22786

08/25/2021, 4:06 PM
do you see any changes made to the
pants
script after running the curl command that Eric gave you?
q

quick-translator-54728

08/25/2021, 4:14 PM
yeah the pants script changed!
w

witty-crayon-22786

08/25/2021, 4:17 PM
ok: and do you have any luck running
./pants --version
with pants 1.27 now?
q

quick-translator-54728

08/25/2021, 4:23 PM
no 😕
Screen Shot 2021-08-25 at 9.23.48 AM.png
h

hundreds-father-404

08/25/2021, 4:25 PM
Kati how about if you temporarily remove the plugins section?
w

witty-crayon-22786

08/25/2021, 4:25 PM
oh. you know what, this isn’t the
pants
script… this is plugin resolution after pants has already loaded.
1
👍 1
q

quick-translator-54728

08/25/2021, 4:26 PM
how do i remove the plugins section?
Screen Shot 2021-08-25 at 9.51.06 AM.png
h

hundreds-father-404

08/25/2021, 4:56 PM
Like that, but run with --no-verify-config
q

quick-translator-54728

08/25/2021, 4:56 PM
it says 1.26.0
h

hundreds-father-404

08/25/2021, 5:12 PM
Okay great, so this confirms that the issue is installing the plugin
pantsbuild.pants.contrib.scrooge
. The issue is not IntelliJ, nor the
./pants
bash script. Now to figure out why that does not work! What is your
[GLOBAL]
section for your original, unmodified pants.toml?
q

quick-translator-54728

08/25/2021, 5:20 PM
Screen Shot 2021-08-25 at 10.20.07 AM.png
h

hundreds-father-404

08/25/2021, 5:22 PM
is there more to that? I'd expect a
plugins
section
q

quick-translator-54728

08/25/2021, 5:58 PM
oh yeah
Screen Shot 2021-08-25 at 10.59.09 AM.png
h

hundreds-father-404

08/25/2021, 6:01 PM
that comment above #plugins was there before? was the value commented out before or that's from your change?
q

quick-translator-54728

08/25/2021, 6:18 PM
i commented it out when you said to remove the plugins section
h

hundreds-father-404

08/25/2021, 6:22 PM
Okay, thanks Kati. I'm trying to understand the comment you all put right above the plugins section. Is there someone at ActionIQ you can ask about it? Maybe try using
git blame
on that line
q

quick-translator-54728

08/25/2021, 7:19 PM
ooh okay lemme ask!
❤️ 1
hi sorry, it took a little bit for people to get back since it was end of the work day
what is it you don’t understand about the comment?
h

hundreds-father-404

08/26/2021, 4:33 PM
What do you mean you don't import plugins? Normally plugins are never directly imported
Also is this working for other people on your team?
c

cold-journalist-66984

08/26/2021, 6:03 PM
Hey! I’m on Kati’s team and this does work for me, but I’m not sure why it’s not working for her. Do you know how it’s trying to resolve the plugins?
h

hundreds-father-404

08/26/2021, 8:23 PM
Hey John! I'm not sure why it's not working either. My first thought is if the Python interpreter or the operating system is not compatible - Pants 1.7.0 is the last version to be released with an "sdist", which allows you to try to build a wheel regardless of your OS/interpreter. Otherwise, you have to have a prebuilt "wheel". The fact that Pants 1.7.0 and earlier shows up in the above error message, but nothing after, is fishy to me. It suggests the platform is not being recognized as compatible Pants 1.26 was released as
pantsbuild.pants-1.26.0-cp36-abi3-macosx_10_11_x86_64.whl
and
pantsbuild.pants-1.26.0-cp36-abi3-manylinux1_x86_64.whl
, which should work with any Python 3.6+ on macOS 10.11 or newer. It's also weird that Pants is installable when downloaded via the
./pants
bash script, only not when via the plugin system
to double check, @quick-translator-54728 are you using an M1?
q

quick-translator-54728

08/26/2021, 8:28 PM
nope! intel
h

hundreds-father-404

08/26/2021, 8:32 PM
Cool, that rules out one possibility
q

quick-translator-54728

08/26/2021, 8:35 PM
am i like
using the wrong version of python
h

hundreds-father-404

08/26/2021, 8:37 PM
I was wondering that but I don't think so because its says
python3.6
in the stacktrace, which is valid Hm another wildcard. What if you try using Pants 1.30.4? I think we may have improved the plugin resolution and I'm curious if it makes a difference
c

cold-journalist-66984

08/26/2021, 8:42 PM
I’m not sure we’ll be able to upgrade to pants 1.30.4 across the org easily, so I know that probably can’t be the end solution for us today 😕
I think @quick-translator-54728 is going to try it locally for the feedback though
h

hundreds-father-404

08/26/2021, 8:44 PM
Makes sense not being able to upgrade right now - but yeah, might give useful info to help debug what's going on
q

quick-translator-54728

08/26/2021, 8:44 PM
how do I upgrade to 1.30.4?
h

hundreds-father-404

08/26/2021, 8:45 PM
Change
pants_version
to
1.30.4
in
pants.toml
, then run
./pants --version
. However, to catch deprecation warnings, it can be helpful to jump one version at a time like 1.26 to 1.27 and so on. See https://www.pantsbuild.org/docs/upgrade-tips
Another unsatisfying workaround I'm wondering about - @quick-translator-54728 are you using Scrooge in the work you *directly do? If not, we can disable Scrooge for you locally while keeping it active for CI and everyone else Very unsatisfying, only trying to brainstorm some options for y'all while we try to figure this out
q

quick-translator-54728

08/26/2021, 8:47 PM
I think we do? I could be wrong though, @cold-journalist-66984?
okay so I upgraded to 1.30.4 and
c

cold-journalist-66984

08/26/2021, 8:50 PM
I think we do use scrooge
h

hundreds-father-404

08/26/2021, 8:50 PM
Oh that's great! It looks like using a newer version of Pants fixes things for some reason! So, next step is to try to find the minimum upgrade you need to make. Try 1.29 and see if it works, then 1.28 and so on
I think we do use scrooge
Yeah it looks like your org uses it. I'm wondering if Kati in particular will need it, or if it's only a portion of the codebase that uses it and Kati won't touch that part of the project But irrelevant if upgrading Pants can be a solution
q

quick-translator-54728

08/26/2021, 8:52 PM
ah 1.29 doesn’t work LOL
😢 1
so i think 1.30.4 is the minimum upgrade
h

hundreds-father-404

08/26/2021, 8:53 PM
Okay, yeah, I think unfortunately either the 1.30.4 upgrade or disabling Scrooge locally for Kati if you identify you won't need it in your daily tasks
c

cold-journalist-66984

08/26/2021, 8:53 PM
@hundreds-father-404 Do you know where that list of versions that ends at 1.7.0 comes from?
Kati and I have nearly identical machines and I think it will be easier from our end to fix that difference so it works for her rather than dealing with the politics of upgrading pants
h

hundreds-father-404

08/26/2021, 8:55 PM
You can see all released versions of Pants at https://pypi.org/project/pantsbuild.pants/#history. If you click on a particular release, you can go to the "download files" tab at https://pypi.org/project/pantsbuild.pants/1.7.0/#files
c

cold-journalist-66984

08/26/2021, 8:56 PM
Does pants 1.26 look there to generate that list or does it cache it somewhere?
I’m wondering if it’s cached locally somewhere on Kati’s machine that we haven’t been able to find
We’ve cleared ~/.cache/pants and ~/PROJECT/.pants.d, but I’m wondering if her python environment could have cached those values somewhere, or maybe another pants project is interfering with that list?
h

hundreds-father-404

08/26/2021, 8:58 PM
Pants's plugin resolution is using Pex to download
pantsbuild.pants.contrib.scrooge
. Under the hood, Pex is using pip
c

cold-journalist-66984

08/26/2021, 8:59 PM
is pex using the pip executable associated with the current python environemnt?
or does it have it’s own?
h

hundreds-father-404

08/26/2021, 8:59 PM
gtg run to doctor, @witty-crayon-22786 if you have a moment do you remember how the plugin resolver works in 1.26? I confirmed in changelog that Pants 1.26 is using Pex 2.1, so it's using pip under the hood I suspect the vendored version of pip is not recognizing macOS Big Sur properly because Big Sur changed its naming scheme to 11.0, rather than 10.x. But that's just a hunch. also cc @enough-analyst-54434 if you're around
c

cold-journalist-66984

08/26/2021, 9:03 PM
@hundreds-father-404 thanks for your help!
❤️ 1
h

hundreds-father-404

08/26/2021, 9:05 PM
John what version of macos are you?
c

cold-journalist-66984

08/26/2021, 9:06 PM
Big Sur 11.4 (20F71)
🤔 1
👍 1
h

hundreds-father-404

08/26/2021, 9:08 PM
Which rules out my macos hunch. Another idea is to download Pex 2.1 and try installing Pex that Scrooge plugin via Pex. If that fails, we will have isolated the issue down to Pex and removed the variable of pants (On phone, don't have the specific command to run)
c

cold-journalist-66984

08/26/2021, 9:09 PM
is there a straighforward way to check which version of pex I’m using on my machine?
e

enough-analyst-54434

08/26/2021, 9:13 PM
I'm catching up here. What version of Pants are we talking about at this point in the thread? Is it 1.30.4, which appears to work - or something else?
q

quick-translator-54728

08/26/2021, 9:14 PM
I’m using 1.30.4 at this very moment in time which gets rid of that scrooge error when i run
./pants --version
, but it doesn’t get rid of the intellij issue
c

cold-journalist-66984

08/26/2021, 9:14 PM
We’re using 1.26 and in my org upgrading would take a lot of socialization, so it’s not a viable solution to upgrade for 1.30.4 for 1 person 😕
e

enough-analyst-54434

08/26/2021, 9:15 PM
Gotcha.
Just a sec.
So, @cold-journalist-66984 and / or @quick-translator-54728 can you try:
Copy code
python3,6 -mvenv pex-test.venv
pex-test.venv/bin/pip install pex==2.1.7
pex-test.venv/bin/pex pantsbuild.pants.contrib.scrooge==1.26.0 -oplugin.pex
That will confirm this is a Pex error. We can drill from there to see if this is solvable. I expect Eric was right though about too new macOS not working with this old a Pex.
👍 1
q

quick-translator-54728

08/26/2021, 9:26 PM
Screen Shot 2021-08-26 at 2.26.19 PM.png
e

enough-analyst-54434

08/26/2021, 9:31 PM
Ok, great - thanks. Now:
Copy code
pex-test/bin/pip install pip==20.0.1
pex-test/bin/pip debug -v
We have just the 1 macos wheel on PyPI: https://files.pythonhosted.org/packages/6c/16/1798b61dfc07ea474adbd3eb64183bcd18d4040961[…]pantsbuild.pants-1.26.0-cp36-abi3-macosx_10_11_x86_64.whl and that wheel tag needs to be in the list you'll see in the pip debug output. Presumably its not.
👍 1
Specifically,
cp36-abi3-macosx_10_11_x86_64
must be in the list generated by
pip debug -v
.
Or you can cut to the chase and try:
Copy code
pex-test/bin/pip install pip==20.0.1
pex-test/bin/pip install pantsbuild.pants==1.26.0
That should fail.
Assuming it does fail, the only workaround I can think of not involving a Pants upgrade is to download the macOS wheel manually, edit it to have a wheel tag that works and then put that wheel on a company server and add the server to the
[pants-repos] repos
list. We can work through this though with a local example 1st to make sure it works for you before going down the road. If it's hard to socialize a Pants upgrade I assume its also hard to get ops to prop up a custom web server and let you add a file to it?
1
The wheel tag edit thing generally should work since mac tends to be backward compatible at the binary level even when they might thrash naming schemes up at the top level.
c

cold-journalist-66984

08/27/2021, 1:57 AM
I assume its also hard to get ops to prop up a custom web server and let you add a file to it?
It depends. We could certainly put it in a s3 bucket. If we need a custom web server that might be ok. The big thing is the impact to other devs would be minimal, but historically upgrading pants has been a huge effort here.
h

hundreds-father-404

08/27/2021, 1:58 AM
but historically upgrading pants has been a huge effort here.
Hm, what have been the pain points?
c

cold-journalist-66984

08/27/2021, 2:04 AM
I think we have a lot of tools that are tightly coupled with pants. I’m looking through old PRs and there were several done to migrate from 1.16 to 1.17 across 65 files. Maybe 1k lines of change
👀 1
h

hundreds-father-404

08/27/2021, 2:04 AM
Oh was that the Python 3 migration, needing to rewrite your plugins to use Py3?
c

cold-journalist-66984

08/27/2021, 2:07 AM
Errr, we might have had to rewrite some plugins, but they live in a separate repo.
This was all build files, tools and strangely some application code
I’m not sure if other changes were coupled in there, the commit messages are a little vague
👍 1
h

hundreds-father-404

08/27/2021, 2:10 AM
I would hope it wouldn't require 1k lines. We try to make upgrades manageable, e.g the deprecation policy. But ack that your org isn't very interested in it
c

cold-journalist-66984

08/27/2021, 2:37 AM
Hmmm, the org is showing an interest in handling our tech debt better. I’ll see if I can bring this up as a potential pain point in the future. I’m not sure we have any eyes on improving our capability to upgrade pants right now, but we probably should have.
🙌 1
h

hundreds-father-404

08/27/2021, 2:38 AM
Remind me, are y'all using Pants for mostly JVM? Or other languages like Python too?
c

cold-journalist-66984

08/27/2021, 4:33 AM
We are using it with python as well
q

quick-translator-54728

08/27/2021, 5:25 PM
yeah i can’t run that above command, there’s no directory
e

enough-analyst-54434

08/27/2021, 5:33 PM
@quick-translator-54728 so then how did you get you results here: https://pantsbuild.slack.com/archives/C046T6T9U/p1630013187031700?thread_ts=1629901353.351400&amp;cid=C046T6T9U For the suggested test here: https://pantsbuild.slack.com/archives/C046T6T9U/p1630013009031500?thread_ts=1629901353.351400&amp;cid=C046T6T9U That too required `pex-test/bin/pip`:
Copy code
python3.6 -mvenv pex-test.venv
pex-test.venv/bin/pip install pex==2.1.7
pex-test.venv/bin/pex pantsbuild.pants.contrib.scrooge==1.26.0 -oplugin.pex
Thanks for plugging along on the debug back-forth. I know its painful.
q

quick-translator-54728

08/27/2021, 5:35 PM
that’s a really good question LOL
no, thank you for trying to help out so much
is it because it’s
pex-test.venv/bin/pip
vs
pex-test/bin/pip
?
e

enough-analyst-54434

08/27/2021, 5:48 PM
Ah - sorry. If you can adjust to that then and let me know the results.
q

quick-translator-54728

08/27/2021, 5:58 PM
Screen Shot 2021-08-27 at 10.58.43 AM.png
😢
e

enough-analyst-54434

08/27/2021, 6:44 PM
Thanks @quick-translator-54728 - that's actually good news in that its expected. Everything is working as expected, not as you want - hah! Ok, so we'd need to try the wheel trick which requires a web server or S3 bucket with indexing mode turned on.
👍 1
I can give more details after you or @cold-journalist-66984 confirms its worth going down this road.
c

cold-journalist-66984

08/27/2021, 7:04 PM
I think that should be fine
q

quick-translator-54728

08/27/2021, 7:29 PM
is the wheel trick synonymous to building pants on my laptop
e

enough-analyst-54434

08/31/2021, 10:58 PM
Please excuse the big delay. Yes. The re-naming trick would just save you the build since the existing wheel should work fine and its just an artificial naming issue getting in the way.
q

quick-translator-54728

09/01/2021, 4:55 PM
it’s totally fine!! I THINK I build pants for my laptop and it’s working now
🚀 1