https://pantsbuild.org/ logo
#general
Title
# general
c

curved-wall-59116

12/10/2021, 10:51 AM
Dear Pants community. My team and i use
mypy==0.91
for type checking. When my colleague runs
./pants typecheck <projects/project>
it works pretty well. When i run it, it will solve the constraints.txt file, and it takes 25+ mins. How come it runs so slowly on my computer?
c

curved-television-6568

12/10/2021, 10:58 AM
That is a considerable amount of time. I assume you have similarly spec’ed machines, so it would perhaps come down to configuration. Do you have any custom config for pip or pants in your home directory, perhaps? Besides that nudge, I think I must defer to the experts, when they wake up 🙂
c

curved-wall-59116

12/10/2021, 11:18 AM
Yes, his might be a little faster, but not enough to justify this difference 🙂 We have a setup script that we run as
Copy code
#!/usr/bin/env bash

set -euo pipefail

# You can change these constants.
PYTHON_BIN=python3.8
VIRTUALENV=.venv
PIP="${VIRTUALENV}/bin/pip"
CONSTRAINTS_FILE=constraints.txt

"${PYTHON_BIN}" -m venv "${VIRTUALENV}"

source .venv/bin/activate

echo "Installing pip"
"${PIP}" install pip --upgrade
echo "Installing poetry"
"${PIP}" install poetry
# Install all our requirements.txt, and also any 3rdparty
# dependencies specified outside requirements.txt, e.g. via a
# handwritten python_requirement_library target.
echo "Poetry update"
poetry update
echo "Installing dependencies"
"${PIP}" install -r <(poetry export --dev --without-hashes) -r <(./pants dependencies --type=3rdparty ::)
echo "Remove lockfile"
rm poetry.lock
echo "Generating constraints.txt"
rm constraints.txt
#pip uninstall pkg-resources==0.0.0 -y
pip list --format freeze  >> constraints.txt
And this should be the same for everyone using our monorepo
m

modern-wolf-36228

12/10/2021, 11:25 AM
so you have the pkg-resources thing as well 🤔 what's that?
b

brash-baker-91190

12/10/2021, 11:44 AM
We have it too (though I've seen it seemingly flip between
pkg-resources
and
pkg_resources
🤔 )... https://stackoverflow.com/questions/39577984/what-is-pkg-resources-0-0-0-in-output-of-pip-freeze-command
🪄 1
c

curved-wall-59116

12/10/2021, 12:04 PM
Yes we have it too, and it seems like pants use it for repository structure, so it can not be uninstalled like i have shown above 😄
b

brash-baker-91190

12/10/2021, 1:54 PM
It's not a Pants thing. We use this script to manage our virtual environment (https://github.com/grapl-security/grapl/blob/main/build-support/manage_virtualenv.sh) and there's only a single pants command in it (
./pants dependencies --type=3rdparty ::
), which doesn't output anything about
pkg-resources
. We end up just filtering that out with a tasteful `grep -v`: https://github.com/grapl-security/grapl/blob/04f6869f723b353471f466c843a3f9c33e83171e/build-support/manage_virtualenv.sh#L59-L61
🙌 1
(an approach swiped from that StackOverflow link)
h

happy-kitchen-89482

12/10/2021, 2:55 PM
Does it do that 25 minute resolve every single time?
I can see it needing to do that once after a constraints file change, of course, but I assume that's not what you mean
Which pants version is this btw?
And also, one thing to try to help debug:
• Run that resolve all the way through once so we know its result is up to date and cached
• Run the same command again, and observe the resolve running when it clearly shouldn't be, and then ctrl-C out in the middle of the resolve
• Run that command a third time, and see if the resolve runs again or not
We've seen weird cases like that, would be good to see if this is related
c

curved-wall-59116

12/10/2021, 3:41 PM
We work with
pants 2.7
And when pants has cached the constraints the constraints solving is fast. However if i rename a file, add a new one or a new dependency it all has to be redone. Will try what you posted above, when it finishes solving again 🙂
h

happy-kitchen-89482

12/10/2021, 4:03 PM
Ah, ok, so I'm trying to distinguish between two different possible issues: 1) Resolves are happening when you don't expect them to 2) Resolves only happen when you expect them to, but they take longer than you'd expect I thought you were saying 1) was the problem, but now it sounds like 2)? Or even both?
Maybe I misunderstood what you wrote, but I thought you were saying that your coworkers had good build times but you didn't?
c

curved-wall-59116

12/10/2021, 4:25 PM
Yeah i would say a combination of the 2!
Some of them have good build times, but we are not sure why that is
h

happy-kitchen-89482

12/10/2021, 4:35 PM
What would you expect to be a reasonable resolve time for your constraints file? For example can you time running
pip install -r constraints.txt
in a clean virtualenv?
7 Views