https://pantsbuild.org/ logo
Join Slack
Powered by
Hi guys, I am using version `2.15.0` and am havin...
# general
a
Hi guys, I am using version 
2.15.0
and am having a problem with 
pants test ::
where it is just stuck at following:
Copy code
59.94s Building requirements.pex with 11 requirements:
⠉ 59.84s Building requirements.pex with 7 requirements:
⠉ 59.81s Building requirements.pex with 9 requirements:
⠉ 59.78s Building requirements.pex with 8 requirements:
⠉ 59.88s Building requirements.pex with 14 requirements:
I know for sure that it has something to do with some package/s not being found and it keeps on trying to build the pex before running tests. I say this because I had encountered this issue once before where I made some changes to my code and requirements.txt and it was trying to find a class not present in one of the requirements. I was able to figure the issue out at that time but right now there has been no change to the codebase nor to the requirements.txt file and that makes it very difficult to pin point the cause. Since there is no change, I tried 
-ldebug
, 
--print-stacktrace
and 
--keep-sandboxes
but nothing gives me any helpful info. 
--keep-sandboxes
saves 
pip.log
files but there is nothing in those files that suggest anything. I’d really appreciate if I get some help in getting some directions where to look at. NOTE: I redacted the packages from above console output because those were enterprise packages (not the only kind of packages in the repo though).
UPDATE: I was able to proceed with the tests (although all of them failed) using the option PANTS_PYTHON_INFER_IMPORTS=False. This suggests that there is something getting inferred from import statements that’s preventing requirements.pex from getting built.
h
Are you using a lockfile?
a
Only the default ones
Is there a way by which I can see what all dependencies are being inferred? And what all is going behind the scenes where it is trying to create the requirements.pex.
Actually, 
PANTS_PYTHON_INFER_IMPORTS=False
skipped the building of requirements.pex step and that's why it proceeded. There's definitely something wrong with one or more of the dependencies and I wish there was some way of looking at inferred dependency graph - which import statement is bringing which package.
h
This is where generating a lockfile for your requirements will be really helpful
Whether it succeeds or fails
a
Ok, I have a couple of questions: 1. Does Pants infer dependencies of dependencies from import statements in the dependencies? 2. Do we need to mention dependencies of dependencies in lock files? 3. How different is pinning down each dependency in requirements.txt from creating a lock file? 4. Any docs to follow to create a lock file?
For now, is there a faster soln to overcome the problem other than creating a lock file?
I quickly skimmed thru the Lockfiles section of docs but I assume that's useful only after my current problem is solved? Because the lock file that will be generated now will give me no idea about what changed between now and before when everything was working fine? Also, it might not even generate.
e
You say "now and before" but the original post only mentions Pants 2.15.0 - perhaps you need to provide more detail, more densely to get a good response.
a
Now - when I am facing the issue I described Before - when there was no problem Note: There has been no change in the codebase whatsoever.
e
And no change in Pants version?
a
No
e
Ok, well @happy-kitchen-89482 is right, you want a lock. Your codebase doesn't change but the outside world does. Without a lockfile a solution yesterday can fail today.
Even when you pin all your deps, transitives usually float.
a
Yeah, I kind of get that thru the docs. But isn't this a catch 22 situation at the moment as 
generate-lockfiles
wouldn't work due to the same reason that requirements.pex is not building?
e
It will take more time to guess than to try.
a
I already tried but the 
generate-lockfile
goal is stuck in the same way as building requirements.pex.
e
So ... Pants defaults to Pip from 2020, you want a newer Pip. You also want to confirm you have interpreter constraints configured and that they are not overly broad.
To use more modern Pip you must be on Pants 2.16 or newer: https://www.pantsbuild.org/docs/reference-python#pip_version
a
Ok. I should upgrade. I have it thru brew and believe the usual brew upgrade command would work?
e
No
a
Regarding interpreter constraints, I have declared that as ==3.7.*
e
In 
pants.toml
you edit:
Copy code
[GLOBAL]
pants_version = "2.16.0"
👍 1
Your interpreter constraint looks good.
👍 1
a
Pants upgraded but building requirements.pex continues perpetually.
e
So, that's with?:
Copy code
[python]
pip_version = "23.0.1"
a
No, I did not do that. I thought upgraded pants will bring in the upgraded pip. Just ran test again after adding pip version but it's the same.
e
Upgrading Pants does not upgrade Pip. At least not until 2.17. You must select the version manually.
And your response is too quick to say it doesn't work. Some locks can take 5 minutes or so to generate.
Give it 15 minutes to be safe, then report back. Also, what is your machine? Is it Mac arm?
👍 1
a
Yes, it's ARM
e
Ok, those are generally horrible since ~no one publishes wheels for Mac arm, which means you're stuck building sdists, which is slow.
a
Ok, it's just that it all used to work before on the same machine without any delays or issues. Is there a way to figure out where is the process of building requirements.pex stuck at? To pinpoint the dependency it's not able to resolve.
e
If you're now locking, you're at a step prior to any Pex building. Please wait out the 15 minutes and report back on lock success or not. We can proceed from there.
👍 1
If you don't mind pasting your 
pants.toml
, that would help build a shared basis as well.
a
Lockfile got generated successfully.
e
Ok, great.
a
Actually, pants test also worked successfully. Could it be only because of pip version?
e
Yes, that's why I had you try that. But you really really really want to keep the lock.
a
Yeah, I got that. So, keeping a lock will make sure that I am isolated from all the changes in the universe around me? And I just need to generate them again whenever I make a change to my requirements.txt?
e
Correct. In fact, pants should error if you edit requirements.txt but do not update the lock.
Should be foolproof.
a
Ok, so update the lock is always manual. It's not like I update the requirements.txt and then run pants test without generating lockfile and it would be updated automatically?
e
Correct. It should be automatic, but Pants can be infuriating in some regards. This is one thing it doesn't do for you for no awesome reason. So manual it is.
a
That's great that it's not automatic so that I am aware of when it was changed.
e
Well, you are because the lock changes anyhow and you check that in. So it's super obvious in the diff.
a
Last thing - I can commit the lockfile and it will be used in CI?
e
You don't need to make excuses for Pants here. It's just not good in this regard. Plenty of other tools do the right thing automatically like cargo in the rust world.
😃 1
Yes, you should definitely commit the lockfile. Other users (including CI) will all be spared the resolve time you just went through to create the lock.
👍 1
a
We saw that this was ultimately a pip version issue and I have a question about that. Would this same issue not have occurred had I used lock files?
e
That's pretty impossible to answer without the full explanation of the 1st sentence. You'd need to lay out the proof / exact inputs, the resulting issue, and how a different Pip version changed the situation.
Basically, if and only if you can provide a public repro of the before / after can I be helpful answering. The details really matter here.
a
As I have mentioned before, there's no before and after for the project I was having the problem with. There was no change at all and 
pants test ::
started failing. And then it worked when I upgraded the pip version to 
23.0.1
and had to upgrade Pants too because 
2.15.0
doesn't support that pip version. My best guess regarding the situation is that probably something changed in some dependency or transitive dependency somewhere and pip's old version had problems with that. I have had such issues in the past in the same project (before we started using Pants) in a 
venv
setup that 
pip install -r requirements.txt
failed with an older version of pip and it went fine after upgrading the pip. I think I should be able to demonstrate that and we can see what exactly the error was as it's been long and I don't recollect it. With all the above information: 1. Could lock files have helped averting the issue (and saved one full work day wondering what's wrong)? 2. As I said that I have had issues with an older pip version where it failed and threw some error that I don't recall, wouldn't it be nice for Pants to throw some user friendly error or propagate error from pip rather just keep trying building requirements.pex. One time I let it run for 25000+ seconds to see if it completes or throw some error but finally, I had to cancel it.
e
My best guess regarding the situation is that probably something changed in some dependency or transitive dependency somewhere and pip's old version had problems with that.
Yes, exactly; so the details matter. I would need to have an open source version of your problem with the exact requirements set you use, etc. Words will not be useful here. The exact details will.
I'll be happy to drill down on the real details if you can provide them in this way.
I think your questions are simply off base.
1. Could lock files have helped ...
It really doesn't matter. Maybe? But there is no reason whatsoever I've ever been made aware of to not use them. So just continue using them, they are good for a host of reasons security / reproducibility and perf related.
2. As I said that I have had issues with an older pip version where it failed and threw some error that I don't recall, wouldn't it be nice for Pants to throw some user friendly error or propagate error from pip rather just keep trying building requirements.pex...
That's the problem - there is no error, old Pip is likely backtracking and resolving very slow. This is where those details would matter to prove this. The solution is to use new Pip like you are now and continue to use new Pip, bumping the version as Pants supports newer versions.
a
The command 
pants generate-lockfiles
is continuing to run today for the same 
pants.toml
, same 
pip_version
, same pants version and the same 
requirements.txt
. Its been 1300+ seconds ☹️
h
The next step is to run 
generate-lockfiles
with 
--keep-sandboxes=always
, inspect the sandbox that the lockfile generation is running in, and look at the 
__run.sh
command there (which will call the pex command line interface) to see if anything pops out at you.
For example, are you setting the interpreter constraints appropriately?
a
Yes, the interpreter constraints are the same as the day it got generated: https://pantsbuild.slack.com/archives/C046T6T9U/p1691110153705439?thread_ts=1691090515.850559&cid=C046T6T9U. There is only one lockfile which is the default one and so no override for interpreter constraints are declared for it separately using this: https://www.pantsbuild.org/docs/reference-python#resolves_to_interpreter_constraints
e
@ancient-rose-27306 my advice to provide real details still applies. Please make the relevant data to repro available.
a
Unfortunately, I can’t share the repo, not even 
pants.toml
(probably masked). Definitely not 
requirements.txt
. I shall try again with @happy-kitchen-89482’s suggestion and see what findings I can report.
@happy-kitchen-89482, Good Morning! I inspected the 
pants-sandbox-
directory that got created with 
--keep-sanboxes=true
but there is no 
__run.sh
in the entire directory structure.
2 Views
Open in Slack
PreviousNext
Powered by