generate-lockfiles

and the curse of

setup.py

Hello team, I think I hit a corner case that no one did previously (based on my Slack search), but I believe it is worth bubbling up. I am basically trying to put

flash-attn

package into our repo and generate a lockfile. It fails with:

/pip-download-p42iufrh/flash-attn_0dfa01adf4c743469f2b404b4e4a39d3/setup.py", line 22, in <module>
pip:        import torch
pip:    ModuleNotFoundError: No module named 'torch'
pip:    ERROR: python setup.py egg_info exited with 1

Because in their setup.py they import torch. Is there a way to specify the environment to run the

generate-lockfiles

command? Ideally from the

requirements.txt

itself and then generate the lockfile with that temporary virtual env?

I'm not sure off hand how this is intended to work even without pants.

torch

imported but not in

setup_requires

? One way to unblock would be to build the needed wheels out of bands and then upload them to a local index

Could you expand on how the local index work? Is there any documentation I can read about it? Thank you!

https://packaging.python.org/en/latest/guides/hosting-your-own-index/ Or a variety of commercial offerings (ex: GitLab, sonatype, AWS, jfrog, buildkite)

I think that is unrelated. It's more because Pants is trying to generate lockfiles in a clean environment where

torch

is not installed. We already have access to torch via PyPI right?

I was suggesting that if you built the needed wheels manually for

flash-attn

and placed them in an index, you could sidestep this bootstrapping problem

Ah... I see... that is true... I should ask the

flash-attn

maintainers to add

torch

in the

setup_requires

field...