https://pantsbuild.org/ logo
#general
Title
# general
w

wooden-thailand-8386

09/09/2020, 1:07 PM
Hi there, I’m trying to use local wheels and I’m not sure if
pants
is actually getting the .whl files from the folder. What I have on my toml file is:
Copy code
[python-repos]
repos.add = "%(buildroot)s/.wheels
One of the reasons I’m doing this is to try to speed up the test/bundle by not having to download any dependency from the internet (we have tensorflow and other ML libraries) and under our proxy things take forever 😞
i

important-librarian-62877

09/09/2020, 2:34 PM
I'm not sure what version this was added (1.27 maybe?) but you could try running this command on your target to see if pants is picking up your third party requirements
Copy code
./pants dependencies --type=3rdparty helloworld/util
FWIW, we have a few wheels that are picked up from a directory and this is what our toml looks like:
Copy code
[python-repos]
repos = [
    "%(buildroot)s/3rdparty/repos"
]
h

hundreds-father-404

09/09/2020, 2:39 PM
Hi! Also see the tooltip on https://www.pantsbuild.org/v2.0/docs/python-third-party-dependencies Turns out that we supported local wheels for a while via PEP 440, but didn’t realize it until last week
Also more like that repos.add still includes PyPI. You likely want Christian’s approach it only
repos
, which will override the default of PyPI.
👍 1
w

wooden-thailand-8386

09/09/2020, 2:45 PM
@important-librarian-62877 it is picking up the dependencies 🙂
Copy code
nltk==3.4.5
numpy==1.19.1
python-json-logger==0.1.11
rasa==1.10.2
scipy==1.4.1
tensorflow==2.1.1
tensorflow-text==2.1.0rc0
and your approach on using
repos =
instead of
repos.add
made it faster.
💯 1
another issue that I’m having is with the
constraints.txt
file. For some reason it seems like it’s not using it. I always have a couple of libs breaking the build process such as:
Copy code
pex.resolver.Unsatisfiable: Failed to resolve compatible distributions:

1: rasa==1.10.2 requires coloredlogs<11.0,>=10.0 but coloredlogs 14.0 was resolved
but I have the
constraints.txt
file set.. my
requirements.txt
file doesnt have everything that’s inside
constraints.txt
only the libs that I add to my
python_requirements
h

hundreds-father-404

09/09/2020, 2:55 PM
Are you using the v2 engine or v1 engine? That feature only works with v2 unfortunately
w

wooden-thailand-8386

09/09/2020, 2:56 PM
Noooooo 😢
I feel bad about asking this but any prospect on that wonderful CA SSL thing being resolved? Is there anything that I can do to help?
h

hundreds-father-404

09/09/2020, 2:58 PM
Hm cc @happy-kitchen-89482, do you have a chance to either look today, or point Thales or me in the right direction? Idk if you started taking a look yet.
I was going to work on https://github.com/pantsbuild/pants/issues/10746 tomorrow, but I’ll switch to it today. I’m not sure if that would fix your issue - it would at least let the no proxy workaround apply
w

wooden-thailand-8386

09/09/2020, 3:03 PM
When I try using the v2 it can’t even download the pex binary when it’s bootstraping. It doenst trust the source because of the self-signed cert that we have to use, that’s why we need to set up the
CURL_CA_BUNDLE
,
REQUESTS_CA_BUNDLE
and others to point to our cert.
😕 1
👍 1
The thing is that Rust doesnt seem to have any env variable to fix that so.. 😞
h

hundreds-father-404

09/09/2020, 3:09 PM
The thing is that Rust doesnt seem to have any env variable to fix that so.. 😞
I was going to say that we can pass any arbitrary env var. But I’m not certain if Rust would recognize it. I haven’t checked. One thing that would help a lot is if you know of some way for us to simulate your proxy and test that this works. Of course, we can’t use your actual proxy, but I’m wondering if there’s some type of config we could use to simulate the environment.
w

wooden-thailand-8386

09/09/2020, 3:22 PM
I’ll see if I can find something to make this environment reproducible. So far I found this
mock-server
tool that might do it. I’ll try to set up that for you.
h

hundreds-father-404

09/09/2020, 3:29 PM
Thank you!
2 Views