One of my engineers uses the package `tensorflow-r...
# general
s
One of my engineers uses the package
tensorflow-ranking
which has a dependency on
tensforflow
. On M1 laptops, engineers install
tensforflow-macos
not
tensorflow
. We get around this by
pip
installing it with
--no-deps
. I tried checking to see if I could specify that flag in
requirements.txt
and solve this outside of
pants
but haven't found anything. Is there a way I can add
--no-deps
to
tensorflow-ranking
in
pants
?
e
Nope. Neither in Pants nor Pex. You can in Pex (
--intransitive
), but Pex, like Pants is global in that you cannot build up a venv piecemeal. You specify a complete venv all at once with both.
It's very unfortunate
tensorflow-macos
exists. In any package ecosystem having the same symbol provided by two different package names is problematic. Here it's made worse by the fact that wheel tags and environment markers are already built to handle this.
Google, like Facebook with PyTorch, is a bad citizen
w
you can make it a platform conditional requirement, i think?
e
You missed the detail I think Stu.
w
can’t remember where i saw that recently… looking.
e
The issue is a transitive dep.
w
ah, yea. i see what you’re saying.
s
This is probably just wishful thinking (and me still learning pants) but I can't do something like
Copy code
"tensorflow-ranking": {
    "dependencies": [
        "//:reqs#tensorflow"
    ]
},  
"tensorflow": {
    "requirements": (
        # Linux or intel Mac
        "tensorflow ~= 2.11.0 ; sys_platform != 'darwin' or platform_machine == 'x86_64'",
        # Mac M1
        "tensorflow-macos ~= 2.11.0 ; sys_platform == 'darwin' and platform_machine == 'arm64'",
    ),
}
in a
BUILD
file right?
e
@silly-queen-7197 it looks like that's a pure python project: https://pypi.org/project/tensorflow-ranking/#files
So just fork, edit it's deps and depend on a VCS requirement
1
s
Thanks, that's a very pragmatic solution
Hrm, good luck with that actually. I have no clue how that wheel publishes - looks like a bazel project and I cannot find the setup.py or project.toml
You are in trouble
s
I think the reality is we'll need to just move to remote development so all engineers have an x86 machine (a plus is spinning up machines with gpus too). Fortunately this is on our roadmap
Thanks for the help though
e
Developing on Linux for Linux is .... sane
c
@witty-crayon-22786 did you think of this thread? https://pantsbuild.slack.com/archives/C046T6T9U/p1676331663918329
👍 1
w
indeed, heh.