Question: We have a dependency on tensorflow. Unti...
# general
r
Question: We have a dependency on tensorflow. Until now, we have been using the
tensorflow
package from linux development machines. To support development on m1 mac, we need to use the
tensorflow-macos
package, with the module mapping
"tensorflow-macos": ["tensorflow"]
. This causes ambiguity for tensorflow dependency inference. Any advice on how to handle this? Thanks!
Copy code
The target src/python/path/to/model.py:../../../model_project imports `tensorflow.keras.layers.LSTM`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['3rdparty/python#tensorflow', '3rdparty/python#tensorflow-macos'].
c
Hi Dave, I’m not exactly sure, my guess would be to try with multiple resolves.. maybe? (there may be better ways)
r
Multiple resolves does fix the immediate target ambiguity issue. But, what I am then struggling with is that one resolve is compatible only with a Mac environment while the other is compatible with Linux. If I parameterize the resolve of this python_source target to include both resolves, then I will end up with an incompatible resolve in either environment. I can probably do something like create multiple targets for any dependees of this python_source, and use tags to exclude those targets which are not compatible with the developers environment. A better approach might be to parametrize the resolve for the dependee targets, but make this parametrization environment-aware such that it excludes any resolves which are incompatible with the local environment. Not sure if this is possible. Maybe I can try a macro
c
Yeah, I’m a bit hazy on all the details here..
r
I’ll let you know how it goes 🙂
🙏 1
h
i had this question before! unfortunately it’s hidden in slack history. this is what we have in our `pyproject.toml`:
Copy code
[tool.poetry.dependencies]
tensorflow = { version = "2.7.0", markers = "sys.platform != 'darwin' or platform_machine != 'arm64'" }
tensorflow-macos = { version = "2.7.0", markers = "sys.platform == 'darwin' and platform_machine == 'arm64'" }
this is what we have in our root `BUILD`:
Copy code
poetry_requirements(
    ...
    overrides={
        "tensorflow": {
            "dependencies": [":tensorflow-macos"],
        },
    },
)
r
Thanks! My current working version is this:
Copy code
python_requirement(
    name="tensorflow",
    requirements=[
        'tensorflow == 2.7.0; sys_platform != "darwin"',
        'tensorflow-macos == 2.7.0;   sys_platform == "darwin"'
    ],
)
👍 1
( I should add the arm markers - funny that tensorflow has the x86 mac artifacts instead of tensorflow-macos
Haven't updated here because I ran into lots of other fun dependency issues with arm mac. (not pants/pex related)