https://pantsbuild.org/ logo
Join Slack
Powered by
Hey folks, Do you have any suggestions on how to ...
# general
p
Hey folks, Do you have any suggestions on how to get generate-lockfiles to work for some python deps on osx when there is a dependency that is linux only?
the error we get is
Copy code
pip: ERROR: Could not find a version that satisfies the requirement cupy-cuda-12x (from versions: none)
 pip: ERROR: No matching distribution found for cupy-cuda-12x
got this when doing pants generate-lockfiles --resolve=python-default .
r
Maybe use pip markers for platform?
p
I still need these to show up in the lockfiles pants generates, using sys_platform seems to make them entirely disappear
Is there a way to have pants run generate-lockfiles in a docker environment?
r
I still need these to show up in the lockfiles pants generates, using sys_platform seems to make them entirely disappear
They disappear completely from generate lockfile? I would imaging the lockfile to show entry for non osx platform?
p
I'm on linux so I'm sort of hacking this up, but yeah, when I add a sys_platform == "win32" to one of our deps, it disappears entirely from the lockfile
😢 1
r
Is there a way to have pants run generate-lockfiles in a docker environment?
I read a while ago some people here were running pants inside docker locally. Just search the slack you might find something
p
yeah, we have it working for the rest of the repo, but lockfile generation seems to not be supported; I guess this is a known issue https://github.com/pantsbuild/pants/issues/19872
c
What requirements are not working exactly? I use for example:
Copy code
python_requirement(
    name="tensorflow",
    requirements=[
        "tensorflow==2.18.0",
        "tensorflow-metal==1.1.0; sys_platform == 'darwin' and platform_machine == 'arm64'",
    ],
)
p
I tried
Copy code
skypilot; sys_playform == "win32"
in my testing in practice the actual package we want to lock on osx is 
cupy-cuda-12x
I am currently trying to run pants as an adhoc_tool so that it can be run inside docker, but I am running into this error:
Copy code
No Pants configuration was found at or above /tmp/pants-sandbox-glKvVS.
Install failed: EOF when reading a line
More information can be found in the log at: /home/kuza/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/logs/configure.log

Error: Failed to establish atomic directory /home/kuza/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/locks/configure-20a10037193484fb32224fbcd6b4ec6177dc21062ec6af6f1ee47f0969330780. Population of work directory failed: Boot binding command failed: exit status: 1
from this BUILD
Copy code
adhoc_tool(
    name="pants-lockfiles",
    runnable=":pants_bin",
    args=["--no-pantsd", "generate-lockfiles"],
    log_output=True,
    workdir="/",
    output_directories=["3rdparty/"],
    execution_dependencies=["//:entire-repo"],
    environment="linux",
)

system_binary(
    name="pants_bin",
    binary_name="pants",
    extra_search_paths=["/home/kuza/bin/"],
)
And this command
Copy code
pants --keep-sandboxes=always export-codegen tools:pants-lockfiles
I thought it might be a problem with my execution_deps, but when I run 
ls
as the command instead I do see my entire repo
@curved-manchester-66006 if you lock that outside osx, is metal in the lockfile?
c
Copy code
{
          "artifacts": [
            {
              "algorithm": "sha256",
              "hash": "1c365669fc9d764edcc1773254a477db7096975b1bff58fb37a11d7b8ca4295f",
              "url": "<https://files.pythonhosted.org/packages/52/56/8373f5751011304a346f07e5423e69f809b626989d2541ae9e816ae7ced2/tensorflow_metal-1.1.0-cp311-cp311-macosx_12_0_arm64.whl>"
            }
          ],
          "project_name": "tensorflow-metal",
          "requires_dists": [
            "six>=1.15.0",
            "wheel~=0.35"
          ],
          "requires_python": null,
          "version": "1.1.0"
        },
        {
          "artifacts": [
            {
              "algorithm": "sha256",
              "hash": "8c1774683e38359dbcaf3697e79b7e6a4e69b9c7b3679e78ee18f43e59e5759b",
              "url": "<https://files.pythonhosted.org/packages/6c/c9/93e9e34703269d5ec3f6a8f49f6748df8737fede8bb1f34165f6ef3d861b/tensorflow_probability-0.24.0-py2.py3-none-any.whl>"
            }
          ],
          "project_name": "tensorflow-probability",
          "requires_dists": [
            "absl-py",
            "cloudpickle>=1.3",
            "decorator",
            "dm-tree",
            "gast>=0.3.2",
            "jax; extra == \"jax\"",
            "jaxlib; extra == \"jax\"",
            "numpy>=1.13.3",
            "six>=1.10.0",
            "tensorflow-datasets>=2.2.0; extra == \"tfds\"",
            "tensorflow>=2.16; extra == \"tf\"",
            "tf-keras>=2.16; extra == \"tf\""
          ],
          "requires_python": ">=3.9",
          "version": "0.24.0"
        },
6 Views
Open in Slack
PreviousNext
Powered by